use strict;
my $str = 'Bão Yagi (được Việt Nam định danh là bão số 3, được phía Philippines đặt tên bão Enteng - tiếng Anh: Severe Tropical Storm Enteng , nguyên văn \'Bão nhiệt đới dữ dội Enteng\')
Some legal "[d]ocuments" contain corrected spel[l]ing, gram(m)ar, or simple typos; and lots of references[1]. By Extension, I included curl{e}y brackets, but not tag brack<e>ts, which are not seenin modern legal documents.
These.are.properly.separated.even.U.S.A., and multiple punctuations are properly ignored.
A. Multiple
B. Choices
-ABC-DEF-
-A-B-C-D-
-1-2-3-4-
-test-
.ABC.DEF.
.AB.CD.EF.D.
.A.B.C.D.E.
.123.456.789.
.12.34.56.78.
.1.2.3.4.
cod3 var1aBl3s
test.U.S.A.test
We\'d want hyphenated words in cases when large words are broken for wrapping in tight column news papers/megazines, while we can still properly separate numbers such as "30-35 pages".
Non-Latin character are separated per character:
出典: フリー百科事典『ウィキペディア(Wikipedia)』
ウィキペディアには現在この名前の項目はありません。
Acceptable failed cases:
- [E]xpected: "[" is considered external "wrapper/enclosure", while the internal "wrappers" are included so that they can be further processed/removed in the future.
- test.U.S.A.test: this happens when no space trailing textContents of a block-level elemetn in HTML files.
- 『ウィキペディア(Wikipedia)』: mixed languages
- CamelCasing s not separated, but can be easily separated in "post-processing step, even though _snake_casing_ works fine by happenstance.
-cod3 var1aBl3s: mixing letters and numbers. It\'s not intentional, but not a big deal when the side-effect is new word is always started with a number.';
my $regex = qr/(?:[^\p{Script=Latin}\s\w;:.,\-[\](){}'"+\/=<>])\B|(?:(?:[\p{Script=Latin}]|[^\s\w;:.,\-[\](){}'"+\/=<>]){2,}(?:(?:[\p{Script=Latin}]|[^\s\w+.,–:;\/\\=<>])?(?:[\p{Script=Latin}]|[^\s\w;\-:.,\[\](){}'"+\/\\=<>]))*)|(?:(?:[0-9\p{Script=Latin}]|[^\s\w;:.,[\](){}\-'"+\/\\=<>])(?:(?:[0-9\p{Script=Latin}]|[^\s\w+\-:;\/\\=<>])?(?:[0-9\p{Script=Latin}]|[^\s\w;:.,\-[\](){}'"+\/\\=<>]))*)/muip;
if ( $str =~ /$regex/g ) {
print "Whole match is ${^MATCH} and its start/end positions can be obtained via \$-[0] and \$+[0]\n";
# print "Capture Group 1 is $1 and its start/end positions can be obtained via \$-[1] and \$+[1]\n";
# print "Capture Group 2 is $2 ... and so on\n";
}
# ${^POSTMATCH} and ${^PREMATCH} are also available with the use of '/p'
# Named capture groups can be called via $+{name}
Please keep in mind that these code samples are automatically generated and are not guaranteed to work. If you find any syntax errors, feel free to submit a bug report. For a full regex reference for Perl, please visit: http://perldoc.perl.org/perlre.html