Separating words that handle many use cases I needed, more than simple /\bword\b/gi.
Below are some targeted test cases and acceptable failed cases:
Bão Yagi (được Việt Nam định danh là bão số 3, được phía Philippines đặt tên bão Enteng - tiếng Anh: Severe Tropical Storm Enteng , nguyên văn 'Bão nhiệt đới dữ dội Enteng')
Some legal "[d]ocuments" contain corrected spel[l]ing, gram(m)ar, or simple typos; and lots of references[1]. By Extension, I included curl{e}y brackets, but not tag brack<e>ts, which are not seenin modern legal documents.
+These.are.properly.separated.even.U.S.A., and multiple punctuations are properly ignored.
Basic patterns
A. Multiple
B. Choices
-ABC-DEF-
-A-B-C-D-
-1-2-3-4-
-test-
.ABC.DEF.
.AB.CD.EF.D.
.A.B.C.D.E.
.123.456.789.
.12.34.56.78.
.1.2.3.4.
cod3 var1aBl3s
test.U.S.A.test
We'd want hyphenated words in cases when large words are broken for wrapping in tight column news papers/megazines, while we can still properly separate numbers such as "30-35 pages".
Non-Latin character are separated per character:
出典: フリー百科事典『ウィキペディア（Wikipedia）』
ウィキペディアには現在この名前の項目はありません。
Acceptable failed cases:
[E]xpected: "[" is considered external "wrapper/enclosure", while the internal "wrappers" are included so that they can be further processed/removed in the future.
test.U.S.A.test: this happens when no space trailing textContents of a block-level elemetn in HTML files.
『ウィキペディア（Wikipedia）』: mixed languages
CamelCasing not separated, but can be easily separated in "post-processing step, even though snake_casing works fine by happenstance.
cod3 var1aBl3s: mixing letters and numbers. It's not intentional, but not a big deal when the side-effect is new word is always started with a number.