⇢ | A | CustomHyphenationTokenizer added |
⇢ | B | CustomHyphenationTokenizer::run() added | |
⇢ | B | CustomHyphenationTokenizer::tokenize() added | |
⇢ | A | CustomHyphenationTokenizer::__construct() added | |
C | ↗ | B | PunctuationTokenizer::run() improved |
C | ↗ | B | WhitespaceTokenizer::run() improved |