Word Stemmer#
The Word Stemmer reduces inflected and derived words to their root form using a stemmer. For example, the sentence "Majority voting is likely foolish" might stem to "major vote is like foolish." Tokens containing delimiters such as apostrophes and hyphens have only their word segments stemmed and the delimiters are preserved, e.g. "something's" stems to "someth's" and "well-known" stems to "well-known".
Parameters#
| # | Name | Default | Type | Description |
|---|---|---|---|---|
| 1 | stemmer | PorterEnglish | Stemmer | The stemmer algorithm to reduce words to their root form. |
Example#
use Rubix\ML\Tokenizers\WordStemmer;
$tokenizer = new WordStemmer();
var_export($tokenizer->tokenize('Majority voting is likely foolish'));
// ['Major', 'vote', 'is', 'like', 'foolish']