Uses of Class
cc.mallet.pipe.SimpleTokenizer

Packages that use SimpleTokenizer

cc.mallet.pipe	Classes for processing arbitrary data into instances.
cc.mallet.util	Miscellaneous utilities including command line processing, math functions, lexing, logging.

Uses of SimpleTokenizer in cc.mallet.pipe

Methods in cc.mallet.pipe that return SimpleTokenizer

SimpleTokenizer

SimpleTokenizer.deepClone()

Methods in cc.mallet.pipe with parameters of type SimpleTokenizer

void	FeatureDocFreqPipe.addPrunedWordsToStoplist(SimpleTokenizer tokenizer, double docFrequencyCutoff) Add all pruned words to the internal stoplist of a SimpleTokenizer.
void	FeatureCountPipe.addPrunedWordsToStoplist(SimpleTokenizer tokenizer, int minimumCount) Add all pruned words to the internal stoplist of a SimpleTokenizer.

Uses of SimpleTokenizer in cc.mallet.util

Methods in cc.mallet.util with parameters of type SimpleTokenizer

static void	BulkLoader.generateStoplist(SimpleTokenizer prunedTokenizer) Read the data from inputFile, then write all the words that do not occur pruneCount.value times or more to the pruned word file.
static void	BulkLoader.writeInstanceList(SimpleTokenizer prunedTokenizer)