edu.umass.cs.mallet.base.fst
Class MultiSegmentationEvaluator

java.lang.Object
  edu.umass.cs.mallet.base.fst.TransducerEvaluator
      edu.umass.cs.mallet.base.fst.MultiSegmentationEvaluator

public class MultiSegmentationEvaluator

extends TransducerEvaluator

Field Summary

Fields inherited from class edu.umass.cs.mallet.base.fst.TransducerEvaluator

alwaysEvaluateWhenFinished, checkpointFilePrefix, checkpointIterationsToSkip, checkpointTransducer, numIterationsToSkip, numIterationsToWait, printModelAtEnd, viterbiOutput, viterbiOutputEncoding, viterbiOutputFilePrefix, viterbiOutputIterationsToSkip, viterbiOutputIterationsToWait

Constructor Summary

MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags, java.lang.Object[] segmentContinueTags)

MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags, java.lang.Object[] segmentContinueTags, boolean showViterbi)

Method Summary

void	batchTest(InstanceList data, java.util.ArrayList predictedSequences, java.lang.String description, java.io.PrintStream viterbiOutputStream) Tests segmentation using an ArrayList of predicted Sequences instead of a Transducer.
boolean	evaluate(Transducer model, boolean finishedTraining, int iteration, boolean converged, double cost, InstanceList training, InstanceList validation, InstanceList testing) Evaluates a Tranducers on a given training, validation, and testing set.
int	numIncorrectSegments(Sequence trueOutput, Sequence predOutput) returns the number of incorrect segments in predOutput
void	test(Transducer model, InstanceList data, java.lang.String description, java.io.PrintStream viterbiOutputStream)

Methods inherited from class edu.umass.cs.mallet.base.fst.TransducerEvaluator

getCheckpointFilePrefix, getCheckpointIterationsToSkip, getCheckpointTransducer, getNumIterationsToSkip, getNumIterationsToWait, getViterbiOutput, getViterbiOutputEncoding, getViterbiOutputFilePrefix, getViterbiOutputIterationsToSkip, getViterbiOutputIterationsToWait, isAlwaysEvaluateWhenFinished, isPrintModelAtEnd, setAlwaysEvaluateWhenFinished, setCheckpointFilePrefix, setCheckpointIterationsToSkip, setCheckpointTransducer, setNumIterationsToSkip, setNumIterationsToWait, setPrintModelAtEnd, setViterbiOutput, setViterbiOutputEncoding, setViterbiOutputFilePrefix, setViterbiOutputIterationsToSkip, setViterbiOutputIterationsToWait, shouldDoEvaluate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiSegmentationEvaluator

public MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags,
                                  java.lang.Object[] segmentContinueTags,
                                  boolean showViterbi)

MultiSegmentationEvaluator

public MultiSegmentationEvaluator(java.lang.Object[] segmentStartTags,
                                  java.lang.Object[] segmentContinueTags)

Method Detail

evaluate

public boolean evaluate(Transducer model,
                        boolean finishedTraining,
                        int iteration,
                        boolean converged,
                        double cost,
                        InstanceList training,
                        InstanceList validation,
                        InstanceList testing)

Description copied from class: TransducerEvaluator

Evaluates a Tranducers on a given training, validation, and testing set. .

The default implementation calls the evaluator's test on the training, validation, and testing sets.

Overrides:

evaluate in class TransducerEvaluator

Parameters:

model - The transducer to evaluate.

finishedTraining - Whether training has finished. Evaluators can be set to always evaluate when training completes.

iteration - The current iteration

converged - Whether training has converged. Most evaluators ignore this.

cost - The current cost during training. Some evaluators may include this in the output.

training - The training set. If null, the argument is ignored.

validation - The validation set. If null, the argument is ignored.

testing - The testing set. If null, the argument is ignored.

Returns:

Whether training should continue.

test

public void test(Transducer model,
                 InstanceList data,
                 java.lang.String description,
                 java.io.PrintStream viterbiOutputStream)

Specified by:

test in class TransducerEvaluator

numIncorrectSegments

public int numIncorrectSegments(Sequence trueOutput,
                                Sequence predOutput)

returns the number of incorrect segments in predOutput

Parameters:

trueOutput - truth

predOutput - predicted

Returns:

number of incorrect segments

batchTest

public void batchTest(InstanceList data,
                      java.util.ArrayList predictedSequences,
                      java.lang.String description,
                      java.io.PrintStream viterbiOutputStream)

Tests segmentation using an ArrayList of predicted Sequences instead of a Transducer. If predictedSequence is null, don't include in stats (useful for error analysis).

Parameters:

data - list of instances to be segmented

predictedSequences - predictions

description - description of trial

viterbiOutputStream - where to print the Viterbi paths