Class CRFTrainerByEntropyRegularization

  extended by cc.mallet.fst.TransducerTrainer
      extended by cc.mallet.fst.semi_supervised.CRFTrainerByEntropyRegularization
All Implemented Interfaces:

public class CRFTrainerByEntropyRegularization
extends TransducerTrainer
implements TransducerTrainer.ByOptimization

A CRF trainer that maximizes the log-likelihood plus a weighted entropy regularization term on unlabeled data. Intuitively, it aims to make the CRF's predictions on unlabeled data more confident. References: Feng Jiao, Shaojun Wang, Chi-Hoon Lee, Russell Greiner, Dale Schuurmans "Semi-supervised conditional random fields for improved sequence segmentation and labeling" ACL 2006 Gideon Mann, Andrew McCallum "Efficient Computation of Entropy Gradient for Semi-Supervised Conditional Random Fields" HLT/NAACL 2007

Gregory Druck

Nested Class Summary
Nested classes/interfaces inherited from class cc.mallet.fst.TransducerTrainer
TransducerTrainer.ByIncrements, TransducerTrainer.ByInstanceIncrements, TransducerTrainer.ByOptimization
Constructor Summary
CRFTrainerByEntropyRegularization(CRF crf)
Method Summary
 int getIteration()
 Optimizer getOptimizer()
 Transducer getTransducer()
 boolean isFinishedTraining()
 void setEntropyWeight(double gamma)
          Sets the scaling factor for the entropy regularization term.
 void setGaussianPriorVariance(double variance)
 boolean train(InstanceList labeled, InstanceList unlabeled, int numIterations)
          Performs CRF training with label likelihood and entropy regularization.
 boolean train(InstanceList trainingSet, int numIterations)
          Train the transducer associated with this TransducerTrainer.
Methods inherited from class cc.mallet.fst.TransducerTrainer
addEvaluator, addEvaluators, removeEvaluator, runEvaluators, train
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail


public CRFTrainerByEntropyRegularization(CRF crf)
Method Detail


public void setGaussianPriorVariance(double variance)


public void setEntropyWeight(double gamma)
Sets the scaling factor for the entropy regularization term. In [Jiao et al. 06], this is gamma.

gamma -


public int getIteration()
Specified by:
getIteration in class TransducerTrainer


public Transducer getTransducer()
Specified by:
getTransducer in class TransducerTrainer


public boolean isFinishedTraining()
Specified by:
isFinishedTraining in class TransducerTrainer


public boolean train(InstanceList trainingSet,
                     int numIterations)
Description copied from class: TransducerTrainer
Train the transducer associated with this TransducerTrainer. You should be able to call this method with different trainingSet objects. Whether this causes the TransducerTrainer to combine both trainingSets or to view the second as a new alternative is at the discretion of the particular TransducerTrainer subclass involved.

Specified by:
train in class TransducerTrainer


public boolean train(InstanceList labeled,
                     InstanceList unlabeled,
                     int numIterations)
Performs CRF training with label likelihood and entropy regularization. The CRF is first trained with label likelihood only. This parameter setting is used as a starting point for the combined optimization.

labeled - Labeled data, only used for label likelihood term.
unlabeled - Unlabeled data, only used for entropy regularization term.
numIterations - Number of iterations.
True if training has converged.


public Optimizer getOptimizer()
Specified by:
getOptimizer in interface TransducerTrainer.ByOptimization