cc.mallet.fst.semi_supervised
Class CRFTrainerByEntropyRegularization

java.lang.Object
  extended by cc.mallet.fst.TransducerTrainer
      extended by cc.mallet.fst.semi_supervised.CRFTrainerByEntropyRegularization
All Implemented Interfaces:
TransducerTrainer.ByOptimization

public class CRFTrainerByEntropyRegularization
extends TransducerTrainer
implements TransducerTrainer.ByOptimization

A CRF trainer that maximizes the log-likelihood plus a weighted entropy regularization term on unlabeled data. Intuitively, it aims to make the CRF's predictions on unlabeled data more confident. References: Feng Jiao, Shaojun Wang, Chi-Hoon Lee, Russell Greiner, Dale Schuurmans "Semi-supervised conditional random fields for improved sequence segmentation and labeling" ACL 2006 Gideon Mann, Andrew McCallum "Efficient Computation of Entropy Gradient for Semi-Supervised Conditional Random Fields" HLT/NAACL 2007

Author:
Gregory Druck

Nested Class Summary
 
Nested classes/interfaces inherited from class cc.mallet.fst.TransducerTrainer
TransducerTrainer.ByIncrements, TransducerTrainer.ByInstanceIncrements, TransducerTrainer.ByOptimization
 
Constructor Summary
CRFTrainerByEntropyRegularization(CRF crf)
           
 
Method Summary
 int getIteration()
           
 Optimizer getOptimizer()
           
 Transducer getTransducer()
           
 boolean isFinishedTraining()
           
 void setEntropyWeight(double gamma)
          Sets the scaling factor for the entropy regularization term.
 void setGaussianPriorVariance(double variance)
           
 boolean train(InstanceList labeled, InstanceList unlabeled, int numIterations)
          Performs CRF training with label likelihood and entropy regularization.
 boolean train(InstanceList trainingSet, int numIterations)
          Train the transducer associated with this TransducerTrainer.
 
Methods inherited from class cc.mallet.fst.TransducerTrainer
addEvaluator, addEvaluators, removeEvaluator, runEvaluators, train
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CRFTrainerByEntropyRegularization

public CRFTrainerByEntropyRegularization(CRF crf)
Method Detail

setGaussianPriorVariance

public void setGaussianPriorVariance(double variance)

setEntropyWeight

public void setEntropyWeight(double gamma)
Sets the scaling factor for the entropy regularization term. In [Jiao et al. 06], this is gamma.

Parameters:
gamma -

getIteration

public int getIteration()
Specified by:
getIteration in class TransducerTrainer

getTransducer

public Transducer getTransducer()
Specified by:
getTransducer in class TransducerTrainer

isFinishedTraining

public boolean isFinishedTraining()
Specified by:
isFinishedTraining in class TransducerTrainer

train

public boolean train(InstanceList trainingSet,
                     int numIterations)
Description copied from class: TransducerTrainer
Train the transducer associated with this TransducerTrainer. You should be able to call this method with different trainingSet objects. Whether this causes the TransducerTrainer to combine both trainingSets or to view the second as a new alternative is at the discretion of the particular TransducerTrainer subclass involved.

Specified by:
train in class TransducerTrainer

train

public boolean train(InstanceList labeled,
                     InstanceList unlabeled,
                     int numIterations)
Performs CRF training with label likelihood and entropy regularization. The CRF is first trained with label likelihood only. This parameter setting is used as a starting point for the combined optimization.

Parameters:
labeled - Labeled data, only used for label likelihood term.
unlabeled - Unlabeled data, only used for entropy regularization term.
numIterations - Number of iterations.
Returns:
True if training has converged.

getOptimizer

public Optimizer getOptimizer()
Specified by:
getOptimizer in interface TransducerTrainer.ByOptimization