Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## cc.mallet.fst.confidence
Class ConfidenceEvaluator

    java.lang.Object
      cc.mallet.fst.confidence.ConfidenceEvaluator

    public class ConfidenceEvaluator

extends java.lang.Object

Nested Class Summary
`static class` | `ConfidenceEvaluator.EntityConfidence` a simple class to store a confidence score and whether or not this labeling is correct

Constructor Summary
`ConfidenceEvaluator(InstanceWithConfidence[] instances, boolean sorted)`
`ConfidenceEvaluator(PipedInstanceWithConfidence[] instances, boolean sorted)`
`ConfidenceEvaluator(Segment[] segments, boolean sorted)`
`ConfidenceEvaluator(java.util.Vector confidences)`
`ConfidenceEvaluator(java.util.Vector confidences, int nBins)`

Method Summary
`double` | `accuracyAtCoverage(double cov)`
`java.lang.String` | `accuracyCoverageValuesToString()`
`java.lang.String` | `accuracyRecallValuesToString(int totalTrue)`
`double` | `correlation()` Calculate pearson's R for the corellation between confidence and correct, where 1 = correct and -1 = incorrect
`double[]` | `getAccuracyCoverageValues()` get accuracy at coverage for each bin of values
`double[][]` | `getAccuracyRecallValues(int totalTrue)` get accuracy at recall for each bin of values
`double` | `getAverageAccuracy()`
`double` | `getAverageCorrectConfidence()` Average confidence score for the incorrect entities
`double` | `getAverageIncorrectConfidence()` Average confidence score for the incorrect entities
`double` | `getAveragePrecision()` IR Average precision measure.
`double` | `getConfidenceMean()`
`double` | `getConfidenceStandardDeviation()` Standard deviation of confidence scores
`double` | `getConfidenceSum()`
`double` | `getWorstAveragePrecision()` For comparison, rank segments as badly as possible (all "incorrect" before "correct").
`int` | `numCorrect()`
`int` | `numCorrectAtCoverage(double cov)`
`double` | `pointBiserialCorrelation()` Correlation when one variable (X) is binary: r = (bar(x1) - bar(x0)) * sqrt(p(1-p)) / sx , where bar(x1) = mean of X when Y is 1 bar(x0) = mean of X when Y is 0 sx = standard deviation of X p = proportion of values where Y=1
`int` | `size()`
`java.lang.String` | `toString()`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Constructor Detail

### ConfidenceEvaluator

    public ConfidenceEvaluator(java.util.Vector confidences,
                               int nBins)

### ConfidenceEvaluator

    public ConfidenceEvaluator(java.util.Vector confidences)

### ConfidenceEvaluator

    public ConfidenceEvaluator(Segment[] segments,
                               boolean sorted)

### ConfidenceEvaluator

    public ConfidenceEvaluator(InstanceWithConfidence[] instances,
                               boolean sorted)

### ConfidenceEvaluator

    public ConfidenceEvaluator(PipedInstanceWithConfidence[] instances,
                               boolean sorted)

Method Detail

### pointBiserialCorrelation

    public double pointBiserialCorrelation()

  Correlation when one variable (X) is binary: r = (bar(x1) - bar(x0)) * sqrt(p(1-p)) / sx , where bar(x1) = mean of X when Y is 1 bar(x0) = mean of X when Y is 0 sx = standard deviation of X p = proportion of values where Y=1

### getAveragePrecision

    public double getAveragePrecision()

  IR Average precision measure. Analogous to ranking _correct_ documents by confidence score.

### getWorstAveragePrecision

    public double getWorstAveragePrecision()

  For comparison, rank segments as badly as possible (all "incorrect" before "correct").

### getConfidenceSum

    public double getConfidenceSum()

### getConfidenceMean

    public double getConfidenceMean()

### getConfidenceStandardDeviation

    public double getConfidenceStandardDeviation()

  Standard deviation of confidence scores

### correlation

    public double correlation()

  Calculate pearson's R for the corellation between confidence and correct, where 1 = correct and -1 = incorrect

### getAccuracyCoverageValues

    public double[] getAccuracyCoverageValues()

  get accuracy at coverage for each bin of values

### accuracyCoverageValuesToString

    public java.lang.String accuracyCoverageValuesToString()

### getAccuracyRecallValues

    public double[][] getAccuracyRecallValues(int totalTrue)

  get accuracy at recall for each bin of values

  Parameters:
    `totalTrue` - total number of true Segments
  Returns:
    2-d array where values[i][0] is coverage and values[i][1] is accuracy at position i.

### accuracyRecallValuesToString

    public java.lang.String accuracyRecallValuesToString(int totalTrue)

### accuracyAtCoverage

    public double accuracyAtCoverage(double cov)

### numCorrectAtCoverage

    public int numCorrectAtCoverage(double cov)

### getAverageAccuracy

    public double getAverageAccuracy()

### numCorrect

    public int numCorrect()

### getAverageIncorrectConfidence

    public double getAverageIncorrectConfidence()

  Average confidence score for the incorrect entities

### getAverageCorrectConfidence

    public double getAverageCorrectConfidence()

  Average confidence score for the incorrect entities

### size

    public int size()

### toString

    public java.lang.String toString()

  Overrides:
    `toString` in class `java.lang.Object`

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD