Package jazzparser :: Package evaluation :: Module taggers
[hide private]
[frames] | no frames]

Module taggers

source code

Tagger evaluation routines.


Note: This was originally jazzparser.utils.stats, but only ended up having tagger evaluation stuff in it.

Author: Mark Granroth-Wilding <mark.granroth-wilding@ed.ac.uk>

Functions [hide private]
 
tagger_entropy(input, grammar, tagger_class, correct_tags, options={})
Given some input, a tagger and a list of correct tags, computes the entropy of the tagger over the input, returning a tuple of the total entropy and the number of chords that were involved in the computation.
source code
 
tagger_agreement(input, grammar, tagger_class, correct_tags, options={}, confusion_matrix=None, topn=1)
Like tagger_entropy, but instead of computing the entropy computes the proportion of top tags assigned that agree with the gold standard.
source code
Variables [hide private]
  logger = logging.getLogger("main_logger")
  __package__ = 'jazzparser.evaluation'
Function Details [hide private]

tagger_entropy(input, grammar, tagger_class, correct_tags, options={})

source code 

Given some input, a tagger and a list of correct tags, computes the entropy of the tagger over the input, returning a tuple of the total entropy and the number of chords that were involved in the computation.

The option dictionary options will be passed to the tagger.

tagger_agreement(input, grammar, tagger_class, correct_tags, options={}, confusion_matrix=None, topn=1)

source code 

Like tagger_entropy, but instead of computing the entropy computes the proportion of top tags assigned that agree with the gold standard.

The option dictionary options will be passed to the tagger.

Optionally puts a confusion matrix in the dictionary given in confusion_matrix. This is a dictionary keyed by correct tags whose values are dictionaries keyed by the incorrect tags that were confused for them, whose values are a count of the confusions.

Returns a tuple (number_agreeing, number_compared).