Package jazzparser :: Package evaluation :: Module taggers

Module taggers

Tagger evaluation routines.

Note: This was originally jazzparser.utils.stats, but only ended up having tagger evaluation stuff in it.

Author: Mark Granroth-Wilding <mark.granroth-wilding@ed.ac.uk>

Functions

tagger_entropy(input, grammar, tagger_class, correct_tags, options={})
Given some input, a tagger and a list of correct tags, computes the entropy of the tagger over the input, returning a tuple of the total entropy and the number of chords that were involved in the computation. source code

tagger_agreement(input, grammar, tagger_class, correct_tags, options={}, confusion_matrix=None, topn=1)
Like tagger_entropy, but instead of computing the entropy computes the proportion of top tags assigned that agree with the gold standard. source code

Variables

[hide private]

logger = logging.getLogger("main_logger")

__package__ = 'jazzparser.evaluation'

Function Details

[hide private]

tagger_entropy(input, grammar, tagger_class, correct_tags, options=`{}`)

source code

Given some input, a tagger and a list of correct tags, computes the entropy of the tagger over the input, returning a tuple of the total entropy and the number of chords that were involved in the computation.

The option dictionary options will be passed to the tagger.

tagger_agreement(input, grammar, tagger_class, correct_tags, options=`{}`, confusion_matrix=None, topn=1)

source code

Like tagger_entropy, but instead of computing the entropy computes the proportion of top tags assigned that agree with the gold standard.

The option dictionary options will be passed to the tagger.

Optionally puts a confusion matrix in the dictionary given in confusion_matrix. This is a dictionary keyed by correct tags whose values are dictionaries keyed by the incorrect tags that were confused for them, whose values are a count of the confusions.

Returns a tuple (number_agreeing, number_compared).

Module taggers

tagger_entropy(input, grammar, tagger_class, correct_tags, options={})

tagger_agreement(input, grammar, tagger_class, correct_tags, options={}, confusion_matrix=None, topn=1)

tagger_entropy(input, grammar, tagger_class, correct_tags, options=`{}`)

tagger_agreement(input, grammar, tagger_class, correct_tags, options=`{}`, confusion_matrix=None, topn=1)