Package jazzparser :: Package utils :: Package nltk :: Package ngram :: Module model :: Class NgramModel
[hide private]
[frames] | no frames]

Class NgramModel

source code

object --+
         |
        NgramModel
Known Subclasses:

A general n-gram model, trained on some labelled data. Generate models using NgramModel.train().

Note that backoff assumes that an estimator with discounting is being used. The backed off probabilities are scaled according to the smoothing probability that would have been assigned to a zero count. If you use this with MLE, the backoff is effectively disabled.

The estimator should be picklable. This means you can't use a lambda, for example.

Instance Methods [hide private]
 
__init__(self, order, label_counts, emission_counts, estimator, backoff_model, label_dom, emission_dom)
x.__init__(...) initializes x; see help(type(x)) for signature
source code
 
__repr__(self)
repr(x)
source code
 
_get_model_type(self)
model_type gives a sensible name to the order of n-gram.
source code
 
_get_transition_backoff_scaler(self, context)
Returns the amount to scale the backed off probabilities by when backing off to an order n-1 model in the given context.
source code
 
backward_log_probabilities(self, sequence, normalize=True)
Removed. See forward_log_probabilities
source code
 
backward_probabilities(self, sequence, normalize=True)
Removed. See forward_log_probabilities
source code
 
clear_cache(self)
Subclasses that use caches should override this method to reset them and call this in the superclass.
source code
 
decode_forward(self, sequence)
Uses the forward probabilities to decode the sequence and returns a list of labels.
source code
 
decode_gamma(self, sequence)
Use the state occupation probabilities to decode a sequence.
source code
 
emission_log_probability(self, emission, label)
Gives the probability P(emission | label).
source code
 
emission_probability(self, emission, label)
Wrapper for emission_log_probability to return an real probability.
source code
 
forward_backward_log_probabilities(self, sequence, normalize=True)
Removed. See forward_log_probabilities.
source code
 
forward_backward_probabilities(self, *args, **kwargs)
Removed. See forward_backward_log_probabilities
source code
 
forward_log_probabilities(self, sequence, normalize=True)
Removed. Use normal_forward_probabilities and take logs if you're happy with normalized probabilities.
source code
 
forward_probabilities(self, sequence, normalize=True)
Removed. See note on forward_log_probabilities.
source code
 
gamma_probabilities(self, sequence, dictionary=False, forward=None, backward=None)
State-occupation probabilities.
source code
list of (label sequence,probability) pairs
generalized_viterbi(self, sequence, N=2)
Applies the N-best variant of the Viterbi algorithm to return N sequences of states that maximize the probability of the sequence of observations.
source code
 
generate(self, length=10, labels=False)
Generate a sequence of emissions at random using the n-gram model.
source code
 
get_all_ngrams(self, n)
Returns all possible ngrams of length n composed of elements from this model's domain.
source code
 
get_backoff_models(self)
Returns a list consisting of this model and all the recursive backoff models, until no more backoff models are provided.
source code
 
get_emission_matrix(self, sequence)
Produces a matrix of the probability of each timestep's emission from each state.
source code
 
get_transition_matrix(self, transpose=False)
Produces a matrix of the transition probabilities from every (n-1)-gram to every state.
source code
 
labeled_sequence_log_probability(self, sequence, labels)
Computes the joint probability that the model assigns to a sequence and its gold standard labels.
source code
 
normal_backward_probabilities(self, sequence)
Return the backward probability matrices a Numpy array.
source code
 
normal_forward_backward_probabilities(self, sequence, forward=None, backward=None)
A faster implementation of forward_backward_probabilities for the case where we're normalizing, using Numpy and non-log probabilities.
source code
 
normal_forward_probabilities(self, sequence, seq_prob=False)
Return the forward probability matrix as a Numpy array.
source code
 
precompute(self)
Creates a PrecomputedNgramModel from this NgramModel.
source code
 
to_picklable_dict(self)
Produces a picklable representation of model as a dict.
source code
 
transition_log_probability(self, *ngram)
Gives the probability P(label_i | label_(i-1), ..., label_(i-n)), where the previous labels are given in the sequence label_context.
source code
 
transition_log_probability_debug(self, *ngram)
Debugging version of the above.
source code
 
transition_probability(self, label, *label_context)
Wrapper for transition_log_probability to return a real probability.
source code
 
transition_probability_debug(self, *ngram)
Debugging version of the above
source code
 
viterbi_decode(self, sequence)
Applies the Viterbi algorithm to return a single sequence of states that maximises the probability of the sequence of observations.
source code
 
viterbi_selector_probabilities(self, sequence)
Returns a probability matrix like that given by the forward-backward algorithm which assigns prob 1.0 to the Viterbi chosen tags and 0.0 to all others.
source code

Inherited from object: __delattr__, __format__, __getattribute__, __hash__, __new__, __reduce__, __reduce_ex__, __setattr__, __sizeof__, __str__, __subclasshook__

Static Methods [hide private]
 
from_picklable_dict(data, *args, **kwargs)
Reproduces an n-gram model that was converted to a picklable form using to_picklable_dict.
source code
 
train(order, labelled_data, label_dom, emission_dom=None, cutoff=0, backoff_order=0, estimator=None, ignore_list=None, backoff_kwargs={})
Trains the n-gram model given some labelled data.
source code
Properties [hide private]
  model_type
model_type gives a sensible name to the order of n-gram.

Inherited from object: __class__

Method Details [hide private]

__init__(self, order, label_counts, emission_counts, estimator, backoff_model, label_dom, emission_dom)
(Constructor)

source code 

x.__init__(...) initializes x; see help(type(x)) for signature

Overrides: object.__init__
(inherited documentation)

__repr__(self)
(Representation operator)

source code 

repr(x)

Overrides: object.__repr__
(inherited documentation)

_get_model_type(self)

source code 

model_type gives a sensible name to the order of n-gram. Uses unigram, bigram, trigram, or x-gram.

_get_transition_backoff_scaler(self, context)

source code 

Returns the amount to scale the backed off probabilities by when backing off to an order n-1 model in the given context. This is presented as alpha in Jurafsky and Martin.

Returned as a base 2 log.

A more efficient way to do this would be to supply a function of the context specific to the discounting technique. In this case it wouldn't be necessary to sum the discounted mass each time.

emission_log_probability(self, emission, label)

source code 

Gives the probability P(emission | label). Returned as a base 2 log.

forward_backward_log_probabilities(self, sequence, normalize=True)

source code 

Removed. See forward_log_probabilities. Use either normal_forward_backward_probabilities or gamma_probabilities.

forward_log_probabilities(self, sequence, normalize=True)

source code 

Removed. Use normal_forward_probabilities and take logs if you're happy with normalized probabilities. Otherwise, normal_forward_probabilities needs to be made to return the sums it normalizes by.

from_picklable_dict(data, *args, **kwargs)
Static Method

source code 

Reproduces an n-gram model that was converted to a picklable form using to_picklable_dict.

Extra args/kwargs are passed to the class constructor.

gamma_probabilities(self, sequence, dictionary=False, forward=None, backward=None)

source code 

State-occupation probabilities.

Parameters:
  • dictionary (bool) - return a list of label dictionaries instead of a numpy matrix

generalized_viterbi(self, sequence, N=2)

source code 

Applies the N-best variant of the Viterbi algorithm to return N sequences of states that maximize the probability of the sequence of observations.

Parameters:
  • N (int) - number of label sequences to return (defaults to 2)
Returns: list of (label sequence,probability) pairs
ordered list of possible decodings, paired with their Viterbi probabilities

See Also: Generalization of the Viterbi Algorithm, Foreman, 1993

generate(self, length=10, labels=False)

source code 

Generate a sequence of emissions at random using the n-gram model. If labels=True, outputs a sequence of (emission,label) pairs indicating what hidden labels emitted the emissions.

The sequence will have maximum length length, but may be shorter if the model so determines.

get_emission_matrix(self, sequence)

source code 

Produces a matrix of the probability of each timestep's emission from each state.

matrix[t,i] = p(o_t | state=i)

get_transition_matrix(self, transpose=False)

source code 

Produces a matrix of the transition probabilities from every (n-1)-gram to every state. Matrix indices are based on enumeration of self.label_dom. The matrix is returned as a numpy array.

The matrix has n dimensions. The first index is the current state, the second the previous, etc. Thus, matrix[i,j,...] = p(state_t = i | state_(t-1) = j, ...).

Probabilities are not logs.

labeled_sequence_log_probability(self, sequence, labels)

source code 

Computes the joint probability that the model assigns to a sequence and its gold standard labels.

Probability is a log, because we'd get underflow otherwise.

normal_backward_probabilities(self, sequence)

source code 

Return the backward probability matrices a Numpy array. This is faster than backward_log_probabilities because it uses Numpy arrays with non-log probabilities and normalizes each timestep.

Returns:
matrix over timesteps and all labels, with a dimension for each state in each (n-1)-gram: for time steps i and labels k, P(word^(i+1), ..., word^T | label_k^i)

normal_forward_backward_probabilities(self, sequence, forward=None, backward=None)

source code 

A faster implementation of forward_backward_probabilities for the case where we're normalizing, using Numpy and non-log probabilities.

This is still an S-dimensional matrix, not the state-occupation probabilities. Use gamma_probabilities to get that.

normal_forward_probabilities(self, sequence, seq_prob=False)

source code 

Return the forward probability matrix as a Numpy array. This is equivalent to forward_probabilities, but much faster. It doesn't need logs because it's normalizing at each timestep.

Parameters:
  • seq_prob - return the log probability of the whole sequence as well as the array (tuple of (array,logprob)).
Returns:
S-dimensional Numpy array, where S is the number of states. The first dimension represents timesteps, the rest the (S-1) states in the ngrams.

to_picklable_dict(self)

source code 

Produces a picklable representation of model as a dict. You can't just pickle the object directly because some of the NLTK classes can't be pickled. You can pickle this dict and reconstruct the model using NgramModel.from_picklable_dict(dict).

train(order, labelled_data, label_dom, emission_dom=None, cutoff=0, backoff_order=0, estimator=None, ignore_list=None, backoff_kwargs={})
Static Method

source code 

Trains the n-gram model given some labelled data. The data should be in the form of a sequence of sequences of tuples (e,t), where e is an emission and t a tag.

If ignore_list is given, all ngrams containing a label in ignore_list will be ignored altogether. One use for this is to ignore blank labels, so we don't learn the gaps in the labelled data.

transition_log_probability(self, *ngram)

source code 

Gives the probability P(label_i | label_(i-1), ..., label_(i-n)), where the previous labels are given in the sequence label_context. The context should be in reverse order, i.e. with the most recent label at the start.

Note that this is the probability of a label given the previous n-1 labels, which is the same as the probability of the n-gram [label_i, ..., label_(i-n+1)] given the ngram [label_(i-1), ..., label_(i-n)], since all but the last element of the ngram overlaps with the condition, so has probability 1.

Caches all computed transition probabilities. This is particularly important for backoff models. Many n-grams will back off to the same (n-1)-gram and we don't want to recompute the transition probability for that each time.

transition_log_probability_debug(self, *ngram)

source code 

Debugging version of the above. Use this only for debugging. It prints stuff out and doesn't cache.


Property Details [hide private]

model_type

model_type gives a sensible name to the order of n-gram. Uses unigram, bigram, trigram, or x-gram.

Get Method:
_get_model_type(self) - model_type gives a sensible name to the order of n-gram.