I have implemented a model of tonal space semantics that takes the sequence as direct input. The aim of this is to establish how much the grammatical information is adding to a supertagging model.
I have experimented with various supertagging models, including some using n-grams. The idea of this experiment is to implement a sequence model of the semantics directly that is equivalent to a model that is performing successfully as a supertagger to narrow the search space of the parser.
N-gram Models
Models
I measured the performance of n-gram models that assign a tonal space point to each chord in a sequence. In keeping with the n-gram models used for supertagging, the input to the model is an observation made up of the chord type and the interval between the chord and that following it.
The output of the model is a combination of an interval and a tonal function. The interval represents the interval in semitones between the transcribed root of the chord and the true root of the tonal space point.
I trained two models: a bigram model and a trigram model. The parameters are those that work well for supertagger, though one of the results of the supertagging experiments was that these parameters don't make much different. I used Katz backoff to a bigram model (in the case of the trigram model) and then to a unigram model. I used a cutoff of 2 on all model counts and smoothed using Witten-Bell discounting.
Evaluation
I evaluated the output of the model in precisely the same way I evaluate the output of the parser - by using a distance measure on the two tonal space paths and converting the result to a percentage.
I haven't yet documented this on the wiki, but see my first year review for an explanation.
Results
Model |
Accuracy (%) |
Accuracy std dev |
Grammarless bigram |
69.11% |
10.33% |
Grammarless trigram |
67.93% |
11.28% |
- TODO: include supertagging results here for comparison