|
Processes the output from decoding the model in the form of a set of
top tags for each timestep. We need to find each possible sequence of
identical tags and combine then into a single span so that the
self-transition gets interpreted as a continuation of the category. We
also want to keep the smaller spans, right down to the single-timestep
categories, so that they can be considered by the parser in combination
with other spans that would otherwise overlap.
We do this using a little CKY chart parser.
|