This guide is written for Linux users and should suit Mac users well enough. I have not run the parser under Windows much myself, but have tested the basic running of the thing. There may be problems with running under Windows that I don't know about.

I'm assuming you understand basic terminal usage. The commands shown below include a ">" command prompt: don't type this.

First download the parser and all of its dependencies.

Basic Usage

Running the parser to parse a short sequence of chords is simple. Once you've downloaded the code, open up a terminal and change to the bin directory. To run the parser's command line interface, run the following command:

> ./jazzparser

If you're using Windows, fire up cmd to get a terminal. You can do all the same things described below using the batch file instead of the Bash script in the previous example, like so:

> jazzparser.bat

If all goes well, you should see a prompt for a chord sequence. We've not given any input, so you'll just see the message "No input string given" and it will wait for input from the command line.

You can give input in various forms. One way is to run the parser as you just did and type a chord sequence, followed by <enter>. You can do this for as many chord sequences as you like. Press Ctrl+D to exit.

Alternatively, you can give a chord sequence as an argument. Here's some simple input to show the parser in action:

> ./jazzparser "A E7 A"

This is performing a full CKY parse with no statistical models. It's using the default parser module, which is a plain CKY parser (called cky), and the default supertagger, which just supplies all possible categories to the parser (called full).

You should get some output like this for this first example:

Processing input: A E7 A (commandline)
Tagging sequence (3 timesteps)
>>> Parsing iteration: 1
Parse finished with 31 results
Parsed: A E7 A
Results:
0>  Db^T : [<3,1>, <0,1>, <3,1>]
...
26>  A^T : [<3,0>, leftonto(<3,0>)]
...
30>  A^T : [<3,0>, leftonto(leftonto(<3,0>))]
  commandline   Complete


Completed 1/1 parses

That result numered 26 is the one we're looking for. Since there's no probabilistic model involved in the parsing, the parser has no idea of how plausible each of these results is.

What you see here are CCG categories: syntactic type before the : and logical form after. The logical form gives the tonal space analysis. In result 26, the sequence starts at A (coordinate (3,2)), then cadences back to A again using a single left step.

Use the -h (or --help) option to see all the command line options the parser can take.

Using a Parsing Model

Use the -p option (or --parser) to select a parsing module. The default, cky, uses no probabilistic model. Instead, let's use pcfg.

> ./jazzparser -p pcfg "A E7 A"

This will cause an error. Each parsing module has a set of options that you can specify using --popt (or --parser-options) on the command line. Some parser modules require certain options to be given. To see a list of all a module's options, type help where the options would be:

> ./jazzparser -p pcfg --popt help

This applies in general wherever a module takes options in this form (for example, supertagger options). Module options can be given in the form option=value and multiple options may be given using multiple --popt options or by separating them with colons (option1=value1:option2=value2).

In the parser option help, we can see that the model option is required for the pcfg parser. This specifies a trained parsing model. A different script is used to train models, but fortunately one is included pre-trained with the parser, called chords.

Now we can load a model and parse using the probabilistic parser.

> ./jazzparser -p pcfg --popt model=chords "A E7 A"
...
Parse finished with 13 results
Parsed: A E7 A
Results:
0>  A^T : [<3,0>, leftonto(<3,0>)]
...
12>  E^T-Db^T : [<0,1>, <3,1>]

This time the parser has ranked the results by their probability. It's still entertained some whacky interpretations, but there are far fewer results than before, because low-probability interpretations have been eliminated, and the one we want is at the top of the list.

The model we've told it to use is a statistical parsing model trained using a corpus of jazz chord sequences annotated with analyses (a treebank). For now, we'll be using the pre-trained models distributed with the code.

Let's try a slightly longer chord sequence, using some different chord types:

> ./jazzparser -p pcfg --popt model=chords "A F#o7 Bm7 E7 AM7"
...
0>  A^T : [<3,0>, leftonto(leftonto(leftonto(<3,0>)))]

The parser has again managed to get the interpretation we want, treating it as an extended cadence, at the top - it is, after all, a pretty simple example.

Using a Supertagging Model

The sort of parsing we've tried so far starts by looking up all possible grammatical categories for each individual chord from the grammar's lexicon (lexical categories). It then tried combining them into a full interpretation. Before we introduced the -p option, it was looking for all possible combinations. And then, when we specified a parsing model, it was able to select just the most promising combinations.

A supertagging model helps the parser out by cutting down the selection of lexical categories the parser has to start with. It does this by looking at the context of each chord and comparing to interpretations previously used in similar contexts.

The parser did fine on its own with the short examples above. Try parsing this chord sequence:

> ./jazzparser "A Bm7 E7 Bm7 E7 AM7"

You'll find it takes much longer, and that's still a very short chord sequence if you compare to real songs. What's more, the parser's already running with very constrained settings (the default settings), so its ranking of the final results is not very good.

Take a look at the parameters that the parser is using when you run it with no options:

> ./jazzparser 
No input string given: accepting input from stdin. Hit Ctrl+d to exit
===== The Jazz Parser =====
Supertagger:         full
...

It's already using a supertagger. In fact, the Jazz Parser always uses a supertagger. It's just that here it's using a special case, the full supertagger, that just gives the parser all the lexical categories that the grammar allows.

Let's speed things up by using a statistical supertagger:

> ./jazzparser -t ngram-multi --topt model=bigram "A Bm7 E7 Bm7 E7 AM7" -p pcfg --popt model=chords
...
Parse finished with 2 results
Parsed: A Bm7 E7 Bm7 E7 AM7
Results:
0>  A^T : [<3,0>, ((\x1.leftonto(leftonto(x1)) & \x1.leftonto(leftonto(x1))) <3,0>)]

Here we used the ngram-multi supertagger, which uses an ngram model (the generalization of an HMM) to choose the categories. The model we chose was a bigram model (i.e. an HMM), trained on the same corpus as the parsing model. The result should have popped out pretty quickly this time.

The parser can now handle much longer chord sequences. Both the supertagger and the parser can produce better quality results if you change their beam settings. By default, these are set so that a result is produced quickly, which is good for firing off parses from the command line like this. When running experiments, it's better to use less constrained settings and leave the parser going longer.

Parser and Supertagger Options

Above, we chose a parsing module using -p and a supertagging module using -t. Each module comes with its own setting of parameters. We had to set one for both the parser and supertagger to tell them what statistical models to use (--topt model=bigram and --popt model=chords).

There are other parsing and supertagging modules as well as pcfg and ngram-multi and each has its own list of parameters. The parameters are specified using --popt and --topt respectively. You can get a list of the available parameters with --popt help (along with -p to select the parsing module).

> ./jazzparser -p pcfg --popt help
Available options for selected parser
=====================================
model             model=X, where X is the name of the   Name of a trained PCFG model to use for   
                  model. (REQUIRED)                     parsing.                                  

max_iter          max_iter=X, where X is an integer.    Maximum number of parser iterations to    
...

> ./jazzparser -t ngram-multi --topt help
Available options for selected tagger
=====================================
model       model=X, where X is the name of a     Model name. This model must have been     
            trained model (REQUIRED)              previously trained. Required              

partition   partition=P, where P is an int        If given, a partitioned version of the
...

You can specify multiple parameter settings by separating them with a colon (e.g. --popt model=chords:inspect=T), or by giving multiple --popts (e.g. --popt model=chords --popt inspect=T). The same goes for --topt and other module options that work in this way.

More Topics

That covers the basic usage of the parser. The following pages document other features.

The rest of this is old

The rest of this should be moved to other pages.

Derivations

For the sake of parsing time and space, the parser by default does not store the necessary information to display derivation trees once parsing has finished. However, the -d option makes it store this information.

Use this with the -i option: then you can use the d command in the interactive shell to print out a textual representation of the derivation tree for a specific result. This includes all the possible derivations of the result, condensed into one tree, so this tree can get pretty huge!

File Input

Instead of typing in individual chord sequences, you can get input from a file using the -f option, followed by the filename. This can allow you to parse multiple sequences in one go.

The Jazz Parser accepts several types of input. The default is just a textual representation of chords, like that we parsed from the command line above. All input types can be given as input files, included textual chords.

Select a filetype using the --ft option. --ft help shows a list of filetypes.

Select an input file with the -f option. Some filetypes have their own options, which you can specify using the --fopt option. Use --fopt help (in conjunction with --ft) to see available options.

Chord Input Files

The simplest input format is plain text chords. This is the same as the input we gave on the command line above. You can just put a chord sequence in a file and take it as input using -f <filename>. (This is filetype chords, but it's the default, so no need for --ft.)

More interestingly, you can put multiple chord sequences in a single file and the parser will process them each in turn. This is filetype bulk-chords.

The input file should contain lines of chord sequences, like this:

Im(6)       II%7, V7        Im           V%7, I7    IV7     #IVo7         Im(6)  bIIIm7, bVI7    II%7          V7        Im(6) Im(6)

Other useful things you can put in there are:

IM7           | IIm7  V7    | IM7           | IIm7  V7    || IM7   I7    | IVM7  #IVo7   | V7            | IM7

// Here are some more chord sequences. This line will be ignored.

>>Rhythm changes A sections

>>=Rhythm changes example 1
...
//=Autumn Leaves A
...