Data-Driven Exploration of Musical Chord Sequences

We present data-driven methods for supporting musical creativity by capturing the statistics of a musical database. Specifically, we introduce a system that supports users in exploring the high-dimensional space of musical chord sequences by parameterizing the variation among chord sequences in popular music. We provide a novel user interface that exposes these learned parameters as control axes, and we propose two automatic approaches for defining these axes. One approach is based on a novel clustering procedure, the other on principal components analysis. A user study compares our approaches for defining control axes both to each other and to an approach based on manually-assigned genre labels. Results show that our automatic methods for defining control axes provide a subjectively better user experience than axes based on manual genre labeling.

Our IUI 2009 paper

Nichols E, Morris D, Basu S. Data-Driven Exploration of Musical Chord Sequences. Proceedings of Intelligent User Interfaces (IUI) 2009, February 2009.

[ pdf (opens in new tab) ]

Our Chord Transition Matrices

Our paper discusses the automatic and semi-automatic generation of chord transition matrices from databases of popular music. We have made those transition matrices available for readers here. The full set of transition matrices, along with an explanation of their format, is available in a single zip file:

nichols et al transitions.zip (opens in new tab)

The remainder of this page presents the format of these matrices; this description is identical to the one available in the zipfile.

Songsmith Compatibility

To enable experimentation and exploration with the data presented on this page, we highlight that these data files (except for the PCA-based files) are in the same format as those used for the Microsoft Research Songsmith (opens in new tab) automatic accompaniment system, which implements the HMM-based chord-generation system described in Simon et al, 2008 (opens in new tab). Songsmith uses a separate set of transition matrices which represent aggregate data from several genres that has been hand-tuned for the product; for convenience and comparison, we also include the Songsmith transition matrices here (they are included with the Songsmith product as well):

Songsmith major-key transition matrix (opens in new tab)
Songsmith minor-key transition matrix (opens in new tab)

Files

The four directories contained here correspond to the four experimental conditions described in our paper. Each directory contains the corresponding data for the four control axes used in that condition, in ASCII text files whose format is described below:

Genre (transitions matrices derived from labeled genre data):

Genre+AbsDiff (transitions matrices derived from our AbsDiff clustering routine using genre labels as cluster seeds):

PCA (principal components analysis of all chord transitions in our database)

Random+AbsDiff (transitions matrices derived from our AbsDiff clustering routine using random seeds):

File Formats

All of the files contained here describe transition probabilities among chords. All songs were transposed into the key of C before analysis. Our dictionary of chords includes five triad types — major, minor, diminished, augmented, suspended — so we define 60 types of chords, one triad type on each of 12 possible roots, numbered as follows:

Additionally we define two special chords that represent “start of song” and “end of song”, so the probability of each chord starting and ending a song can be reflected in our data.

All probabilities are represented as _log_-probability values, so all values are negative.

For all conditions other than the PCA condition, each text file contains one transition matrix, in the following format:

The files in the ‘pca’ directory, which correspond to the PCA-based axes presented in our paper, are in the following format:

Note that unlike the other conditions, these are transition probabilities, not log-probabilities, so they can be combined directly without exponentiating. Also note the introduction of an additional line at the beginning of the file, specifying the variance for this component. We provide four principal components in the files pca_transmodel_[1-4].txt, and the mean — to which scaled principal component values are added to produce a transition matrix — in the file pca_transmodel_mean.txt.