AMT for Multi Tone Piano Music Based on Visualization
Multi Tone Piano Music
Piano music has a high pitch variety which poses challenges for automatic music transcription (AMT). To tackle the challenge, many researchers have developed methods that analyze the musical signal to detect notes and their relationships to other notes. These methods are based on different techniques such as non-negative matrix factorization, neural networks and deep learning. However, most of the time it is difficult to understand the results from these approaches due to their complexity. Scientific computing visualization enables the transformation of piles of data into intuitive geometric images that are easier to understand.
The goal of this study is to develop a method for visualizing the results of AMT for multi tone piano music by using chromatic motion features, and more specifically, by displaying the corresponding chromatic motion graphs on the screen of a tablet computer. This tool can support the analysis of a large number of compositions by providing an intuitive overview of the high-level musical content of the compositions. It also aims to facilitate the interpretation of the results by allowing users to filter and sort the compositions on the basis of their musical characteristics.
The chromatic motion features show the movement of individual pitches over the course of the musical piece. Each point in the graph represents a specific note and the distance between two points is the frequency difference of the note. For example, the interval between C and D is one semitone. A semitone is the smallest difference that can be made between any pair of neighboring notes on a piano keyboard.
AMT for Multi Tone Piano Music Based on Visualization
Aside from pitch, a key feature of a particular piano note is its timbre. Timbre is the overall sound quality of a note, and it depends on the balance between its harmonic series components. For each note, there are 16 possible harmonics. If there are more harmonics than others, the note will have a richer and fuller timbre. On the other hand, if all of the harmonics are canceled out then the note will have a much sparser timbre.
To visualize these features, the authors use a visualization application called CorpusVis. The application allows the users to select a certain composition and visualize the corresponding chromatic motion graphs in three dimensions on the screen of their tablet computer. The resulting chromatic motion graphs make it easy to identify the movement of all the individual tones in the piano music. In addition, the chromatic motion graphs can be sorted according to composers and epochs.
During the user study, several participants expressed that they appreciated the functionality of CorpusVis. In particular, they appreciated the ability to select a composer of interest and view their compositions in detail. Moreover, the feature matrix function enabled them to compare the similarity between compositions composed by the selected composer and atonal pieces from Schonberg and Webern. This helped them confirm that atonal pieces do not have a tonal center and that their objective is not to favor certain pitch classes.