Modeling Temporal Structure in Music for Emotion Prediction using Pairwise Comparisons

Madsen, J., Sand Jensen, B. and Larsen, J. (2014) Modeling Temporal Structure in Music for Emotion Prediction using Pairwise Comparisons. In: ISMIR 2014: 15th International Society of Music Information Retrieval Conference, Taipei, Taiwan, 27-31 Oct 2014,

119554.pdf - Published Version
Available under License Creative Commons Attribution.


Publisher's URL:


The temporal structure of music is essential for the cognitive processes related to the emotions expressed in music. However, such temporal information is often disregarded in typical Music Information Retrieval modeling tasks of predicting higher-level cognitive or semantic aspects of music such as emotions, genre, and similarity. This paper addresses the specific hypothesis whether temporal information is essential for predicting expressed emotions in music, as a prototypical example of a cognitive aspect of music. We propose to test this hypothesis using a novel processing pipeline: 1) Extracting audio features for each track resulting in a multivariate "feature time series". 2) Using generative models to represent these time series (acquiring a complete track representation). Specifically, we explore the Gaussian Mixture model, Vector Quantization, Autoregressive model, Markov and Hidden Markov models. 3) Utilizing the generative models in a discriminative setting by selecting the Probability Product Kernel as the natural kernel for all considered track representations. We evaluate the representations using a kernel based model specifically extended to support the robust two-alternative forced choice self-report paradigm, used for eliciting expressed emotions in music. The methods are evaluated using two data sets and show increased predictive performance using temporal information, thus supporting the overall hypothesis.

Item Type:Conference Proceedings
Glasgow Author(s) Enlighten ID:Jensen, Dr Bjorn
Authors: Madsen, J., Sand Jensen, B., and Larsen, J.
College/School:College of Science and Engineering > School of Computing Science
Copyright Holders:Copyright © 2014 The Authors
Publisher Policy:Reproduced under a Creative Commons License
Related URLs:

University Staff: Request a correction | Enlighten Editors: Update this record