Are the Emotions Expressed in Music Genre-specific? An Audio-based Evaluation of Datasets Spanning Classical, Film, Pop and Mixed Genres

Figures & data

Table 1. List of materials included (N refers to number of stimuli, L to median stimulus length is seconds).

Study	Genre	Abbr.	N	L
Gomez and Danuser (2004)	Classical	C1	16	30
Bigand et al. (2005)	Classical	C2	27	37
Dibben (2004)	Classical	C3	16	35
Eerola and Vuoskoski (2011)	Film	F1	110	15
Gosselin et al. (2005)	Film	F2	40	12
Yang et al. (2007)	Pop	P1	60	30
Schuller et al. (2010)	Pop	P2	2598	30
Nater et al. (2006)	Mix	M1	20	29
Wu and Jeng (2006)	Mix	M2	75	12

Fig. 1 Behavioural ratings in all datasets (normalized) showing valence (x-axis) and arousal (y-axis).

Table 2. The full feature set and the reduced feature set (nine highlighted features).

Domain	No.	Feature	Acronym
Dynamics	1–3	RMS energy	RMSm, RMSd*, RMSl*
	4	Low-Energy ratio	LEm
Articulation	5–6	Attack Time	ATm, Atd
	7	Attack Slope	ASm
Rhythm	8	Event Density	EDm
	9	Fluctuation Peak	FPm
	10–11	Fluctuation Peak (mag.)	FMm *, FCm
	12–13	Tempo	Tm *, Td *
	14–15	Pulse Clarity	PCm, PCd *
Timbre	16–17	Spectral Centroid	SCm, SCd
	18	Spectral Spread	SSm
	19–20	Roughness	Rm, Rd
	21–22	Spectral Flux	SFm *, SFd
	23–24	Regularity	REm, REd
	25–26	Inharmonicity	Im, Id
Pitch	27–28	Pitch	Pm *, Pd
	29–30	Chromagram (unwrapped) centr.	Cm, Cd
Tonal	31–32	Key Clarity	KCm, KCd
	33	Key Mode (majorness)	Mm *
	34	HCDF	Hm
	35	Spectral Entropy (oct. coll.)	SEm
Structure	36	Repetition (Spectrum)	RSm
	37	Repetition (Rhythm)	RRm
	38	Repetition (Tonal)	RTm
	39	Repetition (Register)	RGm
Note: In acronyms, m = mean, d = standard deviation, and l = slope.

Fig. 2 The exhaustive cross-validation scheme used in comparing models across genres.

Table 3. Model performances (R ²) for valence across the datasets.

		Prediction
		C1	C2	C3	F1	F2	P1	P2	M1	M2	Mdn
Training	C1	0.74*	0.03	0.01	0.00	0.16	0.00	0.03*	0.15	0.02	0.03
	C2	0.00	0.73*	0.13	0.17*	0.17	0.01	0.01*	0.41*	0.07	0.13
	C3	0.04	0.13	0.97*	0.05	0.41*	0.19*	0.02*	0.04	0.09	0.09
	F1	0.36	0.43*	0.56*	0.70*	0.50*	0.18*	0.00	0.48*	0.35*	0.43
	F2	0.02	0.17	0.26	0.27*	0.85*	0.05	0.00	0.09	0.05	0.10
	P1	0.08	0.26	0.84*	0.29*	0.37*	0.43*	0.05*	0.29	0.22*	0.29
	P2	0.10	0.02	0.27	0.00	0.18	0.01	0.51*	0.07	0.00	0.07
	M1	0.17	0.31*	0.73*	0.26*	0.39*	0.11	0.01*	0.88*	0.13*	0.26
	M2	0.09	0.22	0.62*	0.51*	0.45*	0.16*	0.00	0.49*	0.60*	0.45
	Mdn	0.09	0.22	0.56	0.26	0.39	0.11	0.01	0.30	0.09
Note: *p < 0.05 using Bonferroni correction for multiple testing.

Table 4. Model performances (R ²) for valence across the four genres.

		Prediction
Classical	Film	Pop	Mixed	Median
Training	Classical	0.31	0.16	0.04	0.13	0.15
	Film	0.30	0.58	0.06	0.25	0.28
	Pop	0.26	0.21	0.25	0.15	0.23
	Mixed	0.36	0.40	0.07	0.52	0.38
	Median	0.31	0.31	0.07	0.20

Table 5. Model performances (R ²) for arousal across the datasets.

		Prediction
C1	C2	C3	F1	F2	P1	P2	M1	M2	Mdn
Training	C1	0.96*	0.57*	0.38	0.38*	0.21*	0.45*	0.09*	0.60*	0.25*	0.38
	C2	0.57*	0.88*	0.47*	0.36*	0.16	0.55*	0.12*	0.64*	0.13*	0.47
	C3	0.81*	0.62*	0.94*	0.52*	0.29*	0.13*	0.00	0.44*	0.41*	0.45
	F1	0.75*	0.46*	0.78*	0.83*	0.17	0.75*	0.16*	0.69*	0.36*	0.69
	F2	0.51*	0.15	0.56*	0.47*	0.87*	0.53*	0.05*	0.51*	0.20*	0.51
	P1	0.71*	0.46*	0.10	0.43*	0.26*	0.86*	0.07*	0.81*	0.34*	0.43
	P2	0.70*	0.44*	0.44*	0.40*	0.31*	0.63*	0.29*	0.76*	0.26*	0.44
	M1	0.62*	0.35*	0.65*	0.30*	0.27*	0.73*	0.24*	0.94*	0.23*	0.35
	M2	0.72*	0.68*	0.53*	0.53*	0.45*	0.35*	0.00	0.75*	0.55*	0.55
	Mdn	0.71	0.46	0.53	0.43	0.27	0.55	0.09	0.69	0.26
Note: *p < 0.05 using Bonferroni correction for multiple testing.

Table 6. Model performances (R ²) for arousal across the four genres.

		Prediction
		Classical	Film	Pop	Mixed	Median
Training	Classical	0.69	0.32	0.22	0.41	0.37
	Film	0.54	0.59	0.37	0.44	0.49
	Pop	0.47	0.35	0.46	0.54	0.47
	Mixed	0.59	0.40	0.35	0.62	0.50
	Median	0.57	0.38	0.36	0.49

Table 7. Contribution of individual predictors for valence across pooled excerpts across genre and random forest regression.

Domain	No.	Name	Class.	Film	Pop	Mix	All	Rank
		N	59	150	2658	95	2962
		R ^2	0.21	0.69	0.44	0.56	0.57
Dynamics	1–3	RMS energy			+7/+7		+11/+9	10
	4	Low–Energy		+6	+6
Articul.	5–6	Attack Time				−/+5	−/+6
	7	Attack Slope			+7		+9
Rhythm	8	Event Dens.			+11		+10
	9	Fluct. Peak			+15	+6	+13	6
	10–11	Fluct. Mag.	−/+ 5		+9/+28	+13/−	+13/+18	3
	12–13	Tempo			+11/+17		+8/+12	9
	14–15	Pulse Clar.			+ 6/−	+9/−	+19/−	2
Timbre	16–17	Sp. Centroid		−/+6	+5/+6		+11/+12	8
	18	Sp. Spread			+7		+11
	19–20	Roughness			+6/+7		+9/+8
	21–22	Sp. Flux			+13/+8		+9/+10
	23–24	Regularity		+15/+8			+11/+7
	25–26	Inharmon.		−/+10	+13/+8	+7/−	+7/+10
Pitch	27–28	Pitch		−/+7	+5/+8		−/+10
	29–30	Chromagram		+ 7/−			+8/−
Tonal	31–32	Key Clar.		+ 14/−			+12/−	7
	33	Key Mode	+15	+29	+5	+22	+36	1
	34	HCDF		+8	+7		+10
	35	Sp. Entropy			+8	+8	+7
Structure	36	Rep. (Sp.)		+7	+8		+8
	37	Rep. (Rh.)			+5	+5	+7
	38	Rep. (Ton.)		+11	+8		+16	4
	39	Rep. (Reg.)		+7	+11		+16	5
Note: Only increments over 5% in variance explained are shown.

Table 8. Contribution of individual predictors for arousal across pooled excerpts across genre and random forest regression.

Domain	No.	Name	Class.	Film	Pop	Mix	All	Rank
		N	59	150	2658	95	2962
		R ^2	0.69	0.75	0.58	0.47	0.65
Dynamics	1–3	RMS energy		+6/−	+8/+6		+9/+10
	4	Low–Energy			+8	+6
Articul.	5–6	Attack Time		+6/+8			+7/+10
	7	Attack Slope		+7	+12		+13
Rhythm	8	Event Dens.	+5		+12		+11
	9	Fluct. Peak			+7		+9
	10–11	Fluct. Mag.		+13/−	+23/+17	+5/−	+15/+12	6
	12–13	Tempo	−/+5		−/+14		+6/+10
	14–15	Pulse Clar.	+7/+5	+6/−	+12/−	+9/−	+21/−	1
Timbre	16–17	Sp. Centroid		−/+6	+18/+6		+13/+13
	18	Sp. Spread			+12		+10
	19–20	Roughness		+8/+9	+7/+7	+5/−	+9/+9
	21–22	Sp. Flux	+6/−	+13/+8	+10/+8	+9/−	+14/+12	7
	23–24	Regularity		+12/+8	−/+5		+12/+9
	25–26	Inharmon.	+9/+8	+8/+12	+22/+11		+20/+14	2, 9
Pitch	27–28	Pitch		−/+13	+8/+8	−/+7	−/+16	5
	29–30	Chromagram		−/+5			+9/+6
Tonal	31–32	Key Clar.					+7/−
	33	Key Mode
	34	HCDF			+14
	35	Sp. Entropy	+12	+10	+15	+8	+18	3
Structure	36	Rep. (Sp.)	+8	+12	+9	+5	+13	10
	37	Rep. (Rh.)	+11	+17	+10	+16	+18	4
	38	Rep. (Ton.)		+6	+11		+14	8
	39	Rep. (Reg.)		+5	+7		+11
Note: Only increments over 5% in variance explained are shown.

Gomez , P. and Danuser , B. 2004 . Affective and physiological responses to environmental noises and music . International Journal of Psychophysiology , 53 ( 2 ) : 91 – 103 .

 PubMed Web of Science ®Google Scholar

Bigand , E. , Vieillard , S. , Madurell , F. , Marozeau , J. and Dacquet , A. 2005 . Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts . Cognition & Emotion , 19 ( 8 ) : 1113 – 1139 .

 Web of Science ®Google Scholar

Dibben , N. 2004 . The role of peripheral feedback in emotional experience with music . Music Perception , 22 ( 1 ) : 79 – 115 .

 Web of Science ®Google Scholar

Eerola , T. and Vuoskoski , J. K. 2011 . A comparison of the discrete and dimensional models of emotion in music . Psychology of Music , 39 ( 1 ) : 18 – 49 .

 Web of Science ®Google Scholar

Gosselin , N. , Peretz , I. , Noulhiane , M. , Hasboun , D. , Beckett , C. , Baulac , M. and Samson , S. 2005 . Impaired recognition of scary music following unilateral temporal lobe excision . Brain , 128 ( 3 ) : 628 – 640 .

 PubMedGoogle Scholar

Yang , Y. , Lin , Y. , Su , Y. and Chen , H. Music emotion classification: A regression approach . Proceedings of the IEEE International Confererence on Multimedia . pp. 208 – 211 . Beijing, China

 Google Scholar

Schuller , B. , Dorfner , J. and Rigoll , G. 2010 . Determination of nonprototypical valence and arousal in popular music: Features and performances . EURASIP Journal on Audio, Speech, and Music Processing , 2010 : 1 – 20 .

 Google Scholar

Nater , U. M. , Abbruzzese , E. , Krebs , M. and Ehlert , U. 2006 . Sex differences in emotional and psychophysiological responses to musical stimuli . International Journal of Psychophysiology , 62 ( 2 ) : 300 – 308 .

 PubMed Web of Science ®Google Scholar

Wu , T.-L. and Jeng , S.-K. Automatic emotion classification of musical segments . Proceedings of the 9th International Conference on Music Perception and Cognition . Bologna

 Google Scholar