Abstract
We propose an intuitive image-to-music retrieval (IMR) framework to improve the user experience on these platforms. The proposed method extracts mood and theme tags by searching for images from a pre-built database that are similar to a query image and then retrieves music with matching tag information. We investigated the system’s effectiveness by comparing participants’ satisfaction, intention to use, and valence between those who interacted with the system and those who did not. We also examined whether using mood or theme attributes affected the user-perceived suitability of the retrieved music. Results showed that all three variables of the interaction group were significantly higher than that of the non-interaction group and that there was no difference in the perceived suitability of music between the mood and theme attributes. Our study concludes that image attributes are effective in successful music retrieval and that interaction is a crucial factor in designing IMR systems.
Disclosure statement
No potential conflict of interest was reported by the author(s).
Data availability statement
The music data that support the findings of this study are openly available in the MTG-Jamendo Dataset at http://hdl.handle.net/10230/42015, and the image data are available from the first author J. Park upon reasonable request.
Correction Statement
This article has been republished with minor changes. These changes do not impact the academic content of the article.
Additional information
Funding
Notes on contributors
Jeongeun Park
Jeongeun Park is currently a PhD candidate at the Graduate School of Information at Yonsei University. She received a BS in electrical and electronic engineering and a master’s degree in information systems from Yonsei University. Her primary research areas are human–computer interaction, generative models, and deep-learning applications.
Minchae Kim
Minchae Kim is a master’s student at the Graduate School of Information at Yonsei University. She obtained her BS degree in law at Sookmyung Women’s University. She is currently conducting research on the application of deep-learning algorithms. Her main research areas are data analysis, image processing, and multimodal learning.
Ha Young Kim
Ha Young Kim is an Associate Professor at the Graduate School of Information at Yonsei University. She received her PhD from the Department of Mathematics at Purdue University. She was a researcher at the Samsung Electronics. Her primary research areas are deep learning, human–computer interaction, and intelligent applications.