Abstract
An image is a very effective tool for conveying emotions. Many researchers have investigated emotions in images by using various features extracted from images. In this paper, we focus on two high-level features, the object and the background, and assume that the semantic information in images is a good cue for predicting emotions. An object is one of the most important elements that define an image, and we discover through experiments that there is a high correlation between the objects and emotions in images in most cases. Even with the same object, there may be slight differences in emotion due to different backgrounds, and we use the semantic information of the background to improve the prediction performance. By combining the different levels of features, we build an emotion-based feedforward deep neural network that produces the emotion values of a given image. The output emotion values in our framework are continuous values in two-dimensional space (valence and arousal), which are more effective than using a small number of emotion categories to describe emotions. Experiments confirm the effectiveness of our network in predicting the emotions of images.
Original language | English |
---|---|
Article number | 8344491 |
Pages (from-to) | 2980-2992 |
Number of pages | 13 |
Journal | IEEE Transactions on Multimedia |
Volume | 20 |
Issue number | 11 |
DOIs | |
Publication status | Published - 2018 Nov |
Bibliographical note
Funding Information:Manuscript received July 3, 2017; revised October 15, 2017, January 5, 2018, and March 23, 2018; accepted March 23, 2018. Date of publication April 20, 2018; date of current version October 15, 2018. This work was supported by the Samsung Research Funding Center of Samsung Electronics under Project SRFC-IT1601-04. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Hantao Liu. (Corresponding author: In-Kwon Lee.) The authors are with the Department of Computer Science, Yonsei University, Seoul 03722, South Korea (e-mail:, rinee86@gmail.com; kys71015@ gmail.com; seonjookim@yonsei.ac.kr; iklee@yonsei.ac.kr).
Publisher Copyright:
© 1999-2012 IEEE.
All Science Journal Classification (ASJC) codes
- Signal Processing
- Media Technology
- Computer Science Applications
- Electrical and Electronic Engineering