TY - JOUR
T1 - Character Detection in Animation Movies Using Multi-Style Adaptation and Visual Attention
AU - Kim, Hayeon
AU - Lee, Eun Cheol
AU - Seo, Yongseok
AU - Im, Donghyuck
AU - Lee, In Kwon
N1 - Publisher Copyright:
IEEE
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
PY - 2020
Y1 - 2020
N2 - Automatic identification of fictional characters is one of the primary analysis techniques for video content. A common approach to detect characters in live-action movies involves detecting human faces; however, this approach cannot be used in non-realistic domains, such as animation movies. Detection of characters in animation movies presents two major challenges: the same subject of character can be expressed in various unique styles and there are no stylistic or other restrictions on the nature and design of character objects. To address these challenges, we introduce the “animation adaptive region-based convolutional neural network (AA-R-CNN)” model to detect characters in animation movies and determine whether the detected characters are human or non-human types. Our model extends the Faster R-CNN model, which is a two-stage object detector, in the following manner: 1) we add a hierarchical animation adaptation (HAA) module to learn the variety of unique styles from animation movies using a single model; 2) we incorporate a double-detector architecture to focus on the regions that are visually important in determining the character class. We build a new dataset for the animated character detection task. Experiments on this dataset show that our model outperforms other existing representative object detector models in terms of character detection. Furthermore, our model achieves significant performance improvements compared with previous state-of-the-art methods used for the character dictionary generation task. Our model is robust for a variety of animation styles and can find common visual representations of all types of characters, providing an effective way to detect animated characters.
AB - Automatic identification of fictional characters is one of the primary analysis techniques for video content. A common approach to detect characters in live-action movies involves detecting human faces; however, this approach cannot be used in non-realistic domains, such as animation movies. Detection of characters in animation movies presents two major challenges: the same subject of character can be expressed in various unique styles and there are no stylistic or other restrictions on the nature and design of character objects. To address these challenges, we introduce the “animation adaptive region-based convolutional neural network (AA-R-CNN)” model to detect characters in animation movies and determine whether the detected characters are human or non-human types. Our model extends the Faster R-CNN model, which is a two-stage object detector, in the following manner: 1) we add a hierarchical animation adaptation (HAA) module to learn the variety of unique styles from animation movies using a single model; 2) we incorporate a double-detector architecture to focus on the regions that are visually important in determining the character class. We build a new dataset for the animated character detection task. Experiments on this dataset show that our model outperforms other existing representative object detector models in terms of character detection. Furthermore, our model achieves significant performance improvements compared with previous state-of-the-art methods used for the character dictionary generation task. Our model is robust for a variety of animation styles and can find common visual representations of all types of characters, providing an effective way to detect animated characters.
UR - http://www.scopus.com/inward/record.url?scp=85087491335&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85087491335&partnerID=8YFLogxK
U2 - 10.1109/TMM.2020.3006372
DO - 10.1109/TMM.2020.3006372
M3 - Article
AN - SCOPUS:85087491335
JO - IEEE Transactions on Multimedia
JF - IEEE Transactions on Multimedia
SN - 1520-9210
ER -