Multi-face tracking in unconstrained videos is a challenging problem as faces of one person often appear drastically different in multiple shots due to significant variations in scale, pose, expression, illumination, and make-up. Low-level features used in existing multitarget tracking methods are not effective for identifying faces with such large appearance variations. In this paper, we tackle this problem by learning discriminative, video-specific face features using convolutional neural networks (CNNs). Unlike existing CNN-based approaches that are only trained on large-scale face image datasets offline, we further adapt the pre-trained face CNN to specific videos using automatically discovered training samples from tracklets. Our network directly optimizes the embedding space so that the Euclidean distances correspond to a measure of semantic face similarity. This is technically realized by minimizing an improved triplet loss function. With the learned discriminative features, we apply the Hungarian algorithm to link tracklets within each shot and the hierarchical clustering algorithm to link tracklets across multiple shots to form final trajectories. We extensively evaluate the proposed algorithm on a set of TV sitcoms and music videos and demonstrate significant performance improvement over existing techniques.
|Title of host publication||Computer Vision - 14th European Conference, ECCV 2016, Proceedings|
|Editors||Bastian Leibe, Jiri Matas, Nicu Sebe, Max Welling|
|Number of pages||19|
|Publication status||Published - 2016|
|Event||14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands|
Duration: 2016 Oct 8 → 2016 Oct 16
|Name||Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)|
|Conference||14th European Conference on Computer Vision, ECCV 2016|
|Period||16/10/8 → 16/10/16|
Bibliographical noteFunding Information:
The work is partially supported by National Basic Research Program of China (973 Program, 2015CB351705), NSFC (61332018), Office of Naval Research (N0014-16-1-2314), R&D programs by NRF (2014R1A1A2058501) and MSIP/IITP (IITP-2016-H8601-16-1005) of Korea, NSF CAREER (1149783) and gifts from Adobe and NVIDIA.
© Springer International Publishing AG 2016.
All Science Journal Classification (ASJC) codes
- Theoretical Computer Science
- Computer Science(all)