Relational Deep Feature Learning for Heterogeneous Face Recognition

Myeong Ah Cho, Taeoh Kim, Ig Jae Kim, Kyungjae Lee, Sangyoun Lee

Research output: Contribution to journalArticlepeer-review

18 Citations (Scopus)


Heterogeneous Face Recognition (HFR) is a task that matches faces across two different domains such as visible light (VIS), near-infrared (NIR), or the sketch domain. Due to the lack of databases, HFR methods usually exploit the pre-trained features on a large-scale visual database that contain general facial information. However, these pre-trained features cause performance degradation due to the texture discrepancy with the visual domain. With this motivation, we propose a graph-structured module called Relational Graph Module (RGM) that extracts global relational information in addition to general facial features. Because each identity's relational information between intra-facial parts is similar in any modality, the modeling relationship between features can help cross-domain matching. Through the RGM, relation propagation diminishes texture dependency without losing its advantages from the pre-trained features. Furthermore, the RGM captures global facial geometrics from locally correlated convolutional features to identify long-range relationships. In addition, we propose a Node Attention Unit (NAU) that performs node-wise recalibration to concentrate on the more informative nodes arising from relation-based propagation. Furthermore, we suggest a novel conditional-margin loss function (C-softmax) for the efficient projection learning of the embedding vector in HFR. The proposed method outperforms other state-of-the-art methods on five HFR databases. Furthermore, we demonstrate performance improvement on three backbones because our module can be plugged into any pre-trained face recognition backbone to overcome the limitations of a small HFR database.

Original languageEnglish
Article number9153000
Pages (from-to)376-388
Number of pages13
JournalIEEE Transactions on Information Forensics and Security
Publication statusPublished - 2021

Bibliographical note

Funding Information:
Manuscript received January 6, 2020; revised April 17, 2020 and June 22, 2020; accepted July 18, 2020. Date of publication July 30, 2020; date of current version August 12, 2020. This work was supported by the Research and Development Program for Advanced Integrated-Intelligence for Identification (AIID) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT under Grant NRF-2018M3E3A1057289. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Domingo Mery. (Corresponding author: Sangyoun Lee.) MyeongAh Cho, Taeoh Kim, and Sangyoun Lee are with the School of Electrical and Electronic Engineering, Yonsei University, Seoul 03722, South Korea (e-mail:;;

Publisher Copyright:
© 2005-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Safety, Risk, Reliability and Quality
  • Computer Networks and Communications


Dive into the research topics of 'Relational Deep Feature Learning for Heterogeneous Face Recognition'. Together they form a unique fingerprint.

Cite this