Thermal-to-visible face recognition using partial least squares

Shuowen Hu, Jonghyun Choi, Alex L. Chan, William Robson Schwartz

Research output: Contribution to journalArticlepeer-review

78 Citations (Scopus)


Although visible face recognition has been an active area of research for several decades, cross-modal face recognition has only been explored by the biometrics community relatively recently. Thermal-to-visible face recognition is one of the most difficult cross-modal face recognition challenges, because of the difference in phenomenology between the thermal and visible imaging modalities. We address the cross-modal recognition problem using a partial least squares (PLS) regression-based approach consisting of preprocessing, feature extraction, and PLS model building. The preprocessing and feature extraction stages are designed to reduce the modality gap between the thermal and visible facial signatures, and facilitate the subsequent one-vs-all PLS-based model building. We incorporate multi-modal information into the PLS model building stage to enhance cross-modal recognition. The performance of the proposed recognition algorithm is evaluated on three challenging datasets containing visible and thermal imagery acquired under different experimental scenarios: time-lapse, physical tasks, mental tasks, and subject-to-camera range. These scenarios represent difficult challenges relevant to real-world applications. We demonstrate that the proposed method performs robustly for the examined scenarios.

Original languageEnglish
Pages (from-to)431-442
Number of pages12
JournalJournal of the Optical Society of America A: Optics and Image Science, and Vision
Issue number3
Publication statusPublished - 2015

Bibliographical note

Publisher Copyright:
© 2015 Optical Society of America.

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Thermal-to-visible face recognition using partial least squares'. Together they form a unique fingerprint.

Cite this