Conditional Multimodal Biometrics Embedding Learning For Periocular and Face in the Wild

Multimodal biometrics has been attributed to achieving better performance compared to unimodal biometrics, despite there being some limitations on its utilization e.g. availability, deployment cost, templates management, etc. In this paper, we revolve around a generalized multimodal biometrics notion, which we coin as Conditional Multimodal Biometrics (CMB). The CMB is substantiated by a learning model which is trained with N multimodal biometrics. During enrollment and query, the trained CMB model is utilized as a feature encoder to transform any x biometric raw input(s) yielding x reference and query instances, respectively, where 1≤x≤ N. Depending on application needs, multimodal biometrics system enjoys better performance by deploying either a single biometrics, a subset, or all N modalities. As a means of realization, we consider face and periocular biometrics and propose a deep CMB network, known as CMB-Net. The CMB-Net is composed of two predictors corresponding to face and periocular with a shared-parameter convolutional backbone. Apart from classification losses for each face and periocular, a CMB loss with regularization is devised to attract periocular-face intra-subject feature embeddings and repel periocular-face inter-subject feature embeddings, whilst each face and periocular regulates one another throughout CMB-Net training. We scrutinize three CMB configurations, namely periocular conditioned by face, face conditioned by periocular and periocular-face, under the CMB regimen. Our experimental results on five periocular-face in the wild datasets demonstrate that all three CMB configurations outperform their respective baselines under both identification and verification modes.