Learning fair representation is crucial for achieving fairness or debiasing sensitive information. Most existing works rely on adversarial representation learning to inject some invariance into representation. However, adversarial learning methods are known to suffer from relatively unstable training, and this might harm the balance between fairness and predictiveness of representation. We propose a new approach, learningFAir Representation via distributional CONtrastive Variational AutoEncoder (FarconVAE), which induces the latent space to be disentangled into sensitive and non-sensitive parts. We first construct the pair of observations with different sensitive attributes but with the same labels. Then, FarconVAE enforces each non-sensitive latent to be closer, while sensitive latents to be far from each other and also far from the non-sensitive latent by contrasting their distributions. We provide a new type of contrastive loss motivated by Gaussian and Student-t kernels for distributional contrastive learning with theoretical analysis. Besides, we adopt a new swap-reconstruction loss to boost the disentanglement further. FarconVAE shows superior performance on fairness, pretrained model debiasing, and domain generalization tasks from various modalities, including tabular, image, and text.
|Title of host publication||KDD 2022 - Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining|
|Publisher||Association for Computing Machinery|
|Number of pages||11|
|Publication status||Published - 2022 Aug 14|
|Event||28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022 - Washington, United States|
Duration: 2022 Aug 14 → 2022 Aug 18
|Name||Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining|
|Conference||28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2022|
|Period||22/8/14 → 22/8/18|
Bibliographical noteFunding Information:
This work was partly supported by Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2021-0-02067,Next generation AI for multi-purpose video search,50%) and the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT). (No. 2021R1F1A1060117, Research and Application of Artificial Intelligence Algorithm for Removing Spurious Bias,50%)
© 2022 ACM.
All Science Journal Classification (ASJC) codes
- Information Systems