Fair-VQA: Fairness-Aware Visual Question Answering through Sensitive Attribute Prediction

Sungho Park, Sunhee Hwang, Jongkwang Hong, Hyeran Byun

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


Visual Question Answering (VQA) is a task that answers questions on given images. Although previous works achieve a great improvement in VQA performance, they do not consider the fairness of answers in terms of ethically sensitive attributes, such as gender. Therefore, we propose a Fair-VQA model that contains two modules: VQA module and SAP (Sensitive Attribute Prediction) module. On top of VQA module, which predicts various kinds of answers, SAP module predicts only sensitive attributes using the same inputs. The predictions of SAP module are utilized to rectify answers from VQA module to be fairer in terms of the sensitive attributes with graceful performance degradation. To validate the proposed method, we conduct extensive experiments on VQA, GQA, and our proposing VQA-Gender datasets. In all the experiments, our method shows the fairest results in various metrics for fairness. Moreover, we demonstrate that our method works interpretably through the analysis of visualized attention maps.

Original languageEnglish
Article number9274341
Pages (from-to)215091-215099
Number of pages9
JournalIEEE Access
Publication statusPublished - 2020

Bibliographical note

Funding Information:
This work was supported in part by the National Research Foundation of Korea (NRF) Grant funded by the Korean Government (MSIT) under Grant 2019R1A2C2003760, and in part by the Institute for Information & Communications Technology Planning & Evaluation (IITP) Grant funded by the Korean Government (MSIT) (Development of framework for analyzing, detecting, mitigating of bias in AI model, and training data) (Artificial Intelligence Graduate School Program (Yonsei University)) under Grant 2019-0-01396 and Grant 2020-0-01361.

Publisher Copyright:
© 2013 IEEE.

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)
  • Electrical and Electronic Engineering


Dive into the research topics of 'Fair-VQA: Fairness-Aware Visual Question Answering through Sensitive Attribute Prediction'. Together they form a unique fingerprint.

Cite this