Ambiguity of objective image quality metrics: A new methodology for performance evaluation

Manri Cheon, Toinon Vigier, Lukáš Krasula, Junghyuk Lee, Patrick Le Callet, Jong Seok Lee

Research output: Contribution to journalArticlepeer-review

2 Citations (Scopus)


Objective image quality metrics try to estimate the perceptual quality of the given image by considering the characteristics of the human visual system. However, it is possible that the metrics produce different quality scores even for two images that are perceptually indistinguishable by human viewers, which have not been considered in the existing studies related to objective quality assessment. In this paper, we address the issue of ambiguity of objective image quality assessment. We propose an approach to obtain an ambiguity interval of an objective metric, within which the quality score difference is not perceptually significant. In particular, we use the visual difference predictor, which can consider viewing conditions that are important for visual quality perception. In order to demonstrate the usefulness of the proposed approach, we conduct experiments with 33 state-of-the-art image quality metrics in the viewpoint of their accuracy and ambiguity for three image quality databases. The results show that the ambiguity intervals can be applied as an additional figure of merit when conventional performance measurement does not determine superiority between the metrics. The effect of the viewing distance on the ambiguity interval is also shown.

Original languageEnglish
Article number116150
JournalSignal Processing: Image Communication
Publication statusPublished - 2021 Apr

Bibliographical note

Funding Information:
This research was supported by the Ministry of Science and ICT (MSIT), Korea, under the “ICT Consilience Creative Program” ( IITP-2018-2017-0-01015 ) supervised by the Institute for Information & communications Technology Promotion (IITP) and also by the IITP grant funded by the Korea government (MSIT) ( R7124-16-0004 , Development of Intelligent Interaction Technology Based on Context Awareness and Human Intention Understanding).

Publisher Copyright:
© 2021 Elsevier B.V.

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering


Dive into the research topics of 'Ambiguity of objective image quality metrics: A new methodology for performance evaluation'. Together they form a unique fingerprint.

Cite this