A new analysis method for paired comparison and its application to 3D quality assessment

Jong Seok Lee, Lutz Goldmann, Touradj Ebrahimi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Citations (Scopus)

Abstract

Among various subjective quality evaluation methodologies, paired comparison has the advantage of improved simplicity of the subjects'evaluation task due to simplified rating scales and direct comparison of two stimuli. Thus, it may lead to more reliable results when individual quality levels are difficult to define, quality differences between stimuli are small or multiple quality factors are involved. This paper proposes a new method to analyze results of paired comparison-based subjective tests. By assuming that ties convey information about significant differences between two stimuli being compared, the confidence intervals for the quality scores are estimated using a maximum likelihood criterion, which enables us to intuitively examine the significance of quality score differences. We describe the complete test methodology including the test procedure, outlier detection and score analysis applied to quality assessment of 3D images acquired using varying camera distances. Experimental results demonstrate the usefulness of the proposed analysis method, as well as the enhanced quality discriminability of the paired comparison methodology in comparison to the conventional single stimulus methodology.

Original languageEnglish
Title of host publicationMM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops
Pages1281-1284
Number of pages4
DOIs
Publication statusPublished - 2011 Dec 29
Event19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11 - Scottsdale, AZ, United States
Duration: 2011 Nov 282011 Dec 1

Publication series

NameMM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops

Other

Other19th ACM International Conference on Multimedia ACM Multimedia 2011, MM'11
CountryUnited States
CityScottsdale, AZ
Period11/11/2811/12/1

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction

Fingerprint Dive into the research topics of 'A new analysis method for paired comparison and its application to 3D quality assessment'. Together they form a unique fingerprint.

  • Cite this

    Lee, J. S., Goldmann, L., & Ebrahimi, T. (2011). A new analysis method for paired comparison and its application to 3D quality assessment. In MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops (pp. 1281-1284). (MM'11 - Proceedings of the 2011 ACM Multimedia Conference and Co-Located Workshops). https://doi.org/10.1145/2072298.2071994