This paper presents a foveated coding method using audio-visual focus of attention and its evaluation through extensive subjective experiments on both standard-definition and high-definition sequences. Regarding a sound-emitting region as the location drawing the human attention, the method applies varying quality levels in an image frame according to the distance of a pixel to the identified sound source. Two experiments are presented to prove the efficiency of the method. Experiment 1 examines the validity and effectiveness of the method in comparison to the constant quality coding for high-quality conditions. In Experiment 2, the method is compared to the fixed bit rate coding for low quality conditions where coding artifacts are noticeable. The results demonstrate that the foveated coding method provides considerable coding gain without significant quality degradation, but uneven distributions of the coding artifacts (blockiness) by the method are often less preferred than the uniform distribution of the artifacts. Additional interesting findings are also discussed, such as content dependence of the performance of the method, the memory effect in multiple viewings, and the difference in the quality perception for frame size variations.
|Number of pages||10|
|Journal||IEEE Journal on Selected Topics in Signal Processing|
|Publication status||Published - 2011 Nov|
Bibliographical noteFunding Information:
Manuscript received February 01, 2011; revised July 11, 2011 and August 08, 2011; accepted August 09, 2011. Date of publication August 18, 2011; date of current version October 19, 2011. This work was supported in part by the European Community’s Seventh Framework Programme (FP7/2007-2011) under Grant 216444 (PetaMedia), and in part by the Swiss NCCR Interactive Multimodal Information Management (IM2). The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Edward J. Delp.
All Science Journal Classification (ASJC) codes
- Signal Processing
- Electrical and Electronic Engineering