Temporal resolution vs. visual saliency in videos: Analysis of gaze patterns and evaluation of saliency models

Manri Cheon, Jong Seok Lee

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Temporal scalability of videos refers to the possibility of changing frame rate adaptively for efficient video transmission. Changing the frame rate may alter the spatial location that the viewers pay attention in the scene, which in turn significantly influences human's quality perception. Therefore, in order to effectively exploit the temporal scalability in applications, it is necessary to understand the relationship between frame rate variation and visual saliency. In this study, we answer the following three research questions: (1) Does the frame rate influence the overall gaze patterns (in an average sense over subjects)? (2) Does the frame rate influence the inter-subject variability of the gaze patterns? (3) Do the state-of-the-art saliency models predict human gaze patterns reliably for different frame rates? To answer the first two questions, we conduct an eye-tracking experiment. Under a free viewing scenario, we collect and analyze gaze-paths of human subjects watching high-definition (HD) videos having a normal or low frame rate. Our results show that both the average gaze-path and subject-wise variability of the gaze-path are influenced by frame rate variation. Then, we apply representative state-of-the-art saliency models to the videos and evaluate their performance by using the gaze pattern data collected from the eye-tracking experiment in order to answer the third question. It is shown that there exists a trade-off relation between accuracy in predicting the gaze pattern and robustness to frame rate variation, which raises necessity of further research in saliency modeling to simultaneously achieve both accuracy and robustness.

Original languageEnglish
Pages (from-to)405-417
Number of pages13
JournalSignal Processing: Image Communication
Volume39
DOIs
Publication statusPublished - 2015 Nov

All Science Journal Classification (ASJC) codes

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Temporal resolution vs. visual saliency in videos: Analysis of gaze patterns and evaluation of saliency models'. Together they form a unique fingerprint.

  • Cite this