CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling

Jinwoo Kim; Sewoong Ahn; Heeseok Oh; Sanghoon Lee

doi:10.1109/ICIP.2019.8803183

CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling

Jinwoo Kim, Sewoong Ahn, Heeseok Oh, Sanghoon Lee

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

In previous quality assessment studies on stereoscopic 3D (S3D) images, researchers have concentrated on deriving manually extracted features which represent the quality of images. These features are based on the human visual system or natural scene statistics, but they have not been revealed as a deterministic function, preventing to guarantee the robustness of features. To solve this problem, we introduce a deep learning method for predicting the quality of S3D images without a reference. A convolutional neural network (CNN) model is trained through two-step learning. First, to overcome the lack of training data, patch-based CNNs are introduced. And then, automatically extracted patch features are pooled into image features. Finally, the trained CNN model parameters are updated iteratively using holistic image labeling, i.e., mean opinion score (MOS). The proposed method represents a significant improvement compared to other no-reference (NR) S3D image quality assessment (IQA) algorithms.

Original language	English
Title of host publication	2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
Publisher	IEEE Computer Society
Pages	1745-1749
Number of pages	5
ISBN (Electronic)	9781538662496
DOIs	https://doi.org/10.1109/ICIP.2019.8803183
Publication status	Published - 2019 Sept
Event	26th IEEE International Conference on Image Processing, ICIP 2019 - Taipei, Taiwan, Province of China Duration: 2019 Sept 22 → 2019 Sept 25

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2019-September
ISSN (Print)	1522-4880

Conference

Conference	26th IEEE International Conference on Image Processing, ICIP 2019
Country/Territory	Taiwan, Province of China
City	Taipei
Period	19/9/22 → 19/9/25

Bibliographical note

Funding Information:
This work was supported by Samsung Research Funding Center of Sam-sung Electronics under Project Number SRFC-IT1702-08

Publisher Copyright:
© 2019 IEEE.

All Science Journal Classification (ASJC) codes

Software
Computer Vision and Pattern Recognition
Signal Processing

Access to Document

10.1109/ICIP.2019.8803183

Cite this

Kim, J., Ahn, S., Oh, H., & Lee, S. (2019). CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling. In 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings (pp. 1745-1749). [8803183] (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September). IEEE Computer Society. https://doi.org/10.1109/ICIP.2019.8803183

@inproceedings{e5f893e784ed4ebe937d3ceb67bc853c,

title = "CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling",

abstract = "In previous quality assessment studies on stereoscopic 3D (S3D) images, researchers have concentrated on deriving manually extracted features which represent the quality of images. These features are based on the human visual system or natural scene statistics, but they have not been revealed as a deterministic function, preventing to guarantee the robustness of features. To solve this problem, we introduce a deep learning method for predicting the quality of S3D images without a reference. A convolutional neural network (CNN) model is trained through two-step learning. First, to overcome the lack of training data, patch-based CNNs are introduced. And then, automatically extracted patch features are pooled into image features. Finally, the trained CNN model parameters are updated iteratively using holistic image labeling, i.e., mean opinion score (MOS). The proposed method represents a significant improvement compared to other no-reference (NR) S3D image quality assessment (IQA) algorithms.",

author = "Jinwoo Kim and Sewoong Ahn and Heeseok Oh and Sanghoon Lee",

note = "Funding Information: This work was supported by Samsung Research Funding Center of Sam-sung Electronics under Project Number SRFC-IT1702-08 Publisher Copyright: {\textcopyright} 2019 IEEE.; 26th IEEE International Conference on Image Processing, ICIP 2019 ; Conference date: 22-09-2019 Through 25-09-2019",

year = "2019",

month = sep,

doi = "10.1109/ICIP.2019.8803183",

language = "English",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE Computer Society",

pages = "1745--1749",

booktitle = "2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings",

address = "United States",

}

Kim, J, Ahn, S, Oh, H & Lee, S 2019, CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling. in 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings., 8803183, Proceedings - International Conference on Image Processing, ICIP, vol. 2019-September, IEEE Computer Society, pp. 1745-1749, 26th IEEE International Conference on Image Processing, ICIP 2019, Taipei, Taiwan, Province of China, 19/9/22. https://doi.org/10.1109/ICIP.2019.8803183

CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling. / Kim, Jinwoo; Ahn, Sewoong; Oh, Heeseok et al.

2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings. IEEE Computer Society, 2019. p. 1745-1749 8803183 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling

AU - Kim, Jinwoo

AU - Ahn, Sewoong

AU - Oh, Heeseok

AU - Lee, Sanghoon

PY - 2019/9

Y1 - 2019/9

N2 - In previous quality assessment studies on stereoscopic 3D (S3D) images, researchers have concentrated on deriving manually extracted features which represent the quality of images. These features are based on the human visual system or natural scene statistics, but they have not been revealed as a deterministic function, preventing to guarantee the robustness of features. To solve this problem, we introduce a deep learning method for predicting the quality of S3D images without a reference. A convolutional neural network (CNN) model is trained through two-step learning. First, to overcome the lack of training data, patch-based CNNs are introduced. And then, automatically extracted patch features are pooled into image features. Finally, the trained CNN model parameters are updated iteratively using holistic image labeling, i.e., mean opinion score (MOS). The proposed method represents a significant improvement compared to other no-reference (NR) S3D image quality assessment (IQA) algorithms.

AB - In previous quality assessment studies on stereoscopic 3D (S3D) images, researchers have concentrated on deriving manually extracted features which represent the quality of images. These features are based on the human visual system or natural scene statistics, but they have not been revealed as a deterministic function, preventing to guarantee the robustness of features. To solve this problem, we introduce a deep learning method for predicting the quality of S3D images without a reference. A convolutional neural network (CNN) model is trained through two-step learning. First, to overcome the lack of training data, patch-based CNNs are introduced. And then, automatically extracted patch features are pooled into image features. Finally, the trained CNN model parameters are updated iteratively using holistic image labeling, i.e., mean opinion score (MOS). The proposed method represents a significant improvement compared to other no-reference (NR) S3D image quality assessment (IQA) algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85076799073&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85076799073&partnerID=8YFLogxK

U2 - 10.1109/ICIP.2019.8803183

DO - 10.1109/ICIP.2019.8803183

M3 - Conference contribution

AN - SCOPUS:85076799073

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 1745

EP - 1749

BT - 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings

PB - IEEE Computer Society

T2 - 26th IEEE International Conference on Image Processing, ICIP 2019

Y2 - 22 September 2019 through 25 September 2019

ER -

CNN-Based Blind Quality Prediction on Stereoscopic Images Via Patch to Image Feature Pooling

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this