Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System

Hanbyol Jang, Kihun Bang, Jinseong Jang, Do Sik Hwang

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Conventional digital displays show images with much less dynamic range than that of human visual perception. High-dynamic range (HDR) displays are being developed for viewing images with higher dynamic range than that of conventional low-dynamic range (LDR) images. However, to view existing LDR images on HDR displays, an inverse tone mapping operator (ITMO), which is a process to extend the dynamic range of LDR images, is required. we propose an adaptive ITMO by effectively learning the differences between LDR and HDR images using a sequential learning process: dynamic range learning followed by color difference learning. Our proposed method enables visualization of colors similar to real-world colors better than conventional ITMOs by learning color differences based on the human visual system properties. For the objective comparison, seven different evaluation metrics optimized for HDR image evaluation were used. Our method resulted in 10%-25% improved HDR-visual difference predictor-2.2 values over those of other ITMOs. Other metrics also demonstrated the superior performance of our method. For the subjective comparison, eight human observers evaluated the estimated HDR images in terms of color appearance and overall preferences. Our method received an average 4.7 out of 5 score, whereas other ITMOs received below 3.5 scores in the evaluation of color appearance. The objective and subjective evaluations' results showed that our proposed method outperformed the conventional ITMOs in estimating the dynamic range and color appearance of ground-truth HDR images.

Original languageEnglish
Article number8466589
Pages (from-to)52058-52072
Number of pages15
JournalIEEE Access
Volume6
DOIs
Publication statusPublished - 2018 Sep 14

Fingerprint

Color
Deep neural networks
Display devices
Visualization

All Science Journal Classification (ASJC) codes

  • Computer Science(all)
  • Materials Science(all)
  • Engineering(all)

Cite this

Jang, Hanbyol ; Bang, Kihun ; Jang, Jinseong ; Hwang, Do Sik. / Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System. In: IEEE Access. 2018 ; Vol. 6. pp. 52058-52072.
@article{f8025fa8f77e40dcb502d50a67966a2e,
title = "Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System",
abstract = "Conventional digital displays show images with much less dynamic range than that of human visual perception. High-dynamic range (HDR) displays are being developed for viewing images with higher dynamic range than that of conventional low-dynamic range (LDR) images. However, to view existing LDR images on HDR displays, an inverse tone mapping operator (ITMO), which is a process to extend the dynamic range of LDR images, is required. we propose an adaptive ITMO by effectively learning the differences between LDR and HDR images using a sequential learning process: dynamic range learning followed by color difference learning. Our proposed method enables visualization of colors similar to real-world colors better than conventional ITMOs by learning color differences based on the human visual system properties. For the objective comparison, seven different evaluation metrics optimized for HDR image evaluation were used. Our method resulted in 10{\%}-25{\%} improved HDR-visual difference predictor-2.2 values over those of other ITMOs. Other metrics also demonstrated the superior performance of our method. For the subjective comparison, eight human observers evaluated the estimated HDR images in terms of color appearance and overall preferences. Our method received an average 4.7 out of 5 score, whereas other ITMOs received below 3.5 scores in the evaluation of color appearance. The objective and subjective evaluations' results showed that our proposed method outperformed the conventional ITMOs in estimating the dynamic range and color appearance of ground-truth HDR images.",
author = "Hanbyol Jang and Kihun Bang and Jinseong Jang and Hwang, {Do Sik}",
year = "2018",
month = "9",
day = "14",
doi = "10.1109/ACCESS.2018.2870295",
language = "English",
volume = "6",
pages = "52058--52072",
journal = "IEEE Access",
issn = "2169-3536",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System. / Jang, Hanbyol; Bang, Kihun; Jang, Jinseong; Hwang, Do Sik.

In: IEEE Access, Vol. 6, 8466589, 14.09.2018, p. 52058-52072.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Inverse Tone Mapping Operator Using Sequential Deep Neural Networks Based on the Human Visual System

AU - Jang, Hanbyol

AU - Bang, Kihun

AU - Jang, Jinseong

AU - Hwang, Do Sik

PY - 2018/9/14

Y1 - 2018/9/14

N2 - Conventional digital displays show images with much less dynamic range than that of human visual perception. High-dynamic range (HDR) displays are being developed for viewing images with higher dynamic range than that of conventional low-dynamic range (LDR) images. However, to view existing LDR images on HDR displays, an inverse tone mapping operator (ITMO), which is a process to extend the dynamic range of LDR images, is required. we propose an adaptive ITMO by effectively learning the differences between LDR and HDR images using a sequential learning process: dynamic range learning followed by color difference learning. Our proposed method enables visualization of colors similar to real-world colors better than conventional ITMOs by learning color differences based on the human visual system properties. For the objective comparison, seven different evaluation metrics optimized for HDR image evaluation were used. Our method resulted in 10%-25% improved HDR-visual difference predictor-2.2 values over those of other ITMOs. Other metrics also demonstrated the superior performance of our method. For the subjective comparison, eight human observers evaluated the estimated HDR images in terms of color appearance and overall preferences. Our method received an average 4.7 out of 5 score, whereas other ITMOs received below 3.5 scores in the evaluation of color appearance. The objective and subjective evaluations' results showed that our proposed method outperformed the conventional ITMOs in estimating the dynamic range and color appearance of ground-truth HDR images.

AB - Conventional digital displays show images with much less dynamic range than that of human visual perception. High-dynamic range (HDR) displays are being developed for viewing images with higher dynamic range than that of conventional low-dynamic range (LDR) images. However, to view existing LDR images on HDR displays, an inverse tone mapping operator (ITMO), which is a process to extend the dynamic range of LDR images, is required. we propose an adaptive ITMO by effectively learning the differences between LDR and HDR images using a sequential learning process: dynamic range learning followed by color difference learning. Our proposed method enables visualization of colors similar to real-world colors better than conventional ITMOs by learning color differences based on the human visual system properties. For the objective comparison, seven different evaluation metrics optimized for HDR image evaluation were used. Our method resulted in 10%-25% improved HDR-visual difference predictor-2.2 values over those of other ITMOs. Other metrics also demonstrated the superior performance of our method. For the subjective comparison, eight human observers evaluated the estimated HDR images in terms of color appearance and overall preferences. Our method received an average 4.7 out of 5 score, whereas other ITMOs received below 3.5 scores in the evaluation of color appearance. The objective and subjective evaluations' results showed that our proposed method outperformed the conventional ITMOs in estimating the dynamic range and color appearance of ground-truth HDR images.

UR - http://www.scopus.com/inward/record.url?scp=85053348568&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85053348568&partnerID=8YFLogxK

U2 - 10.1109/ACCESS.2018.2870295

DO - 10.1109/ACCESS.2018.2870295

M3 - Article

AN - SCOPUS:85053348568

VL - 6

SP - 52058

EP - 52072

JO - IEEE Access

JF - IEEE Access

SN - 2169-3536

M1 - 8466589

ER -