Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network

Seonghyeon Nam, Seon Joo Kim

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAWimages from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a datadriven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.

Original languageEnglish
Title of host publicationProceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1726-1734
Number of pages9
ISBN (Electronic)9781538610329
DOIs
Publication statusPublished - 2017 Dec 22
Event16th IEEE International Conference on Computer Vision, ICCV 2017 - Venice, Italy
Duration: 2017 Oct 222017 Oct 29

Publication series

NameProceedings of the IEEE International Conference on Computer Vision
Volume2017-October
ISSN (Print)1550-5499

Other

Other16th IEEE International Conference on Computer Vision, ICCV 2017
CountryItaly
CityVenice
Period17/10/2217/10/29

Fingerprint

Cameras
Imaging techniques
Image processing
Color
Computer vision
Pipelines
Pixels
Calibration
Deep neural networks
Deep learning

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition

Cite this

Nam, S., & Kim, S. J. (2017). Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017 (pp. 1726-1734). [8237452] (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICCV.2017.190
Nam, Seonghyeon ; Kim, Seon Joo. / Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network. Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 1726-1734 (Proceedings of the IEEE International Conference on Computer Vision).
@inproceedings{3e68341da8074d3f9c84fa70b1bba2e7,
title = "Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network",
abstract = "We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAWimages from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a datadriven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.",
author = "Seonghyeon Nam and Kim, {Seon Joo}",
year = "2017",
month = "12",
day = "22",
doi = "10.1109/ICCV.2017.190",
language = "English",
series = "Proceedings of the IEEE International Conference on Computer Vision",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "1726--1734",
booktitle = "Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017",
address = "United States",

}

Nam, S & Kim, SJ 2017, Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network. in Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017., 8237452, Proceedings of the IEEE International Conference on Computer Vision, vol. 2017-October, Institute of Electrical and Electronics Engineers Inc., pp. 1726-1734, 16th IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, 17/10/22. https://doi.org/10.1109/ICCV.2017.190

Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network. / Nam, Seonghyeon; Kim, Seon Joo.

Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 1726-1734 8237452 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2017-October).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network

AU - Nam, Seonghyeon

AU - Kim, Seon Joo

PY - 2017/12/22

Y1 - 2017/12/22

N2 - We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAWimages from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a datadriven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.

AB - We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAWimages from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a datadriven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.

UR - http://www.scopus.com/inward/record.url?scp=85041899249&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85041899249&partnerID=8YFLogxK

U2 - 10.1109/ICCV.2017.190

DO - 10.1109/ICCV.2017.190

M3 - Conference contribution

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 1726

EP - 1734

BT - Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Nam S, Kim SJ. Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network. In Proceedings - 2017 IEEE International Conference on Computer Vision, ICCV 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 1726-1734. 8237452. (Proceedings of the IEEE International Conference on Computer Vision). https://doi.org/10.1109/ICCV.2017.190