Abstract
RGBW (RGB-White) CFAs (Color Filter Arrays) which contain white pixels as well as RGB pixels have been actively proposed, which is due to the higher SNR (Signal to Noise Ratio) characteristics of the white pixels. The higher SNR characteristics of the white pixels results in the reconstruction of color images with higher resolutions than those reconstructed with widely used RGB CFAs, especially in low illumination environments. Recently, we proposed a new RGBW demosaicking method which uses an RGBW CFA where the white pixels cover 75% of the CFA, and which reconstructs the color image from the RGBW pattern image by use of the colorization method. The use of a large number of white pixels, and the use of colorization as an interpolation method made it possible to reconstruct a color image with high SNR values. In this paper, we extend this method to reconstruct video sequences by extending the colorization method to the 3-D domain, i.e., we propose a 3-D color reconstruction method which takes an RGBW pattern video sequence as the input and produces a reconstructed color video sequence. We implement the colorization as a 3-D diffusion instead of solving an overall colorization matrix, since the computation is high for the 3-D domain. The 3-D diffusion flow stops at temporal edges, which prevents the false colorizing of moving objects. Furthermore, the 3-D diffusion flow also eliminates the incoherence between colors of neighboring frames, which eliminates the flickering artifact. One major difficulty in extending to the temporal axis is that the noise in the few color seeds can propagate throughout the 3-D domain resulting in many false colors. This is especially important in the case when the image/video is taken in low illumination, since the energy of the thermal noise becomes relatively large compared to the light energy making the RGBW pattern image/video very noisy. To prevent the propagation of false colors caused by the noise in the color seeds, we use the 3-D stacked white channels as a guidance for denoising, so that the denoising takes into account only the color seeds that belong to the same object region in the 3-D space. The stacked white channels also guide the 3-D diffusion throughout the 3-D space. Experimental results show that the image/video reconstructed by the proposed method has better visual quality than those reconstructed by conventional RGB/RGBW CFA based methods. Therefore, the proposed method can be used in surveillance camera applications to recover important information from noisy video sequences taken in very low illumination environment.
Original language | English |
---|---|
Pages (from-to) | 87-101 |
Number of pages | 15 |
Journal | Digital Signal Processing: A Review Journal |
Volume | 93 |
DOIs | |
Publication status | Published - 2019 Oct |
Fingerprint
All Science Journal Classification (ASJC) codes
- Signal Processing
- Computer Vision and Pattern Recognition
- Statistics, Probability and Uncertainty
- Computational Theory and Mathematics
- Electrical and Electronic Engineering
- Artificial Intelligence
- Applied Mathematics
Cite this
}
Three dimensional colorization based image/video reconstruction from white-dominant RGBW pattern images. / Lee, Suk ho; Oh, Paul; Kang, Moon Gi.
In: Digital Signal Processing: A Review Journal, Vol. 93, 10.2019, p. 87-101.Research output: Contribution to journal › Article
TY - JOUR
T1 - Three dimensional colorization based image/video reconstruction from white-dominant RGBW pattern images
AU - Lee, Suk ho
AU - Oh, Paul
AU - Kang, Moon Gi
PY - 2019/10
Y1 - 2019/10
N2 - RGBW (RGB-White) CFAs (Color Filter Arrays) which contain white pixels as well as RGB pixels have been actively proposed, which is due to the higher SNR (Signal to Noise Ratio) characteristics of the white pixels. The higher SNR characteristics of the white pixels results in the reconstruction of color images with higher resolutions than those reconstructed with widely used RGB CFAs, especially in low illumination environments. Recently, we proposed a new RGBW demosaicking method which uses an RGBW CFA where the white pixels cover 75% of the CFA, and which reconstructs the color image from the RGBW pattern image by use of the colorization method. The use of a large number of white pixels, and the use of colorization as an interpolation method made it possible to reconstruct a color image with high SNR values. In this paper, we extend this method to reconstruct video sequences by extending the colorization method to the 3-D domain, i.e., we propose a 3-D color reconstruction method which takes an RGBW pattern video sequence as the input and produces a reconstructed color video sequence. We implement the colorization as a 3-D diffusion instead of solving an overall colorization matrix, since the computation is high for the 3-D domain. The 3-D diffusion flow stops at temporal edges, which prevents the false colorizing of moving objects. Furthermore, the 3-D diffusion flow also eliminates the incoherence between colors of neighboring frames, which eliminates the flickering artifact. One major difficulty in extending to the temporal axis is that the noise in the few color seeds can propagate throughout the 3-D domain resulting in many false colors. This is especially important in the case when the image/video is taken in low illumination, since the energy of the thermal noise becomes relatively large compared to the light energy making the RGBW pattern image/video very noisy. To prevent the propagation of false colors caused by the noise in the color seeds, we use the 3-D stacked white channels as a guidance for denoising, so that the denoising takes into account only the color seeds that belong to the same object region in the 3-D space. The stacked white channels also guide the 3-D diffusion throughout the 3-D space. Experimental results show that the image/video reconstructed by the proposed method has better visual quality than those reconstructed by conventional RGB/RGBW CFA based methods. Therefore, the proposed method can be used in surveillance camera applications to recover important information from noisy video sequences taken in very low illumination environment.
AB - RGBW (RGB-White) CFAs (Color Filter Arrays) which contain white pixels as well as RGB pixels have been actively proposed, which is due to the higher SNR (Signal to Noise Ratio) characteristics of the white pixels. The higher SNR characteristics of the white pixels results in the reconstruction of color images with higher resolutions than those reconstructed with widely used RGB CFAs, especially in low illumination environments. Recently, we proposed a new RGBW demosaicking method which uses an RGBW CFA where the white pixels cover 75% of the CFA, and which reconstructs the color image from the RGBW pattern image by use of the colorization method. The use of a large number of white pixels, and the use of colorization as an interpolation method made it possible to reconstruct a color image with high SNR values. In this paper, we extend this method to reconstruct video sequences by extending the colorization method to the 3-D domain, i.e., we propose a 3-D color reconstruction method which takes an RGBW pattern video sequence as the input and produces a reconstructed color video sequence. We implement the colorization as a 3-D diffusion instead of solving an overall colorization matrix, since the computation is high for the 3-D domain. The 3-D diffusion flow stops at temporal edges, which prevents the false colorizing of moving objects. Furthermore, the 3-D diffusion flow also eliminates the incoherence between colors of neighboring frames, which eliminates the flickering artifact. One major difficulty in extending to the temporal axis is that the noise in the few color seeds can propagate throughout the 3-D domain resulting in many false colors. This is especially important in the case when the image/video is taken in low illumination, since the energy of the thermal noise becomes relatively large compared to the light energy making the RGBW pattern image/video very noisy. To prevent the propagation of false colors caused by the noise in the color seeds, we use the 3-D stacked white channels as a guidance for denoising, so that the denoising takes into account only the color seeds that belong to the same object region in the 3-D space. The stacked white channels also guide the 3-D diffusion throughout the 3-D space. Experimental results show that the image/video reconstructed by the proposed method has better visual quality than those reconstructed by conventional RGB/RGBW CFA based methods. Therefore, the proposed method can be used in surveillance camera applications to recover important information from noisy video sequences taken in very low illumination environment.
UR - http://www.scopus.com/inward/record.url?scp=85069829836&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85069829836&partnerID=8YFLogxK
U2 - 10.1016/j.dsp.2019.07.012
DO - 10.1016/j.dsp.2019.07.012
M3 - Article
AN - SCOPUS:85069829836
VL - 93
SP - 87
EP - 101
JO - Digital Signal Processing: A Review Journal
JF - Digital Signal Processing: A Review Journal
SN - 1051-2004
ER -