2D-plus-depth based resolution and frame-rate up-conversion technique for depth video

Jinwook Choi, Dongbo Min, Kwanghoon Sohn

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

We propose a novel framework for upconversion of depth video resolution in both spatial and time domains considering spatial and temporal coherences. Although the Time-of-Flight (TOF) sensor which is widely used in computer vision fields provides depth video in realtime, it also provides a low resolution and a low frame-rate depth video. We propose a cheaper solution that enhances depth video obtained from a TOF sensor by combining it with a Charge-coupled Device (CCD) camera in 3D contents which consist of 2D-plus-depth. Temporal fluctuation problems are also considered for temporally consistent framerate up-conversion. It is important to maintain temporal coherence in depth video, because temporal fluctuation problems may cause eye fatigue and increase bit rates on video coding. We propose a Motion Compensated Frame Interpolation (MCFI) using reliable and rich motion information from a CCD camera and 3-dimensional Joint Bilateral Up-sampling (3D JBU) extended into the temporal domain of depth video. Experimental results show that depth video obtained by the proposed method provides satisfactory quality.

Original languageEnglish
Article number5681132
Pages (from-to)2489-2497
Number of pages9
JournalIEEE Transactions on Consumer Electronics
Volume56
Issue number4
DOIs
Publication statusPublished - 2010 Nov 1

Fingerprint

CCD cameras
Sensors
Image coding
Computer vision
Interpolation
Fatigue of materials
Sampling

All Science Journal Classification (ASJC) codes

  • Media Technology
  • Electrical and Electronic Engineering

Cite this

@article{82abe8d16b834603a8965b6b81c7786f,
title = "2D-plus-depth based resolution and frame-rate up-conversion technique for depth video",
abstract = "We propose a novel framework for upconversion of depth video resolution in both spatial and time domains considering spatial and temporal coherences. Although the Time-of-Flight (TOF) sensor which is widely used in computer vision fields provides depth video in realtime, it also provides a low resolution and a low frame-rate depth video. We propose a cheaper solution that enhances depth video obtained from a TOF sensor by combining it with a Charge-coupled Device (CCD) camera in 3D contents which consist of 2D-plus-depth. Temporal fluctuation problems are also considered for temporally consistent framerate up-conversion. It is important to maintain temporal coherence in depth video, because temporal fluctuation problems may cause eye fatigue and increase bit rates on video coding. We propose a Motion Compensated Frame Interpolation (MCFI) using reliable and rich motion information from a CCD camera and 3-dimensional Joint Bilateral Up-sampling (3D JBU) extended into the temporal domain of depth video. Experimental results show that depth video obtained by the proposed method provides satisfactory quality.",
author = "Jinwook Choi and Dongbo Min and Kwanghoon Sohn",
year = "2010",
month = "11",
day = "1",
doi = "10.1109/TCE.2010.5681132",
language = "English",
volume = "56",
pages = "2489--2497",
journal = "IEEE Transactions on Consumer Electronics",
issn = "0098-3063",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "4",

}

2D-plus-depth based resolution and frame-rate up-conversion technique for depth video. / Choi, Jinwook; Min, Dongbo; Sohn, Kwanghoon.

In: IEEE Transactions on Consumer Electronics, Vol. 56, No. 4, 5681132, 01.11.2010, p. 2489-2497.

Research output: Contribution to journalArticle

TY - JOUR

T1 - 2D-plus-depth based resolution and frame-rate up-conversion technique for depth video

AU - Choi, Jinwook

AU - Min, Dongbo

AU - Sohn, Kwanghoon

PY - 2010/11/1

Y1 - 2010/11/1

N2 - We propose a novel framework for upconversion of depth video resolution in both spatial and time domains considering spatial and temporal coherences. Although the Time-of-Flight (TOF) sensor which is widely used in computer vision fields provides depth video in realtime, it also provides a low resolution and a low frame-rate depth video. We propose a cheaper solution that enhances depth video obtained from a TOF sensor by combining it with a Charge-coupled Device (CCD) camera in 3D contents which consist of 2D-plus-depth. Temporal fluctuation problems are also considered for temporally consistent framerate up-conversion. It is important to maintain temporal coherence in depth video, because temporal fluctuation problems may cause eye fatigue and increase bit rates on video coding. We propose a Motion Compensated Frame Interpolation (MCFI) using reliable and rich motion information from a CCD camera and 3-dimensional Joint Bilateral Up-sampling (3D JBU) extended into the temporal domain of depth video. Experimental results show that depth video obtained by the proposed method provides satisfactory quality.

AB - We propose a novel framework for upconversion of depth video resolution in both spatial and time domains considering spatial and temporal coherences. Although the Time-of-Flight (TOF) sensor which is widely used in computer vision fields provides depth video in realtime, it also provides a low resolution and a low frame-rate depth video. We propose a cheaper solution that enhances depth video obtained from a TOF sensor by combining it with a Charge-coupled Device (CCD) camera in 3D contents which consist of 2D-plus-depth. Temporal fluctuation problems are also considered for temporally consistent framerate up-conversion. It is important to maintain temporal coherence in depth video, because temporal fluctuation problems may cause eye fatigue and increase bit rates on video coding. We propose a Motion Compensated Frame Interpolation (MCFI) using reliable and rich motion information from a CCD camera and 3-dimensional Joint Bilateral Up-sampling (3D JBU) extended into the temporal domain of depth video. Experimental results show that depth video obtained by the proposed method provides satisfactory quality.

UR - http://www.scopus.com/inward/record.url?scp=78651394092&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78651394092&partnerID=8YFLogxK

U2 - 10.1109/TCE.2010.5681132

DO - 10.1109/TCE.2010.5681132

M3 - Article

AN - SCOPUS:78651394092

VL - 56

SP - 2489

EP - 2497

JO - IEEE Transactions on Consumer Electronics

JF - IEEE Transactions on Consumer Electronics

SN - 0098-3063

IS - 4

M1 - 5681132

ER -