3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services

Young Suk Yoon; Sangwon Hwang; Heansung Lee; Sangyoun Lee

doi:10.1109/ICTC52510.2021.9620779

3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services

Young Suk Yoon, Sangwon Hwang, Heansung Lee, Sangyoun Lee

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper, we propose a system that transforms the 3D space of the real world into a 3D mesh to provide augmented reality(AR) services. First, the proposed system acquires a 2D RGB image sequence using a monocular camera in order to model the real world in 3D data. and it creates a 3D point cloud with 3D spatial information of the real world using the obtained RGB image sequence. Also, it segments objects that appear in the RGB image sequence into individual objects. Next, the proposed system tracks the segmented multiple objects and connects identification information of the objects respectively. and it uses both the connected information and the 3D point cloud to perform a 3D point labeling in which object information is assigned to each 3D point. Then, it collects 3D points with the same label for each object. Finally, it converts the 3D points collected for each object into 3D mesh. It was confirmed that the system proposed in this paper can generate a 3D mesh with 2D RGB image sequences obtained with a general monocular camera to provide interactive AR services.

Original language	English
Title of host publication	ICTC 2021 - 12th International Conference on ICT Convergence
Subtitle of host publication	Beyond the Pandemic Era with ICT Convergence Innovation
Publisher	IEEE Computer Society
Pages	1558-1563
Number of pages	6
ISBN (Electronic)	9781665423830
DOIs	https://doi.org/10.1109/ICTC52510.2021.9620779
Publication status	Published - 2021
Event	12th International Conference on Information and Communication Technology Convergence, ICTC 2021 - Jeju Island, Korea, Republic of Duration: 2021 Oct 20 → 2021 Oct 22

Publication series

Name	International Conference on ICT Convergence
Volume	2021-October
ISSN (Print)	2162-1233
ISSN (Electronic)	2162-1241

Conference

Conference	12th International Conference on Information and Communication Technology Convergence, ICTC 2021
Country/Territory	Korea, Republic of
City	Jeju Island
Period	21/10/20 → 21/10/22

Bibliographical note

Funding Information:
This work was supported by 2021 Cultural Heritage Smart Preservation & Utilization R&D Program by Cultural Heritage Administration, National Research Institute of Cultural Heritage [2021A02P01-001, A smart H-BIM modeling technology of wooden architecture for the conservation of Historical and Cultural Environment].

Publisher Copyright:
© 2021 IEEE.

All Science Journal Classification (ASJC) codes

Information Systems
Computer Networks and Communications

Access to Document

10.1109/ICTC52510.2021.9620779

Cite this

Yoon, Y. S., Hwang, S., Lee, H., & Lee, S. (2021). 3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services. In ICTC 2021 - 12th International Conference on ICT Convergence: Beyond the Pandemic Era with ICT Convergence Innovation (pp. 1558-1563). (International Conference on ICT Convergence; Vol. 2021-October). IEEE Computer Society. https://doi.org/10.1109/ICTC52510.2021.9620779

@inproceedings{ea2d4d2b206d44fab220d398dfd7db25,

title = "3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services",

abstract = "In this paper, we propose a system that transforms the 3D space of the real world into a 3D mesh to provide augmented reality(AR) services. First, the proposed system acquires a 2D RGB image sequence using a monocular camera in order to model the real world in 3D data. and it creates a 3D point cloud with 3D spatial information of the real world using the obtained RGB image sequence. Also, it segments objects that appear in the RGB image sequence into individual objects. Next, the proposed system tracks the segmented multiple objects and connects identification information of the objects respectively. and it uses both the connected information and the 3D point cloud to perform a 3D point labeling in which object information is assigned to each 3D point. Then, it collects 3D points with the same label for each object. Finally, it converts the 3D points collected for each object into 3D mesh. It was confirmed that the system proposed in this paper can generate a 3D mesh with 2D RGB image sequences obtained with a general monocular camera to provide interactive AR services.",

author = "Yoon, {Young Suk} and Sangwon Hwang and Heansung Lee and Sangyoun Lee",

note = "Funding Information: This work was supported by 2021 Cultural Heritage Smart Preservation & Utilization R&D Program by Cultural Heritage Administration, National Research Institute of Cultural Heritage [2021A02P01-001, A smart H-BIM modeling technology of wooden architecture for the conservation of Historical and Cultural Environment]. Publisher Copyright: {\textcopyright} 2021 IEEE.; 12th International Conference on Information and Communication Technology Convergence, ICTC 2021 ; Conference date: 20-10-2021 Through 22-10-2021",

year = "2021",

doi = "10.1109/ICTC52510.2021.9620779",

language = "English",

series = "International Conference on ICT Convergence",

publisher = "IEEE Computer Society",

pages = "1558--1563",

booktitle = "ICTC 2021 - 12th International Conference on ICT Convergence",

address = "United States",

}

Yoon, YS, Hwang, S, Lee, H & Lee, S 2021, 3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services. in ICTC 2021 - 12th International Conference on ICT Convergence: Beyond the Pandemic Era with ICT Convergence Innovation. International Conference on ICT Convergence, vol. 2021-October, IEEE Computer Society, pp. 1558-1563, 12th International Conference on Information and Communication Technology Convergence, ICTC 2021, Jeju Island, Korea, Republic of, 21/10/20. https://doi.org/10.1109/ICTC52510.2021.9620779

3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services. / Yoon, Young Suk; Hwang, Sangwon; Lee, Heansung et al.

ICTC 2021 - 12th International Conference on ICT Convergence: Beyond the Pandemic Era with ICT Convergence Innovation. IEEE Computer Society, 2021. p. 1558-1563 (International Conference on ICT Convergence; Vol. 2021-October).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - 3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services

AU - Yoon, Young Suk

AU - Hwang, Sangwon

AU - Lee, Heansung

AU - Lee, Sangyoun

N1 - Funding Information: This work was supported by 2021 Cultural Heritage Smart Preservation & Utilization R&D Program by Cultural Heritage Administration, National Research Institute of Cultural Heritage [2021A02P01-001, A smart H-BIM modeling technology of wooden architecture for the conservation of Historical and Cultural Environment]. Publisher Copyright: © 2021 IEEE.

PY - 2021

Y1 - 2021

N2 - In this paper, we propose a system that transforms the 3D space of the real world into a 3D mesh to provide augmented reality(AR) services. First, the proposed system acquires a 2D RGB image sequence using a monocular camera in order to model the real world in 3D data. and it creates a 3D point cloud with 3D spatial information of the real world using the obtained RGB image sequence. Also, it segments objects that appear in the RGB image sequence into individual objects. Next, the proposed system tracks the segmented multiple objects and connects identification information of the objects respectively. and it uses both the connected information and the 3D point cloud to perform a 3D point labeling in which object information is assigned to each 3D point. Then, it collects 3D points with the same label for each object. Finally, it converts the 3D points collected for each object into 3D mesh. It was confirmed that the system proposed in this paper can generate a 3D mesh with 2D RGB image sequences obtained with a general monocular camera to provide interactive AR services.

AB - In this paper, we propose a system that transforms the 3D space of the real world into a 3D mesh to provide augmented reality(AR) services. First, the proposed system acquires a 2D RGB image sequence using a monocular camera in order to model the real world in 3D data. and it creates a 3D point cloud with 3D spatial information of the real world using the obtained RGB image sequence. Also, it segments objects that appear in the RGB image sequence into individual objects. Next, the proposed system tracks the segmented multiple objects and connects identification information of the objects respectively. and it uses both the connected information and the 3D point cloud to perform a 3D point labeling in which object information is assigned to each 3D point. Then, it collects 3D points with the same label for each object. Finally, it converts the 3D points collected for each object into 3D mesh. It was confirmed that the system proposed in this paper can generate a 3D mesh with 2D RGB image sequences obtained with a general monocular camera to provide interactive AR services.

UR - http://www.scopus.com/inward/record.url?scp=85122945345&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85122945345&partnerID=8YFLogxK

U2 - 10.1109/ICTC52510.2021.9620779

DO - 10.1109/ICTC52510.2021.9620779

M3 - Conference contribution

AN - SCOPUS:85122945345

T3 - International Conference on ICT Convergence

SP - 1558

EP - 1563

BT - ICTC 2021 - 12th International Conference on ICT Convergence

PB - IEEE Computer Society

T2 - 12th International Conference on Information and Communication Technology Convergence, ICTC 2021

Y2 - 20 October 2021 through 22 October 2021

ER -

3D Mesh Transformation System using Multi-Object Tracking for Augmented Reality Services

Abstract

Publication series

Conference

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this