Propagating LSTM: 3D pose estimation based on joint interdependency

Kyoungoh Lee; Inwoong Lee; Sanghoon Lee

doi:10.1007/978-3-030-01234-2_8

Propagating LSTM: 3D pose estimation based on joint interdependency

Kyoungoh Lee, Inwoong Lee, Sanghoon Lee

Department of Electrical and Electronic Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

22 Citations (Scopus)

Abstract

We present a novel 3D pose estimation method based on joint interdependency (JI) for acquiring 3D joints from the human pose of an RGB image. The JI incorporates the body part based structural connectivity of joints to learn the high spatial correlation of human posture on our method. Towards this goal, we propose a new long short-term memory (LSTM)-based deep learning architecture named propagating LSTM networks (p-LSTMs), where each LSTM is connected sequentially to reconstruct 3D depth from the centroid to edge joints through learning the intrinsic JI. In the first LSTM, the seed joints of 3D pose are created and reconstructed into the whole-body joints through the connected LSTMs. Utilizing the p-LSTMs, we achieve the higher accuracy of about 11.2% than state-of-the-art methods on the largest publicly available database. Importantly, we demonstrate that the JI drastically reduces the structural errors at body edges, thereby leads to a significant improvement.

Original language	English
Title of host publication	Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings
Editors	Vittorio Ferrari, Cristian Sminchisescu, Martial Hebert, Yair Weiss
Publisher	Springer Verlag
Pages	123-141
Number of pages	19
ISBN (Print)	9783030012335
DOIs	https://doi.org/10.1007/978-3-030-01234-2_8
Publication status	Published - 2018
Event	15th European Conference on Computer Vision, ECCV 2018 - Munich, Germany Duration: 2018 Sep 8 → 2018 Sep 14

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11211 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	15th European Conference on Computer Vision, ECCV 2018
Country/Territory	Germany
City	Munich
Period	18/9/8 → 18/9/14

Bibliographical note

Funding Information:
Acknowledgement. This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government (MSIT) (No. 2016-0-00406, SIAT CCTV Cloud Platform).

Publisher Copyright:
© Springer Nature Switzerland AG 2018.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-030-01234-2_8

Cite this

Lee, K., Lee, I., & Lee, S. (2018). Propagating LSTM: 3D pose estimation based on joint interdependency. In V. Ferrari, C. Sminchisescu, M. Hebert, & Y. Weiss (Eds.), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings (pp. 123-141). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11211 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-030-01234-2_8

Lee, Kyoungoh ; Lee, Inwoong ; Lee, Sanghoon. / Propagating LSTM : 3D pose estimation based on joint interdependency. Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. editor / Vittorio Ferrari ; Cristian Sminchisescu ; Martial Hebert ; Yair Weiss. Springer Verlag, 2018. pp. 123-141 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{8b0d0e9f4501455ea568801f7c349097,

title = "Propagating LSTM: 3D pose estimation based on joint interdependency",

abstract = "We present a novel 3D pose estimation method based on joint interdependency (JI) for acquiring 3D joints from the human pose of an RGB image. The JI incorporates the body part based structural connectivity of joints to learn the high spatial correlation of human posture on our method. Towards this goal, we propose a new long short-term memory (LSTM)-based deep learning architecture named propagating LSTM networks (p-LSTMs), where each LSTM is connected sequentially to reconstruct 3D depth from the centroid to edge joints through learning the intrinsic JI. In the first LSTM, the seed joints of 3D pose are created and reconstructed into the whole-body joints through the connected LSTMs. Utilizing the p-LSTMs, we achieve the higher accuracy of about 11.2% than state-of-the-art methods on the largest publicly available database. Importantly, we demonstrate that the JI drastically reduces the structural errors at body edges, thereby leads to a significant improvement.",

author = "Kyoungoh Lee and Inwoong Lee and Sanghoon Lee",

note = "Funding Information: Acknowledgement. This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government (MSIT) (No. 2016-0-00406, SIAT CCTV Cloud Platform). Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2018.; 15th European Conference on Computer Vision, ECCV 2018 ; Conference date: 08-09-2018 Through 14-09-2018",

year = "2018",

doi = "10.1007/978-3-030-01234-2_8",

language = "English",

isbn = "9783030012335",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "123--141",

editor = "Vittorio Ferrari and Cristian Sminchisescu and Martial Hebert and Yair Weiss",

booktitle = "Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings",

address = "Germany",

}

Lee, K, Lee, I & Lee, S 2018, Propagating LSTM: 3D pose estimation based on joint interdependency. in V Ferrari, C Sminchisescu, M Hebert & Y Weiss (eds), Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11211 LNCS, Springer Verlag, pp. 123-141, 15th European Conference on Computer Vision, ECCV 2018, Munich, Germany, 18/9/8. https://doi.org/10.1007/978-3-030-01234-2_8

Propagating LSTM : 3D pose estimation based on joint interdependency. / Lee, Kyoungoh; Lee, Inwoong; Lee, Sanghoon.

Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. ed. / Vittorio Ferrari; Cristian Sminchisescu; Martial Hebert; Yair Weiss. Springer Verlag, 2018. p. 123-141 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11211 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Propagating LSTM

T2 - 15th European Conference on Computer Vision, ECCV 2018

AU - Lee, Kyoungoh

AU - Lee, Inwoong

AU - Lee, Sanghoon

N1 - Funding Information: Acknowledgement. This work was supported by Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government (MSIT) (No. 2016-0-00406, SIAT CCTV Cloud Platform). Publisher Copyright: © Springer Nature Switzerland AG 2018.

PY - 2018

Y1 - 2018

N2 - We present a novel 3D pose estimation method based on joint interdependency (JI) for acquiring 3D joints from the human pose of an RGB image. The JI incorporates the body part based structural connectivity of joints to learn the high spatial correlation of human posture on our method. Towards this goal, we propose a new long short-term memory (LSTM)-based deep learning architecture named propagating LSTM networks (p-LSTMs), where each LSTM is connected sequentially to reconstruct 3D depth from the centroid to edge joints through learning the intrinsic JI. In the first LSTM, the seed joints of 3D pose are created and reconstructed into the whole-body joints through the connected LSTMs. Utilizing the p-LSTMs, we achieve the higher accuracy of about 11.2% than state-of-the-art methods on the largest publicly available database. Importantly, we demonstrate that the JI drastically reduces the structural errors at body edges, thereby leads to a significant improvement.

AB - We present a novel 3D pose estimation method based on joint interdependency (JI) for acquiring 3D joints from the human pose of an RGB image. The JI incorporates the body part based structural connectivity of joints to learn the high spatial correlation of human posture on our method. Towards this goal, we propose a new long short-term memory (LSTM)-based deep learning architecture named propagating LSTM networks (p-LSTMs), where each LSTM is connected sequentially to reconstruct 3D depth from the centroid to edge joints through learning the intrinsic JI. In the first LSTM, the seed joints of 3D pose are created and reconstructed into the whole-body joints through the connected LSTMs. Utilizing the p-LSTMs, we achieve the higher accuracy of about 11.2% than state-of-the-art methods on the largest publicly available database. Importantly, we demonstrate that the JI drastically reduces the structural errors at body edges, thereby leads to a significant improvement.

UR - http://www.scopus.com/inward/record.url?scp=85055101177&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055101177&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-01234-2_8

DO - 10.1007/978-3-030-01234-2_8

M3 - Conference contribution

AN - SCOPUS:85055101177

SN - 9783030012335

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 123

EP - 141

BT - Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings

A2 - Ferrari, Vittorio

A2 - Sminchisescu, Cristian

A2 - Hebert, Martial

A2 - Weiss, Yair

PB - Springer Verlag

Y2 - 8 September 2018 through 14 September 2018

ER -

Lee K, Lee I, Lee S. Propagating LSTM: 3D pose estimation based on joint interdependency. In Ferrari V, Sminchisescu C, Hebert M, Weiss Y, editors, Computer Vision – ECCV 2018 - 15th European Conference, 2018, Proceedings. Springer Verlag. 2018. p. 123-141. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-01234-2_8

Propagating LSTM: 3D pose estimation based on joint interdependency

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this