Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks

Hangil Choi, Seungryong Kim, Kihong Park, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

16 Citations (Scopus)

Abstract

This paper presents a method for detecting a pedestrian by leveraging multi-spectral image pairs. Our approach is based on the observation that a multi-spectral image, especially far-infrared (FIR) image, enables us to overcome inherent limitations for pedestrian detection under challenging circumstances, such as even dark environments. For that task, multi-spectral color-FIR image pairs are used in a synergistic manner for pedestrian detection through deep convolutional neural networks (CNNs) learning and support vector regression (SVR). For inferring the confidence of a pedestrian, we first learn CNNs between color images (or FIR images) and bounding box annotations of pedestrians, respectively. Furthermore, for each object proposal, we extract intermediate activation features from network, and learn the probability of pedestrian using SVR. To improve the detection performance, the learned probability of pedestrian for each proposal is accumulated on the image domain. Based on the pedestrian confidence estimated from each network and accumulated pedestrian probabilities, the most probable pedestrian is finally localized among object proposal candidates. Thanks to its high robustness of multi-spectral imaging in dark environments and its high discriminative power of deep CNNs, our framework is shown to surpass state-of-the-art pedestrian detection methods on multi-spectral pedestrian benchmark.

Original languageEnglish
Title of host publication2016 23rd International Conference on Pattern Recognition, ICPR 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages621-626
Number of pages6
ISBN (Electronic)9781509048472
DOIs
Publication statusPublished - 2016 Jan 1
Event23rd International Conference on Pattern Recognition, ICPR 2016 - Cancun, Mexico
Duration: 2016 Dec 42016 Dec 8

Publication series

NameProceedings - International Conference on Pattern Recognition
Volume0
ISSN (Print)1051-4651

Other

Other23rd International Conference on Pattern Recognition, ICPR 2016
CountryMexico
CityCancun
Period16/12/416/12/8

Fingerprint

Infrared radiation
Neural networks
Color
Chemical activation
Imaging techniques

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition

Cite this

Choi, H., Kim, S., Park, K., & Sohn, K. (2016). Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks. In 2016 23rd International Conference on Pattern Recognition, ICPR 2016 (pp. 621-626). [7899703] (Proceedings - International Conference on Pattern Recognition; Vol. 0). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICPR.2016.7899703
Choi, Hangil ; Kim, Seungryong ; Park, Kihong ; Sohn, Kwanghoon. / Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks. 2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 621-626 (Proceedings - International Conference on Pattern Recognition).
@inproceedings{d2a68f707ae646909a0df15944498dda,
title = "Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks",
abstract = "This paper presents a method for detecting a pedestrian by leveraging multi-spectral image pairs. Our approach is based on the observation that a multi-spectral image, especially far-infrared (FIR) image, enables us to overcome inherent limitations for pedestrian detection under challenging circumstances, such as even dark environments. For that task, multi-spectral color-FIR image pairs are used in a synergistic manner for pedestrian detection through deep convolutional neural networks (CNNs) learning and support vector regression (SVR). For inferring the confidence of a pedestrian, we first learn CNNs between color images (or FIR images) and bounding box annotations of pedestrians, respectively. Furthermore, for each object proposal, we extract intermediate activation features from network, and learn the probability of pedestrian using SVR. To improve the detection performance, the learned probability of pedestrian for each proposal is accumulated on the image domain. Based on the pedestrian confidence estimated from each network and accumulated pedestrian probabilities, the most probable pedestrian is finally localized among object proposal candidates. Thanks to its high robustness of multi-spectral imaging in dark environments and its high discriminative power of deep CNNs, our framework is shown to surpass state-of-the-art pedestrian detection methods on multi-spectral pedestrian benchmark.",
author = "Hangil Choi and Seungryong Kim and Kihong Park and Kwanghoon Sohn",
year = "2016",
month = "1",
day = "1",
doi = "10.1109/ICPR.2016.7899703",
language = "English",
series = "Proceedings - International Conference on Pattern Recognition",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "621--626",
booktitle = "2016 23rd International Conference on Pattern Recognition, ICPR 2016",
address = "United States",

}

Choi, H, Kim, S, Park, K & Sohn, K 2016, Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks. in 2016 23rd International Conference on Pattern Recognition, ICPR 2016., 7899703, Proceedings - International Conference on Pattern Recognition, vol. 0, Institute of Electrical and Electronics Engineers Inc., pp. 621-626, 23rd International Conference on Pattern Recognition, ICPR 2016, Cancun, Mexico, 16/12/4. https://doi.org/10.1109/ICPR.2016.7899703

Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks. / Choi, Hangil; Kim, Seungryong; Park, Kihong; Sohn, Kwanghoon.

2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 621-626 7899703 (Proceedings - International Conference on Pattern Recognition; Vol. 0).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks

AU - Choi, Hangil

AU - Kim, Seungryong

AU - Park, Kihong

AU - Sohn, Kwanghoon

PY - 2016/1/1

Y1 - 2016/1/1

N2 - This paper presents a method for detecting a pedestrian by leveraging multi-spectral image pairs. Our approach is based on the observation that a multi-spectral image, especially far-infrared (FIR) image, enables us to overcome inherent limitations for pedestrian detection under challenging circumstances, such as even dark environments. For that task, multi-spectral color-FIR image pairs are used in a synergistic manner for pedestrian detection through deep convolutional neural networks (CNNs) learning and support vector regression (SVR). For inferring the confidence of a pedestrian, we first learn CNNs between color images (or FIR images) and bounding box annotations of pedestrians, respectively. Furthermore, for each object proposal, we extract intermediate activation features from network, and learn the probability of pedestrian using SVR. To improve the detection performance, the learned probability of pedestrian for each proposal is accumulated on the image domain. Based on the pedestrian confidence estimated from each network and accumulated pedestrian probabilities, the most probable pedestrian is finally localized among object proposal candidates. Thanks to its high robustness of multi-spectral imaging in dark environments and its high discriminative power of deep CNNs, our framework is shown to surpass state-of-the-art pedestrian detection methods on multi-spectral pedestrian benchmark.

AB - This paper presents a method for detecting a pedestrian by leveraging multi-spectral image pairs. Our approach is based on the observation that a multi-spectral image, especially far-infrared (FIR) image, enables us to overcome inherent limitations for pedestrian detection under challenging circumstances, such as even dark environments. For that task, multi-spectral color-FIR image pairs are used in a synergistic manner for pedestrian detection through deep convolutional neural networks (CNNs) learning and support vector regression (SVR). For inferring the confidence of a pedestrian, we first learn CNNs between color images (or FIR images) and bounding box annotations of pedestrians, respectively. Furthermore, for each object proposal, we extract intermediate activation features from network, and learn the probability of pedestrian using SVR. To improve the detection performance, the learned probability of pedestrian for each proposal is accumulated on the image domain. Based on the pedestrian confidence estimated from each network and accumulated pedestrian probabilities, the most probable pedestrian is finally localized among object proposal candidates. Thanks to its high robustness of multi-spectral imaging in dark environments and its high discriminative power of deep CNNs, our framework is shown to surpass state-of-the-art pedestrian detection methods on multi-spectral pedestrian benchmark.

UR - http://www.scopus.com/inward/record.url?scp=85019076223&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85019076223&partnerID=8YFLogxK

U2 - 10.1109/ICPR.2016.7899703

DO - 10.1109/ICPR.2016.7899703

M3 - Conference contribution

T3 - Proceedings - International Conference on Pattern Recognition

SP - 621

EP - 626

BT - 2016 23rd International Conference on Pattern Recognition, ICPR 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Choi H, Kim S, Park K, Sohn K. Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks. In 2016 23rd International Conference on Pattern Recognition, ICPR 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 621-626. 7899703. (Proceedings - International Conference on Pattern Recognition). https://doi.org/10.1109/ICPR.2016.7899703