Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks

Hangil Choi, Seungryong Kim, Kihong Park, Kwanghoon Sohn

Research output: Chapter in Book/Report/Conference proceedingConference contribution

46 Citations (Scopus)


This paper presents a method for detecting a pedestrian by leveraging multi-spectral image pairs. Our approach is based on the observation that a multi-spectral image, especially far-infrared (FIR) image, enables us to overcome inherent limitations for pedestrian detection under challenging circumstances, such as even dark environments. For that task, multi-spectral color-FIR image pairs are used in a synergistic manner for pedestrian detection through deep convolutional neural networks (CNNs) learning and support vector regression (SVR). For inferring the confidence of a pedestrian, we first learn CNNs between color images (or FIR images) and bounding box annotations of pedestrians, respectively. Furthermore, for each object proposal, we extract intermediate activation features from network, and learn the probability of pedestrian using SVR. To improve the detection performance, the learned probability of pedestrian for each proposal is accumulated on the image domain. Based on the pedestrian confidence estimated from each network and accumulated pedestrian probabilities, the most probable pedestrian is finally localized among object proposal candidates. Thanks to its high robustness of multi-spectral imaging in dark environments and its high discriminative power of deep CNNs, our framework is shown to surpass state-of-the-art pedestrian detection methods on multi-spectral pedestrian benchmark.

Original languageEnglish
Title of host publication2016 23rd International Conference on Pattern Recognition, ICPR 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781509048472
Publication statusPublished - 2016 Jan 1
Event23rd International Conference on Pattern Recognition, ICPR 2016 - Cancun, Mexico
Duration: 2016 Dec 42016 Dec 8

Publication series

NameProceedings - International Conference on Pattern Recognition
ISSN (Print)1051-4651


Other23rd International Conference on Pattern Recognition, ICPR 2016

Bibliographical note

Funding Information:
This work was supported by Institute for Information and communications Technology Promotion (IITP) grant funded by the Korea government (MSIP) (No. R0115-15-1007, High quality 2d-to-multiview contents generation from large-scale RGB+D database).

Publisher Copyright:
© 2016 IEEE.

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Multi-spectral pedestrian detection based on accumulated object proposal with fully convolutional networks'. Together they form a unique fingerprint.

Cite this