Abstract
We present a novel human body co-segmentation method for unregistered multispectral, color and thermal, images by leveraging CNNs. The main challenges for that tasks are no-alignment between color and thermal images and an absent of ground truth human segmentation labels. To solve these limitations, our key-insight is to formulate the segmentation network for each modality that solve two sub-tasks, correspondence and classification, in a joint and iterative manner. We formulate the learning framework between multispectral images in a way that training labels for one modality are used to learn the network for the other modality. We estimate dense correspondences between multispectral image pairs using intermediate convolutional activations of CNNs and perform human segmentation for each modality through the conditional random fields (CRF) optimization using unary and pairwise fusion. These two steps are formulated as an iterative framework, enables the network to converge on an optimal solution. Experimental results show that our proposed method outperforms conventional state-of-the-art methods on the VAP benchmark consisting of unregistered multispectral color and thermal images.
| Original language | English |
|---|---|
| Title of host publication | 2017 IEEE International Conference on Image Processing, ICIP 2017 - Proceedings |
| Publisher | IEEE Computer Society |
| Pages | 3115-3119 |
| Number of pages | 5 |
| ISBN (Electronic) | 9781509021758 |
| DOIs | |
| Publication status | Published - 2018 Feb 20 |
| Event | 24th IEEE International Conference on Image Processing, ICIP 2017 - Beijing, China Duration: 2017 Sep 17 → 2017 Sep 20 |
Publication series
| Name | Proceedings - International Conference on Image Processing, ICIP |
|---|---|
| Volume | 2017-September |
| ISSN (Print) | 1522-4880 |
Other
| Other | 24th IEEE International Conference on Image Processing, ICIP 2017 |
|---|---|
| Country | China |
| City | Beijing |
| Period | 17/9/17 → 17/9/20 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Software
- Computer Vision and Pattern Recognition
- Signal Processing
Cite this
}
Multispectral human co-segmentation via joint convolutional neural networks. / Choi, Sungil; Kim, Seungryong; Park, Kihong; Sohn, Kwanghoon.
2017 IEEE International Conference on Image Processing, ICIP 2017 - Proceedings. IEEE Computer Society, 2018. p. 3115-3119 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2017-September).Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
TY - GEN
T1 - Multispectral human co-segmentation via joint convolutional neural networks
AU - Choi, Sungil
AU - Kim, Seungryong
AU - Park, Kihong
AU - Sohn, Kwanghoon
PY - 2018/2/20
Y1 - 2018/2/20
N2 - We present a novel human body co-segmentation method for unregistered multispectral, color and thermal, images by leveraging CNNs. The main challenges for that tasks are no-alignment between color and thermal images and an absent of ground truth human segmentation labels. To solve these limitations, our key-insight is to formulate the segmentation network for each modality that solve two sub-tasks, correspondence and classification, in a joint and iterative manner. We formulate the learning framework between multispectral images in a way that training labels for one modality are used to learn the network for the other modality. We estimate dense correspondences between multispectral image pairs using intermediate convolutional activations of CNNs and perform human segmentation for each modality through the conditional random fields (CRF) optimization using unary and pairwise fusion. These two steps are formulated as an iterative framework, enables the network to converge on an optimal solution. Experimental results show that our proposed method outperforms conventional state-of-the-art methods on the VAP benchmark consisting of unregistered multispectral color and thermal images.
AB - We present a novel human body co-segmentation method for unregistered multispectral, color and thermal, images by leveraging CNNs. The main challenges for that tasks are no-alignment between color and thermal images and an absent of ground truth human segmentation labels. To solve these limitations, our key-insight is to formulate the segmentation network for each modality that solve two sub-tasks, correspondence and classification, in a joint and iterative manner. We formulate the learning framework between multispectral images in a way that training labels for one modality are used to learn the network for the other modality. We estimate dense correspondences between multispectral image pairs using intermediate convolutional activations of CNNs and perform human segmentation for each modality through the conditional random fields (CRF) optimization using unary and pairwise fusion. These two steps are formulated as an iterative framework, enables the network to converge on an optimal solution. Experimental results show that our proposed method outperforms conventional state-of-the-art methods on the VAP benchmark consisting of unregistered multispectral color and thermal images.
UR - http://www.scopus.com/inward/record.url?scp=85045326472&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85045326472&partnerID=8YFLogxK
U2 - 10.1109/ICIP.2017.8296856
DO - 10.1109/ICIP.2017.8296856
M3 - Conference contribution
T3 - Proceedings - International Conference on Image Processing, ICIP
SP - 3115
EP - 3119
BT - 2017 IEEE International Conference on Image Processing, ICIP 2017 - Proceedings
PB - IEEE Computer Society
ER -