Progressive Representation Adaptation for Weakly Supervised Object Localization

Dong Li, Jia Bin Huang, Yali Li, Shengjin Wang, Ming Hsuan Yang

Research output: Contribution to journalArticlepeer-review

9 Citations (Scopus)


We address the problem of weakly supervised object localization where only image-level annotations are available for training object detectors. Numerous methods have been proposed to tackle this problem through mining object proposals. However, a substantial amount of noise in object proposals causes ambiguities for learning discriminative object models. Such approaches are sensitive to model initialization and often converge to undesirable local minimum solutions. In this paper, we propose to overcome these drawbacks by progressive representation adaptation with two main steps: 1) classification adaptation and 2) detection adaptation. In classification adaptation, we transfer a pre-trained network to a multi-label classification task for recognizing the presence of a certain object in an image. Through the classification adaptation step, the network learns discriminative representations that are specific to object categories of interest. In detection adaptation, we mine class-specific object proposals by exploiting two scoring strategies based on the adapted classification network. Class-specific proposal mining helps remove substantial noise from the background clutter and potential confusion from similar objects. We further refine these proposals using multiple instance learning and segmentation cues. Using these refined object bounding boxes, we fine-tune all the layer of the classification network and obtain a fully adapted detection network. We present detailed experimental validation on the PASCAL VOC and ILSVRC datasets. Experimental results demonstrate that our progressive representation adaptation algorithm performs favorably against the state-of-the-art methods.

Original languageEnglish
Article number8642851
Pages (from-to)1424-1438
Number of pages15
JournalIEEE transactions on pattern analysis and machine intelligence
Issue number6
Publication statusPublished - 2020 Jun 1

Bibliographical note

Funding Information:
This work is supported by the National Natural Science Foundation of China under Grant Nos.61701277, 61771288 and the state key development program in 13th Five-Year under Grant No. 2016YFB0801301. This work is also supported in part to Dr. Ming-Hsuan Yang by NSF CAREER Grant (1149783) and to Dr. Jia-Bin Huang by NSF under Grant No. (1755785).

Publisher Copyright:
© 1979-2012 IEEE.

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics


Dive into the research topics of 'Progressive Representation Adaptation for Weakly Supervised Object Localization'. Together they form a unique fingerprint.

Cite this