Unsupervised visual representation learning by graph-based consistent constraints

Dong Li, Wei Chih Hung, Jia Bin Huang, Shengjin Wang, Narendra Ahuja, Ming Hsuan Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

32 Citations (Scopus)


Learning rich visual representations often require training on datasets of millions of manually annotated examples. This substantially limits the scalability of learning effective representations as labeled data is expensive or scarce. In this paper, we address the problem of unsupervised visual representation learning from a large, unlabeled collection of images. By representing each image as a node and each nearest-neighbor matching pair as an edge, our key idea is to leverage graph-based analysis to discover positive and negative image pairs (i.e., pairs belonging to the same and different visual categories). Specifically, we propose to use a cycle consistency criterion for mining positive pairs and geodesic distance in the graph for hard negative mining. We show that the mined positive and negative image pairs can provide accurate supervisory signals for learning effective representations using Convolutional Neural Networks (CNNs). We demonstrate the effectiveness of the proposed unsupervised constraint mining method in two settings: (1) unsupervised feature learning and (2) semi-supervised learning. For unsupervised feature learning, we obtain competitive performance with several state-of-the-art approaches on the PASCAL VOC 2007 dataset. For semisupervised learning, we show boosted performance by incorporating the mined constraints on three image classification datasets.

Original languageEnglish
Title of host publicationComputer Vision - 14th European Conference, ECCV 2016, Proceedings
EditorsBastian Leibe, Jiri Matas, Nicu Sebe, Max Welling
PublisherSpringer Verlag
Number of pages17
ISBN (Print)9783319464923
Publication statusPublished - 2016
Event14th European Conference on Computer Vision, ECCV 2016 - Amsterdam, Netherlands
Duration: 2016 Oct 82016 Oct 16

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume9908 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference14th European Conference on Computer Vision, ECCV 2016

Bibliographical note

Funding Information:
This work is supported in part by the Initiative Scientific Research Program of Ministry of Education under Grant #20141081253. J.-B. Huang and N. Ahuja are supported in part by Office of Naval Research under Grant N00014-16-1-2314. W.-C. Hung and M.-H. Yang are supported in part by the NSF CAREER Grant #1149783 and gifts from Adobe and Nvidia.

Publisher Copyright:
© Springer International Publishing AG 2016.

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Unsupervised visual representation learning by graph-based consistent constraints'. Together they form a unique fingerprint.

Cite this