Transfer learning in computer vision tasks: Remember where you come from

Xuhong Li, Yves Grandvalet, Franck Davoine, Jingchun Cheng, Yin Cui, Hang Zhang, Serge Belongie, Yi Hsuan Tsai, Ming Hsuan Yang

Research output: Contribution to journalArticlepeer-review

10 Citations (Scopus)

Abstract

Fine-tuning pre-trained deep networks is a practical way of benefiting from the representation learned on a large database while having relatively few examples to train a model. This adjustment is nowadays routinely performed so as to benefit of the latest improvements of convolutional neural networks trained on large databases. Fine-tuning requires some form of regularization, which is typically implemented by weight decay that drives the network parameters towards zero. This choice conflicts with the motivation for fine-tuning, as starting from a pre-trained solution aims at taking advantage of the previously acquired knowledge. Hence, regularizers promoting an explicit inductive bias towards the pre-trained model have been recently proposed. This paper demonstrates the versatility of this type of regularizer across transfer learning scenarios. We replicated experiments on three state-of-the-art approaches in image classification, image segmentation, and video analysis to compare the relative merits of regularizers. These tests show systematic improvements compared to weight decay. Our experimental protocol put forward the versatility of a regularizer that is easy to implement and to operate that we eventually recommend as the new baseline for future approaches to transfer learning relying on fine-tuning.

Original languageEnglish
Article number103853
JournalImage and Vision Computing
Volume93
DOIs
Publication statusPublished - 2020 Jan

Bibliographical note

Funding Information:
This work was carried out with the supports of the China Scholarship Council and of a PEPS grant through the DESSTOPT project jointly managed by the National Institute of Mathematical Sciences and their Interactions (INSMI) and the Institute of Information Science and their Interactions (INS2I) of the CNRS, France. It was carried out in the framework of SIVALab, a joint laboratory between Renault and Heudiasyc (UTC/CNRS). We acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research.

Funding Information:
This work was carried out with the supports of the China Scholarship Council and of a PEPS grant through the DESSTOPT project jointly managed by the National Institute of Mathematical Sciences and their Interactions (INSMI) and the Institute of Information Science and their Interactions (INS2I) of the CNRS, France. It was carried out in the framework of SIVALab, a joint laboratory between Renault and Heudiasyc (UTC/CNRS). We acknowledge the support of NVIDIA Corporation with the donation of GPUs used for this research.

Publisher Copyright:
© 2019 Elsevier B.V.

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Transfer learning in computer vision tasks: Remember where you come from'. Together they form a unique fingerprint.

Cite this