Training confidence-calibrated classifiers for detecting out-of-distribution samples

Kimin Lee, Honglak Lee, Kibok Lee, Jinwoo Shin

Research output: Contribution to conferencePaperpeer-review

163 Citations (Scopus)

Abstract

The problem of detecting whether a test sample is from in-distribution (i.e., training distribution by a classifier) or out-of-distribution sufficiently different from it arises in many real-world machine learning applications. However, the state-of-art deep neural networks are known to be highly overconfident in their predictions, i.e., do not distinguish in- and out-of-distributions. Recently, to handle this issue, several threshold-based detectors have been proposed given pre-trained neural classifiers. However, the performance of prior works highly depends on how to train the classifiers since they only focus on improving inference procedures. In this paper, we develop a novel training method for classifiers so that such inference algorithms can work better. In particular, we suggest two additional terms added to the original loss (e.g., cross entropy). The first one forces samples from out-of-distribution less confident by the classifier and the second one is for (implicitly) generating most effective training samples for the first one. In essence, our method jointly trains both classification and generative neural networks for out-of-distribution. We demonstrate its effectiveness using deep convolutional neural networks on various popular image datasets.

Original languageEnglish
Publication statusPublished - 2018
Event6th International Conference on Learning Representations, ICLR 2018 - Vancouver, Canada
Duration: 2018 Apr 302018 May 3

Conference

Conference6th International Conference on Learning Representations, ICLR 2018
Country/TerritoryCanada
CityVancouver
Period18/4/3018/5/3

Bibliographical note

Funding Information:
This work was supported in part by the Institute for Information & communications Technology Promotion(IITP) grant funded by the Korea government(MSIT) (No.2017-0-01778, Development of Explainable Human-level Deep Machine Learning Inference Framework), the ICT R&D program of MSIP/IITP [R-20161130-004520, Research on Adaptive Machine Learning Technology Development for Intelligent Autonomous Digital Companion], DARPA Explainable AI (XAI) program #313498 and Sloan Research Fellowship.

Publisher Copyright:
© Learning Representations, ICLR 2018 - Conference Track Proceedings.All right reserved.

All Science Journal Classification (ASJC) codes

  • Language and Linguistics
  • Education
  • Computer Science Applications
  • Linguistics and Language

Fingerprint

Dive into the research topics of 'Training confidence-calibrated classifiers for detecting out-of-distribution samples'. Together they form a unique fingerprint.

Cite this