Geometry of decision boundaries of neural networks

Chulhee Lee, Ohjae Kwon, Eunsuk Jung

Research output: Contribution to journalConference article

1 Citation (Scopus)

Abstract

In this paper, we provide a thorough analysis of decision boundaries of neural networks when they are used as a classifier. It has been shown that the classifying mechanism of the neural network can be divided into two parts: dimension expansion by hidden neurons and linear decision boundary formation by output neurons. In this paradigm, the input data is first warped into a higher dimensional space by the hidden neurons and the output neurons draw linear decision boundaries in the expanded space (hidden neuron space). We also note that the decision boundaries in the hidden neuron space are not completely independent. This dependency of decision boundaries is extended to multiclass problems, providing a valuable insight into formation of decision boundaries in the hidden neuron space. This analysis provides a new understanding of how neural networks construct complex decision boundaries and explains how different sets of weights may provide similar results.

Original languageEnglish
Pages (from-to)167-179
Number of pages13
JournalProceedings of SPIE - The International Society for Optical Engineering
Volume4471
DOIs
Publication statusPublished - 2001 Dec 1
EventAlgorithms and Systems for Optical Information Processing - San Diego, CA, United States
Duration: 2001 Jul 312001 Aug 2

Fingerprint

neurons
Neurons
Neuron
Neural Networks
Neural networks
Geometry
geometry
output
Output
Multi-class
classifiers
classifying
High-dimensional
Classifiers
Classifier
Paradigm
expansion

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

@article{a9880051beb14fc78008df0cc54524c7,
title = "Geometry of decision boundaries of neural networks",
abstract = "In this paper, we provide a thorough analysis of decision boundaries of neural networks when they are used as a classifier. It has been shown that the classifying mechanism of the neural network can be divided into two parts: dimension expansion by hidden neurons and linear decision boundary formation by output neurons. In this paradigm, the input data is first warped into a higher dimensional space by the hidden neurons and the output neurons draw linear decision boundaries in the expanded space (hidden neuron space). We also note that the decision boundaries in the hidden neuron space are not completely independent. This dependency of decision boundaries is extended to multiclass problems, providing a valuable insight into formation of decision boundaries in the hidden neuron space. This analysis provides a new understanding of how neural networks construct complex decision boundaries and explains how different sets of weights may provide similar results.",
author = "Chulhee Lee and Ohjae Kwon and Eunsuk Jung",
year = "2001",
month = "12",
day = "1",
doi = "10.1117/12.449334",
language = "English",
volume = "4471",
pages = "167--179",
journal = "Proceedings of SPIE - The International Society for Optical Engineering",
issn = "0277-786X",
publisher = "SPIE",

}

Geometry of decision boundaries of neural networks. / Lee, Chulhee; Kwon, Ohjae; Jung, Eunsuk.

In: Proceedings of SPIE - The International Society for Optical Engineering, Vol. 4471, 01.12.2001, p. 167-179.

Research output: Contribution to journalConference article

TY - JOUR

T1 - Geometry of decision boundaries of neural networks

AU - Lee, Chulhee

AU - Kwon, Ohjae

AU - Jung, Eunsuk

PY - 2001/12/1

Y1 - 2001/12/1

N2 - In this paper, we provide a thorough analysis of decision boundaries of neural networks when they are used as a classifier. It has been shown that the classifying mechanism of the neural network can be divided into two parts: dimension expansion by hidden neurons and linear decision boundary formation by output neurons. In this paradigm, the input data is first warped into a higher dimensional space by the hidden neurons and the output neurons draw linear decision boundaries in the expanded space (hidden neuron space). We also note that the decision boundaries in the hidden neuron space are not completely independent. This dependency of decision boundaries is extended to multiclass problems, providing a valuable insight into formation of decision boundaries in the hidden neuron space. This analysis provides a new understanding of how neural networks construct complex decision boundaries and explains how different sets of weights may provide similar results.

AB - In this paper, we provide a thorough analysis of decision boundaries of neural networks when they are used as a classifier. It has been shown that the classifying mechanism of the neural network can be divided into two parts: dimension expansion by hidden neurons and linear decision boundary formation by output neurons. In this paradigm, the input data is first warped into a higher dimensional space by the hidden neurons and the output neurons draw linear decision boundaries in the expanded space (hidden neuron space). We also note that the decision boundaries in the hidden neuron space are not completely independent. This dependency of decision boundaries is extended to multiclass problems, providing a valuable insight into formation of decision boundaries in the hidden neuron space. This analysis provides a new understanding of how neural networks construct complex decision boundaries and explains how different sets of weights may provide similar results.

UR - http://www.scopus.com/inward/record.url?scp=0035767543&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0035767543&partnerID=8YFLogxK

U2 - 10.1117/12.449334

DO - 10.1117/12.449334

M3 - Conference article

AN - SCOPUS:0035767543

VL - 4471

SP - 167

EP - 179

JO - Proceedings of SPIE - The International Society for Optical Engineering

JF - Proceedings of SPIE - The International Society for Optical Engineering

SN - 0277-786X

ER -