Diverse evolutionary neural networks based on information theory

Kyung Joong Kim; Sung Bae Cho

doi:10.1007/978-3-540-69162-4_105

Diverse evolutionary neural networks based on information theory

Kyung Joong Kim, Sung Bae Cho

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

1 Citation (Scopus)

Abstract

There is no consensus on measuring distances between two different neural network architectures. Two folds of methods are used for that purpose: Structural and behavioral distance measures. In this paper, we focus on the later one that compares differences based on output responses given the same input. Usually neural network output can be interpreted as a probabilistic function given the input signals if it is normalized to 1. Information theoretic distance measures are widely used to measure distances between two probabilistic distributions. In the framework of evolving diverse neural networks, we adopted information-theoretic distance measures to improve its performance. Experimental results on UCI benchmark dataset show the promising possibility of the approach.

Original language	English
Title of host publication	Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers
Pages	1007-1016
Number of pages	10
Edition	PART 2
DOIs	https://doi.org/10.1007/978-3-540-69162-4_105
Publication status	Published - 2008
Event	14th International Conference on Neural Information Processing, ICONIP 2007 - Kitakyushu, Japan Duration: 2007 Nov 13 → 2007 Nov 16

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Number	PART 2
Volume	4985 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	14th International Conference on Neural Information Processing, ICONIP 2007
Country/Territory	Japan
City	Kitakyushu
Period	07/11/13 → 07/11/16

Bibliographical note

Funding Information:
This research was supported by Brain Science and Engineering Research Program sponsored by Korean Ministry of Commerce, Industry and Energy.

All Science Journal Classification (ASJC) codes

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-540-69162-4_105

Cite this

Kim, K. J., & Cho, S. B. (2008). Diverse evolutionary neural networks based on information theory. In Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers (PART 2 ed., pp. 1007-1016). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4985 LNCS, No. PART 2). https://doi.org/10.1007/978-3-540-69162-4_105

Kim, Kyung Joong ; Cho, Sung Bae. / Diverse evolutionary neural networks based on information theory. Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers. PART 2. ed. 2008. pp. 1007-1016 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2).

@inproceedings{561bf4959bfb4734a6b23a741bfe281a,

title = "Diverse evolutionary neural networks based on information theory",

abstract = "There is no consensus on measuring distances between two different neural network architectures. Two folds of methods are used for that purpose: Structural and behavioral distance measures. In this paper, we focus on the later one that compares differences based on output responses given the same input. Usually neural network output can be interpreted as a probabilistic function given the input signals if it is normalized to 1. Information theoretic distance measures are widely used to measure distances between two probabilistic distributions. In the framework of evolving diverse neural networks, we adopted information-theoretic distance measures to improve its performance. Experimental results on UCI benchmark dataset show the promising possibility of the approach.",

author = "Kim, {Kyung Joong} and Cho, {Sung Bae}",

note = "Funding Information: This research was supported by Brain Science and Engineering Research Program sponsored by Korean Ministry of Commerce, Industry and Energy.; 14th International Conference on Neural Information Processing, ICONIP 2007 ; Conference date: 13-11-2007 Through 16-11-2007",

year = "2008",

doi = "10.1007/978-3-540-69162-4_105",

language = "English",

isbn = "3540691596",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

number = "PART 2",

pages = "1007--1016",

booktitle = "Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers",

edition = "PART 2",

}

Kim, KJ & Cho, SB 2008, Diverse evolutionary neural networks based on information theory. in Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers. PART 2 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 2, vol. 4985 LNCS, pp. 1007-1016, 14th International Conference on Neural Information Processing, ICONIP 2007, Kitakyushu, Japan, 07/11/13. https://doi.org/10.1007/978-3-540-69162-4_105

Diverse evolutionary neural networks based on information theory. / Kim, Kyung Joong; Cho, Sung Bae.

Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers. PART 2. ed. 2008. p. 1007-1016 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4985 LNCS, No. PART 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Diverse evolutionary neural networks based on information theory

AU - Kim, Kyung Joong

AU - Cho, Sung Bae

N1 - Funding Information: This research was supported by Brain Science and Engineering Research Program sponsored by Korean Ministry of Commerce, Industry and Energy.

PY - 2008

Y1 - 2008

N2 - There is no consensus on measuring distances between two different neural network architectures. Two folds of methods are used for that purpose: Structural and behavioral distance measures. In this paper, we focus on the later one that compares differences based on output responses given the same input. Usually neural network output can be interpreted as a probabilistic function given the input signals if it is normalized to 1. Information theoretic distance measures are widely used to measure distances between two probabilistic distributions. In the framework of evolving diverse neural networks, we adopted information-theoretic distance measures to improve its performance. Experimental results on UCI benchmark dataset show the promising possibility of the approach.

AB - There is no consensus on measuring distances between two different neural network architectures. Two folds of methods are used for that purpose: Structural and behavioral distance measures. In this paper, we focus on the later one that compares differences based on output responses given the same input. Usually neural network output can be interpreted as a probabilistic function given the input signals if it is normalized to 1. Information theoretic distance measures are widely used to measure distances between two probabilistic distributions. In the framework of evolving diverse neural networks, we adopted information-theoretic distance measures to improve its performance. Experimental results on UCI benchmark dataset show the promising possibility of the approach.

UR - http://www.scopus.com/inward/record.url?scp=54049090462&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=54049090462&partnerID=8YFLogxK

U2 - 10.1007/978-3-540-69162-4_105

DO - 10.1007/978-3-540-69162-4_105

M3 - Conference contribution

AN - SCOPUS:54049090462

SN - 3540691596

SN - 9783540691594

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 1007

EP - 1016

BT - Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers

T2 - 14th International Conference on Neural Information Processing, ICONIP 2007

Y2 - 13 November 2007 through 16 November 2007

ER -

Kim KJ, Cho SB. Diverse evolutionary neural networks based on information theory. In Neural Information Processing - 14th International Conference, ICONIP 2007, Revised Selected Papers. PART 2 ed. 2008. p. 1007-1016. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 2). https://doi.org/10.1007/978-3-540-69162-4_105

Diverse evolutionary neural networks based on information theory

Abstract

Publication series

Other

Bibliographical note

All Science Journal Classification (ASJC) codes

Access to Document

Other files and links

Fingerprint

Cite this