Abstract
In this paper, we investigate and analyze the weight distribution of feedforward two-layer neural networks in order to understand and improve the time-consuming training process of neural networks. Generally, it takes a long time to train neural networks. However, when a new problem is presented, neural networks have to be trained again without any benefit from previous training. In order to address this problem, we view training process as finding a solution weight point in a weight space and analyze the distribution of solution weight points in the weight space. Then, we propose a weight initialization method that uses the information on the distribution of the solution weight points. Experimental results show that the proposed weight initialization method provides a better performance than the conventional method that uses a random generator in terms of convergence speed.
| Original language | English |
|---|---|
| Pages (from-to) | 840-849 |
| Number of pages | 10 |
| Journal | Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) |
| Volume | 3138 |
| Publication status | Published - 2004 Dec 1 |
Fingerprint
All Science Journal Classification (ASJC) codes
- Theoretical Computer Science
- Computer Science(all)
Cite this
}
Analyzing weight distribution of feedforward neural networks and efficient weight initialization. / Go, Jimvook; Baek, Byungjoon; Lee, Chulhee.
In: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 3138, 01.12.2004, p. 840-849.Research output: Contribution to journal › Article
TY - JOUR
T1 - Analyzing weight distribution of feedforward neural networks and efficient weight initialization
AU - Go, Jimvook
AU - Baek, Byungjoon
AU - Lee, Chulhee
PY - 2004/12/1
Y1 - 2004/12/1
N2 - In this paper, we investigate and analyze the weight distribution of feedforward two-layer neural networks in order to understand and improve the time-consuming training process of neural networks. Generally, it takes a long time to train neural networks. However, when a new problem is presented, neural networks have to be trained again without any benefit from previous training. In order to address this problem, we view training process as finding a solution weight point in a weight space and analyze the distribution of solution weight points in the weight space. Then, we propose a weight initialization method that uses the information on the distribution of the solution weight points. Experimental results show that the proposed weight initialization method provides a better performance than the conventional method that uses a random generator in terms of convergence speed.
AB - In this paper, we investigate and analyze the weight distribution of feedforward two-layer neural networks in order to understand and improve the time-consuming training process of neural networks. Generally, it takes a long time to train neural networks. However, when a new problem is presented, neural networks have to be trained again without any benefit from previous training. In order to address this problem, we view training process as finding a solution weight point in a weight space and analyze the distribution of solution weight points in the weight space. Then, we propose a weight initialization method that uses the information on the distribution of the solution weight points. Experimental results show that the proposed weight initialization method provides a better performance than the conventional method that uses a random generator in terms of convergence speed.
UR - http://www.scopus.com/inward/record.url?scp=35048841654&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=35048841654&partnerID=8YFLogxK
M3 - Article
AN - SCOPUS:35048841654
VL - 3138
SP - 840
EP - 849
JO - Lecture Notes in Computer Science
JF - Lecture Notes in Computer Science
SN - 0302-9743
ER -