TITLE:
Mutual Information-Based Modified Randomized Weights Neural Networks
AUTHORS:
Jian Tang, Zhiwei Wu, Meiying Jia, Zhuo Liu
KEYWORDS:
Randomized Weights Neural Networks, Mutual Information, Feature Selection
JOURNAL NAME:
Journal of Computer and Communications,
Vol.3 No.11,
November
19,
2015
ABSTRACT:
Randomized weights neural
networks have fast learning speed and good generalization performance with one
single hidden layer structure. Input weighs of the hidden layer are produced randomly.
By employing certain activation function, outputs of the hidden layer are
calculated with some randomization. Output weights are computed using pseudo inverse.
Mutual information can be used to measure mutual dependence of two variables quantitatively
based on the probability theory. In this paper, these hidden layer’s outputs
that relate to prediction variable closely are selected with the simple mutual
information based feature selection method. These hidden nodes with high mutual
information values are maintained as a new hidden layer. Thus, the size of the
hidden layer is reduced. The new hidden layer’s output weights are learned with
the pseudo inverse method. The proposed method is compared with the original
randomized algorithms using concrete compressive strength benchmark dataset.
