Simulation of Objective Function for Training of New Hidden Units in constructive Neural Networks

Abstract

The present research article represent the mathematical analysis of objective function for training new hidden units in constructive algorithms for multi-layer feed-forward networks. Neural research, now days, is highly attractive wing under research community which may lead the development of some hidden prospects by using mathematical modeling which involve the design of neurons network. The network size is highly important for neural network. Small network as well as large network size cannot be learned very well, but there is an optimum size for which the neural network can be involved for good results. Constructive algorithms started with a small network size and then grow additional hidden units until a satisfactory solution is found. A network, having n-1 hidden units, is directly connected to the output unit, which is modeled as $f_{n-1}=\sum\limits_{j=1}^{n-1}\beta_{j}g_{j}$ and $e_{n-1}=f-f_{n-1}$ is the residual error function for current network with $n-1$ hidden units. A new hidden unit is added under a process in input as a linear combination of $g_n$ with the current network $f_{n-1}+\beta_n g_n$ which governed the minimum residual error $\|e_n\|$ in the output process by keeping $g_n$ fixed and adjusted value of $\beta_n$ so as to minimize residual error. The function to be optimized during input training is $\frac{\langle e_{n-1},g_n\rangle^2}{\|g_n\|^2}$ and corresponding objective function is $s_1=\frac{(\Sigma_pE_p H_p)^2}{\|g_n\|^2}$, where $H_p$ is the activation function of the new hidden unit and $E_p$ is the corresponding residual error before this new hidden unit is added.