Boundedness and Convergence of Batch Gradient Method for Training Pi-Sigma Neural Network with Inner-Penalty and Momentum
Kh. Sh. Mohamed1, Xiong Yan2, Zhengxue Li3, Z. A. Habtamu4, Abdrhaman. M. Adam5
1Kh. Sh. Mohamed, School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, PR China.
2Xiong Yan, School of Science, Liaoning University of Science & Technology, Anshan 114051, China.
3Zhengxue Li, School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, PR China.
4Z. A. Habtamu, School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, PR China.
5Abdrhaman M. Adam, School of Mathematical Sciences, Dalian University of Technology, Dalian 116024, PR China.
Manuscript received on December 14, 2016 . | Revised Manuscript received on December 28, 2016 . | Manuscript published on January 05, 2016 . | PP: 71-78 | Volume-5 Issue-6, January 2016 . | Retrieval Number: F2786015616 /2016©BEIESP
Open Access | Ethics and Policies | Cite
©The Authors. Published By: Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/
Abstract: In the process industries convergence of a batch gradient method with inner-penalty and adaptive momentum is inspection for training pi-sigma neural networks. The role of the usual penalty is considered, which is a term proportional to the norm of the weights to control the magnitude of the weights and improve the generalization performance of the network. The monotonicity theorem and two convergence theorems of our gradient algorithm with inner-penalty term is guaranteed during the training iteration.
Keywords: Convergence, pi-sigma neural network, batch gradient method, inner-penalty, momentum, boundedness.