DOI : https://doi.org/10.18517/ijaseit.7.5.2972

An Optimized Back Propagation Learning Algorithm with Adaptive Learning Rate

Nazri Mohd Nawi (1), Faridah Hamzah (2), Norhamreeza Abdul Hamid (3), Muhammad Zubair Rehman (4), Mohammad Aamir (5), Azizul Ramli Azhar (6)
(1) Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400, Johor, Malaysia
(2) Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400, Johor, Malaysia
(3) Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400, Johor, Malaysia
(4) Department of Computer Science and Information Technology, University of Lahore, Islamabad Campus, Pakistan
(5) Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400, Johor, Malaysia
(6) Faculty of Computer Science and Information Technology, Universiti Tun Hussein Onn Malaysia, 86400, Johor, Malaysia
Fulltext View | Download
How to cite (IJASEIT) :
[1]
N. M. Nawi, F. Hamzah, N. A. Hamid, M. Z. Rehman, M. Aamir, and A. R. Azhar, “An Optimized Back Propagation Learning Algorithm with Adaptive Learning Rate”, Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 5, pp. 1693–1700, Oct. 2017.
Citation Format :
Back Propagation (BP) is commonly used algorithm that optimize the performance of network for training multilayer feed-forward artificial neural networks. However, BP is inherently slow in learning and it sometimes gets trapped at local minima. These problems occur mailnly due to a constant and non-optimum learning rate (a fixed step size) in which the fixed value of learning rate is set to an initial starting value before training patterns for an input layer and an output layer. This fixed learning rate often leads the BP network towrds failure during steepest descent. Therefore to overcome the limitations of BP, this paper introduces an improvement to back propagation gradient descent with adapative learning rate (BPGD-AL) by changing the values of learning rate locally during the learning process. The simulation results on selected benchmark datasets show that the adaptive learning rate significantly improves the learning efficiency of the Back Propagation Algorithm
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who publish with this journal agree to the following terms:

    1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
    2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
    3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).