ON MODIFICATION OF SYMMETRIC RANK ONE FOR TRAINING NEURAL NETWORK BASED ON GRADIENT VECTOR
Keywords:
SR1 method, Quasi-Newton method, Neural Network, optimization.
Abstract
In this paper, a modification of Symmetric Rank One (SR1) is propounded on the grounds of Modifying gradient-difference vector which meets Quasi condition and positive definite conditions. The new method is compared with the standard test results of the SR1 algorithm. In general, the modified method is more superior and efficient when compared to the standard Quasi-Newton method
Downloads
Download data is not yet available.
References
Yao, Evolving artificial neural networks, Proceedings of the IEEE, 1999, vol. 87, no. 9, pp. 1423-1447,.
J. Dayhoff, An Introduction to Neural Network Architectures, New York: Van Nostrand Reinhold, 1990.
J. Nocedal and S. J. Wright. Numerical Optimization. Springer, New York, 2006, 2nd edition.
J.E. Dennis and J.J. Mor6, Quasi-Newton methods, motivation and theory, SIAM Review 1977, 19, 46-89.
J.E. Dennis and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983, 4.
K. Mehrotra, C. K. Mohan, and S. Ranka, Elements of Artificial Neural Networks, Cambridge, MA: MIT Press, 1997.
L. Bottou, F. Curtis, and J. Nocedal. Optimization methods for large-scale machine learning, SIAM Review, 2018, 60(2):223311.
M. Al-Baali and H. Khalfan, An Overview of Some Practical QuasiNewton Methods for Unconstrained Optimization, SQU Journal For Science, 2007, 12 (2) 199-209.
N. Andrei, Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization, J. Comput. Appl. Math, 2009, 230, no. 2, 570–582.
Ngoc Tam Bui and Hiroshi Hasegawa, Training Artificial Neural Network Using Modification of Differential Evolution Algorithm, International Journal of Machine Learning and Computing, February 2015, Vol. 5, No. 1.
P.E. Gill, W. Murray and M.H. Wright, Practical Optimization Academic Press, New York, 1981, 8.
R. Fletcher, Practical Methods of Optimization: Unconstrained Optimization, Wiley, Chichester, 1980.
Stephen J. Wright and Jorge Nocedal, numerical optimization. Springer new york 2006
J. Dayhoff, An Introduction to Neural Network Architectures, New York: Van Nostrand Reinhold, 1990.
J. Nocedal and S. J. Wright. Numerical Optimization. Springer, New York, 2006, 2nd edition.
J.E. Dennis and J.J. Mor6, Quasi-Newton methods, motivation and theory, SIAM Review 1977, 19, 46-89.
J.E. Dennis and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice-Hall, Englewood Cliffs, NJ, 1983, 4.
K. Mehrotra, C. K. Mohan, and S. Ranka, Elements of Artificial Neural Networks, Cambridge, MA: MIT Press, 1997.
L. Bottou, F. Curtis, and J. Nocedal. Optimization methods for large-scale machine learning, SIAM Review, 2018, 60(2):223311.
M. Al-Baali and H. Khalfan, An Overview of Some Practical QuasiNewton Methods for Unconstrained Optimization, SQU Journal For Science, 2007, 12 (2) 199-209.
N. Andrei, Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization, J. Comput. Appl. Math, 2009, 230, no. 2, 570–582.
Ngoc Tam Bui and Hiroshi Hasegawa, Training Artificial Neural Network Using Modification of Differential Evolution Algorithm, International Journal of Machine Learning and Computing, February 2015, Vol. 5, No. 1.
P.E. Gill, W. Murray and M.H. Wright, Practical Optimization Academic Press, New York, 1981, 8.
R. Fletcher, Practical Methods of Optimization: Unconstrained Optimization, Wiley, Chichester, 1980.
Stephen J. Wright and Jorge Nocedal, numerical optimization. Springer new york 2006
Published
2020-06-02
How to Cite
JAMEEL, H. H., & SHAREEF, S. G. (2020). ON MODIFICATION OF SYMMETRIC RANK ONE FOR TRAINING NEURAL NETWORK BASED ON GRADIENT VECTOR. Journal of Duhok University, 22(2), 109-114. https://doi.org/10.26682/sjuod.2019.22.2.12
Section
Pure and Engineering Sciences
It is the policy of the Journal of Duhok University to own the copyright of the technical contributions. It publishes and facilitates the appropriate re-utilize of the published materials by others. Photocopying is permitted with credit and referring to the source for individuals use.
Copyright © 2017. All Rights Reserved.
