A NEW IMAGE CLASSIFICATION SYSTEM USING DEEP CONVOLUTION NEURAL NETWORK AND MODIFIED AMSGRAD OPTIMIZER

  • ARMAN I. MOHAMMED • Dept.Information Technology, Presidency, Duhok Polytechnic University, Kurdistan Region, Iraq
  • AHMED AK. TAHIR Dept. Of Computer Science, College of Science, University of Duhok, Kurdistan Region, Iraq
Keywords: Adam, AMSgrad, CNN, Deep neural networks, Image classification, Optimization algorithms.

Abstract

A new deep Convolutional Neural Network (CNN) with six convolutional layers and one fully-connected layer is developed and trained by backpropagation using a new optimization algorithm called Fast-AMSgrad which is modified from AMSgrad. The aims are to speed up the training process while achieving acceptable accuracy. The application of the network using both, the Fast-AMSgrad and the AMSgrad algorithms to CIFAR-10 dataset for image classification reveals that the developed CNN performs better when trained with Fast-AMSgrad for both cases, with and without Batch Normalization (BN) layers. The training time is reduced by 50% when Fast-AMSgrad algorithm is used. Also the accuracy and loss values of the training and validation are improved when Fast-AMSgrad is used.  The training and validation accuracies provided by Fast-AMSgrad with BN are (91.18% and 86.92%) at epoch number (50) and (94.13% and 86.758%) at epoch number (100), while the corresponding accuracies that are provided by AMSgrad with BN are (82.65% and 81.4%) at epoch (50) and (88.82% and 85.85%) at epoch (100). The overall test accuracy and classification metric measures indicate that the given architecture of CNN and optimization algorithm perform reasonably well

Downloads

Download data is not yet available.

References

Dalal N. and Triggs B., 2005, “Histograms of oriented gradients for human detection”, in international Conference on computer vision & Pattern Recognition, ,CVPR'05, June, IEEE Computer Society ,Vol. 1, pp. (886-893).
Deng J., Dong W., Socher R., Li L.J., Li K., and Fei-Fei, L., 2009, “ImageNet: A large-scale hierarchical image database”, IEEE Conference on Computer Vision and Pattern Recognition. (pp. 248-255).
Drăgulescu B., Bucos M., Vasiu R., 2015, “Predicting Assignment Submissions in a Multi-class Classification Problem”, TEM Journal, Vol. 4, No. 3, Pp.(244-254).
Duchi J., Hazan E. and Singer Y., 2011, “Adaptive subgradient methods for online learning and stochastic optimization”, Journal of Machine Learning Research, 12(Jul), pp. (2121-2159).
Floyhub Server, “Deep Learning Platform-Cloud GP”, https://www.floyhub.com/jobs, Visiting Date, Oct-2018.
He K., Zhang X., Ren S. and Sun J., 2016, ”Deep residual learning for image recognition”, in Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR). (pp. 770-778).
Hinton G. E., Srivastava N., Krizhevsky A., Sutskever I., SalakhutdinovR.R., 2012, “Improving neural networks by preventing co-adaptation of feature detectors”, arXiv:pp. (1207.0580).
Hoseini1 F., Shahbahrami A. and Bayat P., 2018, “An Efficient Implementation of Deep Convolutional Neural Networks for MRI Segmentation”, Journal of Digital Imaging, Vol. 31, No. 5, pp. (738-747).
Hoseini1 F., Shahbahrami A. and Bayat P., 2019, “AdaptAhead Optimization Algorithm for Learning Deep CNN Applied to MRI Segmentation”, Journal of Digital Imaging, Society of imaging informatics in medicine, Springer, Vol. 32, issue 1, Pp. (105-115).
Huang G., Liu Z., Van Der Maaten L. and Weinberger K.Q., 2017, “Densely connected convolutional networks”, in Proceedings of the IEEE conference on computer vision and pattern recognition. (pp. 4700-4708).
Hu J., Shen L. and Sun G., 2018, “Squeeze-and-excitation networks”, in Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141).
Johnson M., Schuster M., Le Q. V., Krikun M., Wu Y., Chen, Z., … Dean, J. (2017). Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation. Transactions of the Association for Computational Linguistics, Vol. 5, (pp. 339–351).
Kingma, D. P., and Ba, J. L., 2015, “Adam: A Method for Stochastic Optimization”, in Proceedings of the International Conference on Learning Representations (ICLR), pp. (1-15).
Korzeniowski F., 2018, “Experiments with AMSGrad” Retrieved December 24, 2018, from https://fdlm.github.io/post/amsgrad/.
Krizhevsky A., 2009, “Learning Multiple Layers of Features from Tiny Images”, Chapter 3, Object Classification Experiments, pp. (32-35).
Krizhevsky A., Sutskever I. and Hinton G.E., 2012, “ImageNet classification with deep convolutional neural networks”, Proceedings of the 25th International Conference on neural information processing systems (NIPS), Lake Tahoe, December, pp. (1097-1105).
Lin M., Chen Q. and Yan S., 2013, “Network In Network”, arXiv preprint: (pp. 1312.4400).
Lowe, D.G. (1999) Object Recognition from Local Scale-Invariant Features. Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, 20-27 September, pp. (1150-1157
Published
2020-06-02
How to Cite
MOHAMMED, A. I., & TAHIR, A. A. (2020). A NEW IMAGE CLASSIFICATION SYSTEM USING DEEP CONVOLUTION NEURAL NETWORK AND MODIFIED AMSGRAD OPTIMIZER. Journal of Duhok University, 22(2), 89-101. https://doi.org/10.26682/sjuod.2019.22.2.10
Section
Pure and Engineering Sciences