Appropriate Selection for Numbers of neurons and layers in a Neural Network Architecture: A Brief Analysis

Asif Aziz; Talha Khan; Umar Iftikhar; Irfan Tanoli; Asif khalid Qureshi

doi:10.33317/ssurj.438

Authors

Asif Aziz Bahria University karachi campus
Talha Khan Multimedia University Malaysia
Umar Iftikhar
Irfan Tanoli
Asif khalid Qureshi

DOI:

https://doi.org/10.33317/ssurj.438

Keywords:

Neural Network, Neurons, Layers, Neural Network Architecture, Perceptron

Abstract

Identification of optimal number of neurons and layers in a proposed neural architecture is very complex for the better results. The determination of the hidden layer number is also very difficult task for the proposed network. The recognition of the effective neural network model in terms of accuracy and precision in results as well as in terms of computational resources is very crucial in the community of the computer scientists. An effective proposed neural network architecture must comprise the appropriate numbers of perceptrons and number of layers. Another research gap was also reported by the researchers community that the perceptron stuck during the training phase in finding minima or maxima for stochastic gradient to solve any engineering application. Therefore to resolve the problem of selection of neurons and layers an analysis was performed to evaluate the performance of the neural network architecture with different neurons and layers on the same data set. The results revealed that the justified network architecture would contain justified number of neurons and layers as more number of neurons and layers increase more computational resources and training time. It was suggested that a neural network architecture should be proposed comprising of minimum 2 to 5 layers. Entropy and Mean square error was considered as a yardstick to measure the neural network architecture performance. Results depicted that the an effective neural network architecture must initially be simulated or checked with minimum number of instances to evaluate the model.

References

Thomas, L., Kumar, M., & Annappa, B. (2016, August). Discoveryof optimal neurons and hidden layers in feed-forward Neural DOI: https://doi.org/10.1109/EmergiTech.2016.7737353

Network. In 2016 IEEE International conference on emerging technologies and innovative business practices for the

transformation of societies (EmergiTech) (pp. 286-291). IEEE.

Khare, S. K., & Bajaj, V. (2020). Time–frequency representation and convolutional neural network-based emotion recognition. IEEE DOI: https://doi.org/10.1109/TNNLS.2020.3008938

transactions on neural networks and learning systems, 32(7), 2901-

Shafi, I., Ahmad, J., Shah, S. I., & Kashif, F. M. (2006, December). Impact of varying neurons and hidden layers in neural network

architecture for a time frequency application. In 2006 IEEE International Multitopic Conference (pp. 188-193). IEEE.

Kouhara, T., & Okabe, Y. (1993, October). Learning algorithm based on moderationism for multi-layer neural networks. In

Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan) (Vol. 1, pp. 487-490). IEEE.

Maruno, S., Kohda, T., Kojima, Y., Sakaue, S., Yamamoto, H., & Shimeki, Y. (1991, November). Multifunctional layered network

with quantizer neurons. In [Proceedings] 1991 IEEE International Joint Conference on Neural Networks (pp. 541-546). IEEE.

Ninomiya, H., & Sasaki, A. (2002, May). A study on generalization ability of 3-layer recurrent neural networks. In Proceedings of the

International Joint Conference on Neural Networks. IJCNN'02 (Cat. No. 02CH37290) (Vol. 2, pp. 1063-1068). IEEE.

Abe, S., Kayama, M., & Takenaga, H. (1991, November). Synthesizing neural networks for pattern recognition. In DOI: https://doi.org/10.1109/IJCNN.1991.170544

[Proceedings] 1991 IEEE International Joint Conference on Neural Networks (pp. 1105-1110). IEEE.

Zhang, M., Xu, S., & Lu, B. (1999, July). Neuron-adaptive higher order neural network group models. In IJCNN'99. International

Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339) (Vol. 1, pp. 333-336). IEEE.

Ishii, N., Deguchi, T., Kawaguchi, M., & Sasaki, H. (2011, May). Colaborative Activities of Layered Neural Network. In 2011 First DOI: https://doi.org/10.1109/CNSI.2011.37

ACIS/JNU International Conference on Computers, Networks, Systems and Industrial Engineering (pp. 417-422). IEEE.

Huang, W., Oh, S. K., & Pedrycz, W. (2017). Hybrid fuzzy wavelet neural networks architecture based on polynomial neural networks

and fuzzy set/relation inference-based wavelet neurons. IEEE transactions on neural networks and learning systems, 29(8), 3452-

Wang, Q., Aoyama, T., & Nagashima, U. (2002, May). High precision expansion of multi-layer neural networks. In Proceedings

of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No. 02CH37290) (Vol. 2, pp. 1493-1498). IEEE.

Horita, T., Murata, T., & Takanami, I. (2006, October). A multiple- weight-and-neuron-fault tolerant digital multilayer neural network. DOI: https://doi.org/10.1109/DFT.2006.8

In 2006 21st IEEE International Symposium on Defect and Fault Tolerance in VLSI Systems (pp. 554-562). IEEE.

Li, S., Chen, Y., & Leiss, E. L. (1994, June). GMDP: a novel unified neuron model for multilayer feedforward neural networks. In

Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94) (Vol. 1, pp. 107-112). IEEE.

Shin-ike, K. (2010, August). A two phase method for determining the number of neurons in the hidden layer of a 3-layer neural

network. In Proceedings of SICE Annual Conference 2010 (pp. 238- 242). IEEE.

Aizenberg, I., Paliy, D. V., Zurada, J. M., & Astola, J. T. (2008). Blur identification by multilayer neural network based on DOI: https://doi.org/10.1109/TNN.2007.914158

multivalued neurons. IEEE Transactions on Neural Networks, 19(5), 883-898.

Khan, T. A., Alam, M. M., Shahid, Z., & Su’ud, M. M. (2020). Prior recognition of flash floods: concrete optimal neural network

configuration analysis for multi-resolution sensing. IEEE Access, 8,

-210022.

Khan, T. A., Alam, M. M., Shahid, Z., & Su’Ud, M. M. (2020). Investigation of flash floods on early basis: A factual comprehensive review. IEEE Access, 8, 19364-19380. DOI: https://doi.org/10.1109/ACCESS.2020.2967496

Khan, T. A., Alam, M., Shahid, Z., Ahmed, S. F., & Mazliham, M. S. (2018, November). Artificial Intelligence based Multi-modal

sensing for flash flood investigation. In 2018 IEEE 5th International Conference on Engineering Technologies and Applied Sciences

(ICETAS) (pp. 1-6). IEEE.

Khan, T. A., Alam, M., Kadir, K., Shahid, Z., & Mazliham, M. S. (2018, July). A Novel approach for the investigation of flash floods

using soil flux and CO 2: An implementation of MLP with less false alarm rate. In 2018 2nd International Conference on Smart Sensors

and Application (ICSSA) (pp. 130-134). IEEE.

Karabayir, I., Akbilgic, O., & Tas, N. (2020). A novel learning algorithm to optimize deep neural networks: evolved gradient DOI: https://doi.org/10.1109/TNNLS.2020.2979121

direction optimizer (EVGO). IEEE transactions on neural networks and learning systems, 32(2), 685-694.

Romlay, M. R. M., Rashid, M. M., & Toha, S. F. (2016, July). Development of Particle Swarm Optimization Based Rainfall-

Runoff Prediction Model for Pahang River, Pekan. In 2016 International Conference on Computer and Communication Engineering (ICCCE) (pp. 306-310). IEEE.

Zhang, S., Lu, L., Yu, J., & Zhou, H. (2016, July). Short-term water level prediction using different artificial intelligent models. In 2016 DOI: https://doi.org/10.1109/Agro-Geoinformatics.2016.7577678

Fifth International Conference on Agro-Geoinformatics (Agro- Geoinformatics) (pp. 1-6). IEEE.

Fister Jr, I., Fister, D., & Fister, I. (2013). A comprehensive review of cuckoo search: variants and hybrids. International Journal of

Mathematical Modelling and Numerical Optimisation, 4(4), 387- 409.

Solanki, L. S., Singh, S., & Singh, D. (2016, July). An ANN approach for false alarm detection in microwave breast cancer DOI: https://doi.org/10.1109/CEC.2016.7743948

detection. In 2016 IEEE Congress on Evolutionary Computation (CEC) (pp. 1370-1374). IEEE.

Khan, T. A., Alam, M., Kadir, K., Shahid, Z., & Mazliham, M. S. (2019). A comparison review based on classifiers and regression

models for the investigation of flash floods. Editorial Preface Fromthe Desk of Managing Editor…, 10(9).