Efficiency of particle swarm-based methods in optimizing ICLPSO training of neural networks

Larionov Vyacheslav Sergeevich, Safiullina Lina Khatypovna

Subject of research: The possibility of using the particle swarm optimization algorithm (PSO) and its modifications (CLPSO and ICLPSO variance) in classification problems instead of the error back-propagation method (EBP) for optimizing the learning process of feedforward neural networks (FNN). The work aims to search for more advanced FNN optimization algorithms. Five datasets with various lengths and unknown weight numbers were used for calculations; neural network training was carried out for 500 and 1000 epochs in 50 independent program runs. For each of the loss functions used, activation functions are selected for each of the layers which lead the network to give the best output value: sigmoid (with MSE used, for hidden and output layer neurons), hyperbolic tangent (with Cross-Entropy used, for hidden layer neurons) or Softmax (with Cross-Entropy used, for output layer neurons). The accuracy analysis showed an advantage of particle swarm optimization methods only in the case of 1000 epochs, which leads to increased computing power required for training. To improve the accuracy of classification using particle swarm optimization methods, it is also necessary to increase their total number (that is, to increase the weight numbers)

Keywords:neural network, optimization, EBP, PSO, CLPSO, ICLPSO.