METHODS AND ALGORITHMS FOR PROCESSING TEXTS IN THE NATIONAL LANGUAGES OF RUSSIA: DEVELOPMENT AND TRAINING OF NEURAL NETWORKS

Nazipov Rustam Salavatovich

This article discusses methods and algorithms aimed at processing texts in the national languages of Russia using neural networks. Modern machine learning approaches, such as recurrent neural networks (RNN), long short-term memory networks (LSTM), and transformers, and their application to specific text processing tasks in national languages are described. Issues related to the limited amount of data and the high morphological complexity of these languages are taken into account. New methods and algorithms that can improve the accuracy and performance of models are proposed. The article also addresses text preprocessing issues, including tokenization, lemmatization, and morphological analysis, and their impact on modeling quality. Results of comparative analysis of different methods are presented, and directions for further research are identified.

Keywords:Text processing, AI, national languages, neural networks, machine learning, RNN, LSTM, transformers.