ANALYSIS OF MODERN FUNDAMENTAL ARCHITECTURAL APPROACHES AND METHODS FOR TRAINING LARGE LANGUAGE MODELS: FROM THE TRANSFORMER REVOLUTION TO A NEW PARADIGM OF EFFECTIVE ARTIFICIAL INTELLIGENCE

Belov Vyacheslav Viktorovich, Nikishov Sergey Ivanovich

This article presents a comprehensive analysis of the current state of research in the field of Large Language Models (LLMs). It examines the evolution of fundamental architectural solutions from the classical transformer to modern specialized approaches: decoder models with effective attention mechanisms, Mixture of Experts, and multimodal architectures. This book provides a detailed analysis of modern learning methods, including optimal scaling according to Chincha's law, advanced approaches to training data curation, methods for aligning with human preferences (RLHF, DPO), and effective fine-tuning strategies (PEFT, LoRA). Key trends are identified: the shift from extensive parameter augmentation to intelligent architecture design, the democratization of access through open models, and the shift toward next-generation multimodal systems. Particular attention is given to promising research areas, including state-space models for infinite contexts and hybrid architectures.

Keywords:Large language models, transformer architecture, mixed experts, multimodal learning, AI alignment, efficient fine-tuning, long context, open models