Modern trends in neural networks development

In the last 30 years, neural networks have been one of the most rapidly developing areas of artificial intelligence. They are widely used in sound and image processing, medicine, content analysis, generation problems and others. Significant increase in computing power, the ability to process large amounts of data, and the development of neural network theory itself made the advancement possible. The paper provides an analysis of the development of learning algorithms and neural network architectures from their emergence to the state-of-the-art. The most actively developing areas were reviewed including large language models, giant networks and multimodal models. Kolmogorov-Arnold networks were also indicated as perspective research area.