RECENT METHODS OF USING THE INVERTED INDEX

Shrank Alexey Alexandrovich

The inverted index algorithm is one of the most popular algorithms used in search engines and enterprise document management systems. The simplicity of the algorithm makes it universal for application to any data representations, which has led to the creation of many adaptations and improvements. Examples of these improvements can be algorithms such as TF-IDF or BM25. The article considers the intrastate and international experience of using the inverted index, its modifications and ways of adapting to the specifics of recent search engines. The evolution and development vector of the algorithm, the pros and cons of various modifications and their applications are considered. At the moment, most search engines use neural language models. This leads to the forced use of embeddings as a way of presenting data. With the analysis of past decisions, a method for applying the inverted index algorithm to neural network embeddings was presented. This solution will allow the use of artificial intelligence where previously it was impossible due to the use of an inverted index, as well as improve search engines using neural network models.

Keywords:inverted index, neural networks, search, search engines, search algorithms, embedding, language models.