Introduction
Index SDK Introduction
With the development of AI technologies in recent years, advanced algorithm models can effectively extract feature representations from unstructured data such as images, text, and speech. These are structured vector features. In practical scenarios, quickly and accurately finding vectors similar to the query vector has become an important requirement for many intelligent applications. Therefore, you need an efficient retrieval system based on vector features. One of its core components is an efficient retrieval engine.
Against this backdrop, Index SDK on the Huawei Ascend platform implements an efficient vector feature retrieval engine. You can build application-oriented retrieval systems on top of this engine.
FeatureRetrieval is a heterogeneous retrieval acceleration framework built on Faiss for Ascend NPUs. It provides high-performance retrieval for large-scale data in high-dimensional spaces. It uses C++ in the Faiss style together with TBE operators and supports the Arm and x86_64 platforms. FeatureRetrieval supports two retrieval library types: small-library search (full retrieval) and large-library search (approximate retrieval). Small libraries usually contain 300,000 to 1,000,000 records, whereas large libraries can reach tens of millions or even hundreds of millions of records. The supported feature vector dimensions range from 64 to 512 dimensions, depending on the algorithm.
-
Small-library search (full retrieval) mainly implements brute-force retrieval algorithms such as Flat, SQ, and INT8. It performs a full search over the feature vectors in the base library and returns the TopK results sorted by distance.
- The INT8 algorithm performs brute-force retrieval based on feature quantization. Therefore, it is also called "int8flat" (for example, the operator generation script
int8flat_generate_model.py). - The SQ algorithm uses internal quantization. Because it uses 8-bit integers for quantization, it is also called "SQ8" (for example,
sq8_generate_model.py).
- The INT8 algorithm performs brute-force retrieval based on feature quantization. Therefore, it is also called "int8flat" (for example, the operator generation script
-
Large-library search (approximate retrieval) implements the IVFSQ algorithm on the Ascend platform based on the Faiss feature retrieval framework and the IVF approach. Here, IVF differs from the traditional inverted index. Its basic idea is to cluster features first and then narrow the retrieval range through the cluster centers. This method trades accuracy for performance.
The underlying implementation of each algorithm uses TBE operators accelerated on the Ascend platform.
In addition, FeatureRetrieval supports attribute-filtered retrieval and batch retrieval across multiple indexes.
- Attribute-filtered retrieval lets you add time- and spatial-related attributes during base-library vector ingestion and then retrieve using base-library data under specific time and spatial conditions.
- Batch retrieval across multiple indexes lets you use multiple indexes to split libraries and retrieve multiple base libraries at once through a unified interface.
As a high-performance vector retrieval SDK, Index SDK offers the following benefits:
- Compatibility with mainstream frameworks: native Faiss APIs, ready to use.
- High performance: on the same compute card, full retrieval outperforms the industry standard, and batch retrieval outperforms serial scenarios.
- Large memory capacity: supports vector retrieval on a single card with a base library containing hundreds of millions of records.
Software Architecture
The software architecture of Index SDK is shown in Figure 1, software architecture, and the key modules in the architecture are described in Table 1, Index SDK module introduction.
Figure 1 Software architecture
Table 1 Index SDK module introduction
| Module | Description |
|---|---|
| Index SDK API layer | Provides Faiss-compatible C++ interfaces. Upper-layer applications can implement feature ingestion, query, deletion, and training functions. |
| Algorithm logic layer | Implements the logical flow of retrieval algorithms. The currently supported algorithms mainly include brute-force retrieval, approximate retrieval, and attribute-filtering algorithms. |
| Operator layer | Provides acceleration operators for retrieval algorithms on the Ascend platform, including distance computation operators, TopK sorting operators, and attribute-filtering mask operators. |
Getting Started
As shown in Figure 1 Index SDK usage process, using Index SDK for feature retrieval can be divided into the following steps.
Figure 1 Index SDK usage process
-
Deployment.
- Learn about the supported hardware form factors and OSs for Index SDK. See "Supported hardware and OSs".
- Learn how to install the required dependencies. See "Install dependencies".
- Obtain and verify the Index SDK software package. See "Obtain the Index SDK software package".
- Learn how to install and deploy Index SDK. See "Install Index SDK".
-
Determine the retrieval type and algorithm.
Learn about the retrieval types supported by Index SDK and the algorithms included in each retrieval type, including the use cases for each algorithm, the operators that need to be generated, and the sample descriptions. Analyze your actual service requirements and determine the retrieval type and algorithm you need to use. See "Algorithm introduction".
-
Generate operators.
Generate the operators required by the algorithm. See "Generate operators".
-
Call the API to implement the algorithm and obtain the retrieval results. See "API reference".
- The current Index SDK FeatureRetrieval is developed and adapted based on Ascend AI Processors and the open-source Faiss similarity search framework. Compatibility with any other hardware or heterogeneous computing platform is outside the scope of this document and the product.
- The deployment method for FeatureRetrieval is implemented through AscendCL interfaces. Therefore,
aclInithas already been called internally, and you do not need to call it again. - The maximum capacity supported by a single Index (base library) depends on the device-side memory size of the specific Ascend AI Processor. The service side needs to plan the number of indexes according to actual requirements to prevent memory overruns. You are advised to create fewer than 10,000 indexes. After you exceed 10,000, the resulting memory fragmentation is significant and may cause the capacity of the
addoperation to be smaller than expected.
Supported Hardware and OSs
| Product Series | Product Model | OSs (64-bit Only) |
|---|---|---|
| Atlas inference products | Atlas 300I Pro inference card | |
| Atlas 300V video analysis card | ||
| Atlas 300V Pro video analysis card | ||
| Atlas 300I Duo inference card | ||
| Atlas 200I SoC A1 core board | ||
| Atlas 200/300/500 inference products | Atlas 300I inference card (Model 3000) | |
| Atlas 300I inference card (Model 3010) | ||
| Atlas A2 inference products
Note: Atlas A2 inference products support the AscendIndexFlat and AscendIndexInt8Flat algorithms. |
Atlas 800I A2 inference server | |
| Atlas A3 inference products Note: Currently, only the AscendIndexFlat algorithm is supported. |
Atlas 800I A3 SuperNode server |