boostsra:基于鲲鹏平台的搜推广使能套件项目

鲲鹏boostkit社区介绍

Branch3Tags0

BoostSRA

Project Introduction

Overview

The BoostSRA suite delivers high-performance application-layer acceleration for search, recommendation, and advertising (SRA) services. It features advanced retrieval algorithms for recall scenarios and optimized model inference frameworks to enhance ranking performance.

Architecture

architecture

Community

BoostSRA

Recall Algorithms

Algorithm Description Repository Path
KBest Kunpeng Blazing-fast embedding similarity search thruster (KBest) is a proprietary, efficient graph-based retrieval algorithm. It is designed for large-scale embedding similarity search scenarios. KBest documentation
Only the documentation repository path is provided. The code repository will be released as open source at a later stage.
KScaNN Kunpeng Scalable Nearest Neighbors (KScaNN) is an inverted index-based vector retrieval algorithm that deeply optimizes index layout, algorithmic logic, and computing process to fully unlock the chip potential. KScaNN documentation
Only the documentation repository path is provided. The code repository will be released as open source at a later stage.
KRL Kunpeng Retrieval Library (KRL) is an operator library optimized for the Kunpeng platform to accelerate vector retrieval. KRL operators can be enabled to accelerate algorithms such as HNSW, PQFS, IVFPQ, and IVFPQFS—implemented in the open-source Facebook AI Similarity Search (Faiss) library. KRL documentation
Only the documentation repository path is provided. The code repository will be released as open source at a later stage.
KVecTurbo KVecTurbo is a proprietary vector retrieval acceleration component and can work with the openGauss vector database. It quantifies and compresses high-dimensional vectors to quickly obtain the near neighbors of a query. In addition, KVecTurbo uses the SIMD instructions to accelerate distance calculation for multidimensional Nearest Neighbor Search (NNS). KVecTurbo

Recall Algorithm Extensions

Algorithm Description Repository Path
PforDelta extension Engineered for the recall pipeline, it uses Kunpeng SIMD instructions to accelerate decompression of inverted index lists compressed with PForDelta.
hnswlib extension Optimized for Kunpeng processors, it supports FP16 data type through vectorization and leverages optimization policies such as prefetching and instruction rescheduling.
Faiss extension Tailored to the Kunpeng architecture, it integrates vectorization, dimension-interleaved lookup and accumulation, and vector filtering and compression.
RaBitQ extension It adapts the RaBitQ algorithm for the Arm Architecture 64-bit (AArch64) architecture, introducing FP16 precision optimization, NEON SIMD vectorization, assembly-level Lookup Table (LUT) acceleration, Spilling with Orthogonality-Amplified Residuals (SOAR) vector allocation, and ML-based adaptive nprobe.

Real-time Inference Extensions

Algorithm Description Repository Path
Embeddinglookup extension It reduces the lookup latency in the core modules of real-time recommendation systems through key techniques such as compiler option tuning, spinlock optimization, memory alignment optimization, and Arm SIMD vectorization.

Ranking Inference

Algorithm Description Repository Path
KDNN Based on the microarchitecture features of the Kunpeng processor, Kunpeng Deep Neural Network Library (KDNN) enhances the performance of core DNN operators through vectorization, assembly-level optimizations, and algorithmic improvements. KDNN documentation
Only the documentation repository path is provided. The code repository will be released as open source at a later stage.
ANNC Accelerated Neural Network Compiler (ANNC) speeds up neural network computing. It accelerates inference for recommendation systems and foundation models by optimizing computational graphs, generating and integrating high-performance fused operators, and applying efficient code generation and optimization. In addition, ANNC works with popular open-source inference frameworks. ANNC repository

Ranking Inference Extensions

Algorithm Description Repository Path
TensorFlow extension Optimized the TensorFlow framework for the Kunpeng platform, including native TensorFlow (TF) operator optimizations.
TensorFlow Serving extension Optimized the TensorFlow Serving framework for the Kunpeng platform, including thread scheduling optimizations.
TVM extension Optimized the TVM framework for the Kunpeng platform, including softmax operator optimizations.
oneDNN extension KDNN can be integrated into the open-source oneDNN as a plugin to provide full DNN capabilities.
TensorRT-LLM extension It includes features such as operator optimizations, memory access optimizations, and parameter configuration features.

Tools

Benchmark

SRA benchmark: includes the ranking inference performance benchmarks for multiple models, such as DLRM and Wide & Deep.

Documents

Kunpeng BoostKit for SRA Feature List.

Discussions

If you encounter any issues, join the discussion and contact us.

License Agreement

By using the source code and its accompanying software, you acknowledge that you have read, understood, and agreed to be bound by the terms and conditions of the software license agreement.

Introduction

鲲鹏boostkit社区介绍

Customize your domain