Release Notes

Version Mapping

Product Version Information

Product Name	Kunpeng BoostKit
Product Name	26.0.RC1
Software Name and Version	OmniOperator 2.1.0

Software Version Mapping

Item	Version
OS	CentOS 7.9, openEuler 20.03 LTS SP1, openEuler 22.03 LTS SP1
JDK	BiSheng JDK 1.8 (BiSheng JDK 1.8.0_342 preferred). openEuler 22.03 LTS SP1 is incompatible with BiSheng JDK 1.8.0_262 and must be replaced with BiSheng JDK 1.8.0_342.
Hadoop	3.2.0
Spark	3.1.1, 3.3.1, 3.4.3, 3.5.2
Hive	3.1.0
Python	3.10.2 or later
File system	HDFS
Data format	ORC, PARQUET

Hardware Version Mapping

Processor	Kunpeng 920 series processors
Memory Size	32 GB or greater

Virus Scan Result

The software packages, release documents, and product documents have been scanned by multiple antivirus software, and no virus is found.

Engine Name	QiAnXin
Engine Version	8.0.5.5260
Virus Lib Version	2026-03-10 08:00:00.0
Scan Time	2026-03-11 22:44:53
Scan Result	OK

Engine Name	Bitdefender
Engine Version	7.5.1.200224
Virus Lib Version	7.99958
Scan Time	2026-03-11 22:45:17
Scan Result	OK

Engine Name	Kaspersky
Engine Version	12.0.0.6672
Virus Lib Version	2026-03-11 10:04:00
Scan Time	2026-03-11 22:44:59
Scan Result	OK

V2.1.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Added InsertIntoHadoopFsRelationCommand to support HDFS insertion, WriteFile to support ORC write, Window to support array data segmentation, FileSourceScanExec to support array data read, and LocalLimitExec to support array data truncation.
• Added the expressions: datediff, pmod, charTypeWriteSideCheck, least, concat_ws, and get_json_object.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

V2.0.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Added the adaptation layer Gluten 1.3 for Spark.
• Added support for the concat_ws, regexp, regexp_replace, trim, and floor expressions to SparkExtension.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

V1.9.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Added column-based write in Parquet format for Spark 3.3.1.
• Added support for CentOS 7.9.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

V1.8.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Added support for Spark 3.4.3 and Spark 3.5.2.
• Added the LIMIT...OFFSET... syntax support to Spark 3.4.3 and 3.5.2.
• Added the expressions: try_add, try_divide, try_multiply, try_subtract, try_avg, and try_sum.
• Added support for the join type and build side co-directional logic of the open-source ShuffledHashJoin operator to Spark 3.5.2.
• Added the WindowGroupLimit operator for Spark 3.5.2 to optimize the ROW_NUMBER and RANK function performance for Top-K calculation.

Modified Features

None

Deleted Features

This version does not adapt to CentOS 7.9.

Resolved Issues

None

Known Issues

None

V1.7.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Optimized the Partial Agg logic to improve the query efficiency.
• Optimized the vectorized instructions of the Sort, HashAgg, and HashJoin operators.
• Optimized the execution plan for Spark operation acceleration in the Agg+Sort+Limit scenario and the Scan execution plan to reduce the performance overhead and improve the query efficiency.
• Added the ColumnarDataWritingCommandExec operator for Spark operator acceleration.
• Added stage-level operator rollback for Spark operator acceleration. In some scenarios, the performance loss caused by row-column conversion can be reduced.
• Added the timestamp data type for Spark operator acceleration.
• Added the unix_timestamp and from_unixtime expressions for Spark operator acceleration.
• Added SIMPLE_EDGE shuffle support for Hive operator acceleration and added fusion of the Filter and Select operators.
• Added the POWER expression for Hive operator acceleration.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

V1.6.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Optimized the prerequisites of bloomFilter and subquery broadcast to improve the reuse of subqueries.
• Added support for the greatest/contains expression, and skips the rollback of the filter operators that contain a scalar subquery expression.
• Optimized the TableScan, HashJoin, Shuffle, and RollUp operators.
• Optimized the OmniOperator deployment method, in which the Yarn resource management model enables the OmniOperator binary software package on which the Spark Executor process depends to be automatically deployed.

Modified Features

None

Deleted Features

None

Resolved Issues

Trouble Ticket No.	DTS2024060329127
Severity	Minor
Symptom	In a scenario where a Spark INSERT statement is executed with only one data partition, if 50 tables undergo consecutive Sort Merge Join (SMJ) operations, it may cause the SMJ operator to allocate vector memory using the new statement when off-heap memory is exhausted, thereby triggering a core dump issue.

Known Issues

None

V1.5.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Optimized memory for OmniOperator to support big wide table queries. The optimizations cover aggregate state memory usage, HashAggregator serialization memory usage, unified aggregator and operator memory allocation, and HashAggregator Spill sorting.
• Added the Not expression and the AnsiCast expression in the Spark insert scenario.
• Added support for the Hive engine. No exception occurs when Hive Extension executes 99 TPC-DS SQL statements. When vectorization is enabled, the performance for the ORC format is improved by 20% compared with that of the open-source Hive engine.
• Added support for more operators in Hive Extension, including Filter, Select, GroupBy, MapJoin, MergeJoin, PTF, Sort and TableScan.
• Added secure cluster support for ORC files in Hive Extension.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

Trouble Ticket No.	DTS2024060329127
Severity	Minor
Symptom	In a scenario where a Spark INSERT statement is executed with only one data partition, if 50 tables undergo consecutive Sort Merge Join (SMJ) operations, it may cause the SMJ operator to allocate vector memory using the new statement when off-heap memory is exhausted, thereby triggering a core dump issue.
Cause Analysis	Since OmniOperator currently uses columnar processing, compared to the row-based processing in the open-source Spark version, it consumes more memory. Additionally, the resources allocated during the SMJ operator's computation can only be released after the task completes. The problem occurs when INSERT statements are executed and there is only one data partition. Spark generates only one task. As a result, Sort Merge Join on 50 tables is executed in one task. In this case, the configured 38 GB off-heap memory is used up by the 50 consecutive SMJ operators during the calculation. When the new statement is used to apply for memory, a core dump occurs.
Impact Assessment	This test case falls under a high-load scenario. Spark jobs are designed to leverage the parallelism advantages of large-scale clusters, and under normal circumstances there is no service scenario where a single task (single thread) performs join operations across a large number of tables. This problem has not occurred in actual service scenarios and has little impact on customers.
Workaround	Adjust the **spark.memory.offHeap.size** parameter to increase the off-heap memory and run the service process again. Roll back to the open-source Spark version to trigger the service.
Progress	A troubleshooting case has been added to the *Feature Guide*. It helps quickly locate and rectify the fault. Resolve this problem in the next commercial release of Kunpeng BoostKit 24.0.0.

V1.4.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Optimized the execution plan and added three new rules: DeduplicateRightSideOfLeftSemiJoin, RewriteSelfJoinInInPredicate, and MergeSubqueryFilters.
• Added support for the NullType data type.
• Added the SubqueryBroadcastExec, CoalesceExecTransformer, and Limit Omni operators.
• Optimized operator functions: HashAggregator RollUp optimization, TableScan operator Parquet data read optimization, and Radix Sort for Sort operator.
• Optimized end expressions: adding the instr, startswith, and endswith functions, allowing conversion between the string type and int/long type, optimizing decimal data processing, and optimizing the expressions of the string type.
• Optimized functions in a Kerberos security cluster: operator acceleration in Spark local or Yarn mode, and ORC/Parquet data read in native mode.
• Added the spill function for Window and HashAggregator operators.
• Optimized NEON instructions, covering HashJoin, Sort, and Aggregator operators.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

V1.3.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Improved the performance of the 99 TPC-DS benchmark queries by 30%. The optimizations include vectorized computing of AVG/SUM aggregators, sort spills based on memory usage, shuffle write for fewer spills in temporary files, and join reorder without CBO.
• Added TopNSort operators and Sort-Merge Join and Sort integration.
• Added the table scan native processing for Parquet files, and security clusters for ORC and Parquet.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

V1.2.0

Change Description

The OmniRuntime OmniOperator feature of Kunpeng BoostKit for Big Data uses a unified infrastructure to support different engines (such as Spark), reducing repeated optimization work, fully exploring common and heterogeneous computing power, and promoting the Kunpeng ecosystem.

New Features

• Added support for the 99 TPC-DS benchmark queries.

  • Added support for LeftSemi Join in ShuffledHashJoin.
  • Added support for LeftAnti Join and LeftSemi Join in SortMergeJoin.
  • Added support for aggregation of non-grouped columns in HashAggregation.

• It optimizes the configuration items, decimal data type, aggregation operator, codegen expression, and vector data type.

Modified Features

None

Deleted Features

None

Resolved Issues

None

Known Issues

None

Related Documentation

Related Documentation

Document	Description	Delivery Method
2.1.0 Release Notes	Provides OmniStateStore version update and release information.	Open-source repository
Quick Start	Provides quick start tutorials to help users quickly understand and use OmniStateStore.	Open-source repository
Installation Guide	Provides guidance on how to install and deploy OmniStateStore.	Open-source repository
User Guide	Provides guidance on how to use OmniStateStore.	Open-source repository
FAQs	Records the issues that may occur during the installation, deployment, and use and their solutions.	Open-source repository

Obtaining Documentation

Visit the Open-source repository to view or download related documents.