已关闭
[新需求]: Enable String Builder optimizations for strings #7554
Ivanov Mikhail创建于  2025年9月22日关闭于  2月11日
Ivanov Mikhail
2025年9月22日 创建

感谢提出新需求。

新需求提供了什么功能?

Enable String Builder optimizations for strings

该需求带来的价值、应用场景?

Test files list:

plugins/ets/tests/checked/ets_string_builder_append_merge*.ets
plugins/ets/tests/checked/ets_string_builder_hoist.ets
plugins/ets/tests/checked/ets_string_builder_merge*.ets
plugins/ets/tests/checked/ets_string_concat*.ets
plugins/ets/tests/checked/ets_stringbuilder*.ets
likedislike
IIvanov Mikhail
2025年9月22日 添加了label:enhancement
openharmony_ci
openharmony_ci成员
2025年9月22日 评论:

感谢提交Issue!关于Issue的交互操作,请访问OpenHarmony社区支持命令清单。如果有问题,请联系 [@weng-changcheng](https://gitcode.com/weng-changcheng) [@Prof](https://gitcode.com/Prof) [@dingding5](https://gitcode.com/dingding5) [@jjpei](https://gitcode.com/jjpei) [@v-cherkashin](https://gitcode.com/v-cherkashin) 。如果需要调整订阅PR、Issue的变更状态,请访问链接。


Thanks for submitting the issue. For more commands, please visit OpenHarmony Command List. If you have any questions, please refer to committer gitcode for help. If you need to change the subscription of a Pull Request or Issue, please visit the link.

likedislike
openharmony_ciopenharmony_ci成员
2025年9月22日 添加了label:waiting_for_assign
IIvanov Mikhail
2025年9月22日 修改了issue 的描述
ipm
ipm
2025年9月25日 评论:

Some SB checked tests failed at the moment,
2bec2e8726924d64bb1206edca0e5142.gz

likedislike
Ivanov Mikhail
2025年9月25日 评论:

Some SB checked tests failed at the moment,
2bec2e8726924d64bb1206edca0e5142.gz

Try to remove these lines 1092-1094 from simplify_string_builder.cpp and rerun tests

            if (inputInst->GetBasicBlock() != loopBlock) {
                continue;
            }
likedislike
IIvanov Mikhail
2025年9月25日 修改了issue 的描述
Ivanov Mikhail
2025年9月25日 评论:

StringCalculation.ets

function GenerateFakeRandomInteger(): Int32Array {
  let resource: Int32Array = new Int32Array([12, 43, 56, 76, 89, 54, 45, 32, 35, 47, 46, 44, 21, 37, 84]);
  return resource;
}

// String运算
function StringCalculation() {
  let count: int = 3000000 / 1000;
  let str1: string = "h";
  let res: string = "11";
  let resources: Int32Array = GenerateFakeRandomInteger();
  let start: number = Date.now();
  for (let i: int = 0; i < count; i++) {
    if (resources[i % res.length % 15] > resources[(i + resources[i % 15]) % res.length % 15]) {
      res += str1 + i;
    } else {
      res += str1;
    }
  }
  let end: number = Date.now();
  console.log(""+res.length);
  let time = end - start
  console.log("Numerical Calculation - StringCalculation:\t"+String(time)+"\tms");
  return time;
}
likedislike
valentinagetmanovavalentinagetmanova
2025年9月29日 关联了pull request:Enable String Builder for BCO
alexanderandreevmir
alexanderandreevmir
2025年9月29日 评论:

b2a56d3de66347cd8810026fcfdc03f8.gz
Possible fixes fo String Builder (no all tests passed).

likedislike
ipm
ipm
2025年9月29日 评论:

Some SB checked tests failed at the moment,
2bec2e8726924d64bb1206edca0e5142.gz

Try to remove these lines 1092-1094 from simplify_string_builder.cpp and rerun tests

           if (inputInst->GetBasicBlock() != loopBlock) {
               continue;
           }

That not fixes SimplifyStringBuilder::OptimizeStringConcatenation.

likedislike
ipm
ipm
2025年9月29日 评论:

b2a56d3de66347cd8810026fcfdc03f8.gz
Possible fixes fo String Builder (no all tests passed).

That's possibly correct fix of SimplifyStringBuilder::OptimizeStringConcatenation for compile time errors of ets_string_concat_loop.checked test. But checker errors of that test still remains:
: Test failed: CHECKER AOT IR Builder, check String concatenation loop
: Method: "ets_string_concat_loop.ETSGLOBAL::concat_loop0"
: Pass before: BranchElimination
: IR not found: /StringBuilder::<ctor>/.
That's because of SimplifySB pass completed from BCO at frontend stage.

Moreover, that not fixes compile-time errors of tests:

  1. ets_string_builder_merge.checked :
    Instruction doesn't dominate its user
    input: bb 1 8.ref LoadAndInitClass 'std.core.StringBuilder' v137 -> (v111, v92, v61, v41, v9)
    user: bb 1 61.ref NewObject 561 v8, v59 -> (v74, v71, v66, v62)
    Method: ets_string_builder_merge.ETSGLOBAL::chain_concat0 0x1581.
  2. ets_stringbuilder_length.checked :
    Instruction doesn't dominate its user
    input: bb 1 53.ref LoadClass 'std.core.String' v196 -> (v138, v155, v54)
    user: bb 6 138. CheckCast 466 v135, v53, v136
    Method: ets_stringbuilder_length.ETSGLOBAL::StringCalculation02 0x604.

Obviously more fixes of SimplifyStringBuilder should be performed.

likedislike
Ivanov Mikhail
2025年9月30日 评论:

base64..ets

const TIME_MULTIPLIER: int = 1000;
const TOBASE64_NUMBER_2: int = 2;
const TOBASE64_NUMBER_3: int = 3;
const TOBASE64_NUMBER_4: int = 4;
const TOBASE64_NUMBER_6: int = 6;
const TOBASE64_NUMBER_8: int = 8;
const TOBASE64_NUMBER_25: int = 25;
const TOBASE64_NUMBER_97: int = 97;
const TOBASE64_NUMBER_120: int = 20;
const TOBASE64_NUMBER_8192: int = 8192;
const TOBASE64_NUMBER_16384: int = 16384;

const TOBASE64_NUMBER_0X03: int = 0x03;
const TOBASE64_NUMBER_0X0F: int = 0x0f;
const TOBASE64_NUMBER_0X3F: int = 0x3f;
const TOBASE64_NUMBER_0X7F: int = 0x7f;
const TOBASE64_NUMBER_0XFF: int = 0xff;

let inDebug = false;
function log(str: string): void {
  if (inDebug) {
    console.println(str);
  }
}
function currentTimestamp13(): number {
  return Date.now();
}

/* Convert data (an array of integers) to a Base64 string. */
let toBase64Table = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/';
let base64Pad = '=';

function toBase64(data: string): string {
  let resultStr = '';
  let length = data.length.toInt();
  let i = 0;

  // Convert every three bytes to 4 ascii characters.
  while (i < length - TOBASE64_NUMBER_2) {
    resultStr += toBase64Table[(data.charCodeAt(i) >> TOBASE64_NUMBER_2).toInt()];
    resultStr +=
      toBase64Table[
      ((data.charCodeAt(i) & TOBASE64_NUMBER_0X03) << TOBASE64_NUMBER_4).toInt() +
      (data.charCodeAt(i + 1) >> TOBASE64_NUMBER_4).toInt()
      ];
    resultStr +=
      toBase64Table[
      ((data.charCodeAt(i + 1) & TOBASE64_NUMBER_0X0F) << TOBASE64_NUMBER_2).toInt() +
      (data.charCodeAt(i + TOBASE64_NUMBER_2) >> TOBASE64_NUMBER_6).toInt()
      ];
    resultStr += toBase64Table[(data.charCodeAt(i + TOBASE64_NUMBER_2) & TOBASE64_NUMBER_0X3F).toInt()];
    i += TOBASE64_NUMBER_3;
  }

  // Convert the remaining 1 or 2 bytes, pad out to 4 characters.
  if (length % TOBASE64_NUMBER_3) {
    i = length - (length % TOBASE64_NUMBER_3);
    resultStr += toBase64Table[(data.charCodeAt(i) >> TOBASE64_NUMBER_2).toInt()];
    if (length % TOBASE64_NUMBER_3 === TOBASE64_NUMBER_2) {
      resultStr +=
        toBase64Table[
        ((data.charCodeAt(i) & TOBASE64_NUMBER_0X03) << TOBASE64_NUMBER_4).toInt() +
        (data.charCodeAt(i + 1) >> TOBASE64_NUMBER_4).toInt()
        ];
      resultStr += toBase64Table[((data.charCodeAt(i + 1) & TOBASE64_NUMBER_0X0F) << TOBASE64_NUMBER_2).toInt()];
      resultStr += base64Pad;
    } else {
      resultStr += toBase64Table[((data.charCodeAt(i) & TOBASE64_NUMBER_0X03) << TOBASE64_NUMBER_4).toInt()];
      resultStr += base64Pad + base64Pad;
    }
  }

  return resultStr;
}

/* Convert Base64 data to a string */
let toBinaryTable: FixedArray<int> = [
  -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, //binary1
  -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1,
  -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, 62, -1, -1, -1, 63, //binary2
  52, 53, 54, 55, 56, 57, 58, 59, 60, 61, -1, -1, -1, 0, -1, -1,
  -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, //binary3
  15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, -1, -1, -1, -1, -1,
  -1, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, //binary4
  41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, -1, -1, -1, -1, -1
];

function base64ToString(data: string): string {
  let result = '';
  let leftbits = 0; // number of bits decoded, but yet to be appended
  let leftdata = 0; // bits decoded, but yet to be appended

  // Convert one by one.
  let data_len:int = data.length.toInt()
  for (let i = 0; i < data_len; i++) {
    let c = toBinaryTable[(data.charCodeAt(i) & TOBASE64_NUMBER_0X7F).toInt()];
    let padding = data.charCodeAt(i) === base64Pad.charCodeAt(0);
    // Skip illegal characters and whitespace;
    if (c === -1) {
      continue;
    }

    // Collect data into leftdata, update bitcount;
    leftdata = (leftdata << TOBASE64_NUMBER_6) | c;
    leftbits += TOBASE64_NUMBER_6;

    // If we have 8 or more bits, append 8 bits to the result;
    if (leftbits >= TOBASE64_NUMBER_8) {
      leftbits -= TOBASE64_NUMBER_8;
      // Append if not padding;
      if (!padding) {
        result += String.fromCharCode((leftdata >> leftbits) & TOBASE64_NUMBER_0XFF);
      }
      leftdata &= (1 << leftbits) - 1;
    }
  }

  // If there are any bits left, the base64 string was corrupted;
  if (leftbits > 0) {
    //log("Corrupted base64 string")
  }
  return result;
}

function run(): void {
  let str = '';

  for (let i = 0; i < TOBASE64_NUMBER_8192; i++) {
    str += String.fromCharCode(TOBASE64_NUMBER_25 * Math.random() + TOBASE64_NUMBER_97);
  }

  let i = TOBASE64_NUMBER_8192;
  while (i <= TOBASE64_NUMBER_16384) {
    let base64: string;
    base64 = toBase64(str);
    //log(`string-base64: toBase64 ${i} = ${base64}`)
    let encoded = base64ToString(base64);
    //log(`string-base64: toBase64 ${i} = ${encoded}`)
    if (encoded !== str) {
      throw new Error(`bad result: expected ${str} but got ${encoded}`);
    }
    // Double the string
    str += str;
    i *= TOBASE64_NUMBER_2;
  }
}

/*
 * @State
 * @Tags Jetstream2
 */
class Benchmark {
  /*
   * @Benchmark
   */
  runIteration(): void {
    for (let i = 0; i < TOBASE64_NUMBER_8; i++) {
      let startTimeInLoop = currentTimestamp13();
      run();
      let endTimeInLoop = currentTimestamp13();
      //log(`base64: index= ${i} time= ${endTimeInLoop - startTimeInLoop}`)
    }
  }
}

// declare interface ArkTools {
//   timeInUs(args: int): int;
// }
class ArkBenchMark {
  static run() {
    let startTime = currentTimestamp13();
    let benchmark = new Benchmark();
    for (let i = 0; i < 3; i++) {
      benchmark.runIteration();
    }
    let endTime = currentTimestamp13();
    console.println(`base64: ms = ${endTime - startTime}`);
  }
}
likedislike
ipm
ipm
2025年9月30日 评论:
likedislike
valentinagetmanovavalentinagetmanova
2025年10月1日 关联了pull request:Enable String Builder for BCO
IIvanov Mikhail
2025年10月1日 修改了issue 的描述
ipm
ipm
2025年10月7日 评论:

Some improvements & fixes added to https://gitcode.com/valentinagetmanova/arkcompiler_runtime_core/tree/string_builder_opt .

  1. Evaluation of dynamic Ruby expressions added for RUN_BCO test function. This can be used to recompile input abc test file with options required by test checker from ets source file. More checker.rb functions may be upgraded later to support dynamic Ruby expressions from ets test checkers to improve flexibility of ets checked tests.
  2. Test ets_string_concat_loop.checked fixed & upgraded to run SimplifyStringBuilder AOT/JIT tests without SSB BCO pass at precompile time. New checks added to test SSB BCO at compile time with subsequent test execution in cpp & irtoc interpreter modes.
  3. Externally compiled ets_string_concat_loop.checked test.abc not used at the moment due to specific test requirements which stored in ets source file instead of CMakeLists or somewhere else.
likedislike
IIvanov Mikhail
2025年10月10日 关联了pull request:Add fake dependency between String.length and StringBuilder.stringLength
IIvanov Mikhail
2025年10月10日 关联了pull request:Add StringBuilder.stringLength getter to a black list as an exception
valentinagetmanova
valentinagetmanova
2025年10月10日 评论:

A task with the IR unit test. Some instruction is created to generate a graph.

INST(3U, Opcode::LoadAndInitClass).ref().Inputs(2U).Class(graph->GetRuntime()->GetStringBuilderClass());

The result:
3.ref LoadAndInitClass 'UnknownClass' v2 -> (v4)

Expectations
3.ref LoadAndInitClass 'std.core.StringBuilder' v2 -> (v4)

The problem is passing the class name in the instructions.

The following approach was tried to solve the problem
auto runtime = graph->GetRuntime();
auto sbPtr = runtime->GetStringBuilderClass();

The result:
class ptr: 0x10044453d
IsClassStringBuilder: false

This is caused by the absence of the panda file (a fake file is being used). The class was not passed to the instruction when it was created.
All trying to pass to std.core.StringBuilder had the result ‘’ or 'UnknownClass'

likedislike
ipm
ipm
2025年10月11日 评论:

Some improvements & fixes added to https://gitcode.com/valentinagetmanova/arkcompiler_runtime_core/tree/string_builder_opt .

  1. Evaluation of dynamic Ruby expressions added for RUN_BCO test function. This can be used to recompile input abc test file with options required by test checker from ets source file. More checker.rb functions may be upgraded later to support dynamic Ruby expressions from ets test checkers to improve flexibility of ets checked tests.
  2. Test ets_string_concat_loop.checked fixed & upgraded to run SimplifyStringBuilder AOT/JIT tests without SSB BCO pass at precompile time. New checks added to test SSB BCO at compile time with subsequent test execution in cpp & irtoc interpreter modes.
  3. Externally compiled ets_string_concat_loop.checked test.abc not used at the moment due to specific test requirements which stored in ets source file instead of CMakeLists or somewhere else.

More ets checked tests fixed to run with AOT/JIT and extended with BCO SimplifySB checks: ets_stringbuilder_length, ets_string_builder_merge, ets_string_builder_merge_uber_export.
CMakeLists fixed to support verbose execution of checker.rb if build configured with -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON.

likedislike
alexanderandreevmir
alexanderandreevmir
2025年10月13日 评论:

Benchmark results are very good.
Increase in some benchmarks (top30) up to 85%.

Need to RUN FULL CI.

likedislike
alexanderandreevmir
alexanderandreevmir
2025年10月22日 评论:

At the moment, two jobs are falling in the CI:

  1. unknown problem with the test - AccessNBody.ets.
    There is no use of StringBuilder pass in the test, but we are getting assertion when making calculations in it.
    Localy it isn't produced (X86-64, Arm64).

  2. unknown problem with the test - Device Tests. CI says there has been degradation...but it's not clear where.
    All tests into this Job - passed.

There are many reviewers comments related to PR.

likedislike
alexanderandreevmir
alexanderandreevmir
2025年10月27日 评论:

All reviewer comments were responded to and fixed.
CI is GREEN!!!!
PR was merged.

likedislike
openharmony_dcp
openharmony_dcp成员
2月11日 评论:

开发自提单,长时间未处理,经领域审视,issue先关闭

likedislike
openharmony_dcpopenharmony_dcp成员
2月11日 issue状态由 待办的 改变为 已完成
openharmony_dcpopenharmony_dcp成员
2月11日 关闭了 issue