Star8
34
代码
代码
Issues3
Pull Requests7
项目讨论
Wiki
分析
Star8
34
已开启
Add dependency-hub ranking instrumentation (LINT-HUB-RANK) #875
zmw1创建于 8 天前
Add dependency-hub ranking instrumentation (LINT-HUB-RANK) #875
已开启
zmw1创建于 8 天前
从zmw1/third_party_typescriptarkts-lint-hub-rank合入到openharmony/third_party_typescriptmaster
共 3 个文件变更+187-1
Adocs/design/arkts-lint-hub-rank.md+82-0
@@ -0,0 +1,82 @@
1+# ArkTS Linter 依赖枢纽文件排名打点（LINT-HUB-RANK）设计
2+ 
3+| 项 | 内容 |
4+|---|---|
5+| 主题 | 通过一次全量编译，事前定位"被广泛依赖"的枢纽文件（修改会引发大规模增量重编/重查） |
6+| 改动文件 | `src/linter/ArkTSLinter_1_0/LinterRunner.ts`、`src/linter/ArkTSLinter_1_1/LinterRunner.ts` |
7+| 特性形态 | 纯增量式打点（日志输出），环境变量门控，默认零开销，不改变任何编译/lint 行为 |
8+ 
9+## 1. 背景与目标
10+ 
11+大型工程中普遍存在"枢纽文件"：它们被大量文件直接或传递依赖（典型如聚合再导出的 `Index.ets`、公共类型定义、底层工具模块）。修改这类文件时，增量构建会把它们的整个传递依赖锥拖入重查/重编，表现为"改一个文件、重编上千上万个"，增量退化为接近全量。
12+ 
13+事后从构建耗时或重查计数只能推断"发生了大范围重编"，无法直接回答"**哪些文件**是雷"。本打点在**一次全量编译**中利用完整的模块依赖图（builder state 的 `referencedMap`），直接计算每个文件的传递依赖锥大小并排名，为依赖治理（拆分再导出面、下沉公共代码、降耦合）提供量化目标清单。
14+ 
15+## 2. 用法
16+ 
17+```bash
18+# 全量构建时开启（hvigor 会将环境变量传递给 ets_loader 子进程）
19+ARKTS_LINT_HUB_RANK=1 hvigorw ... assembleHap 2>&1 | tee build.log
20+ 
21+# 可选：排名条数，默认 20
22+ARKTS_LINT_HUB_RANK_TOP=50
23+```
24+ 
25+构建日志中每个 linter 运行输出一行：
26+ 
27+```json
28+[LINT-HUB-RANK] {"ts":"...","linter":"ArkTS_1_1","totalFiles":43246,
29+ "filesWithDependents":9000,
30+ "top":[{"file":"/src/common/Index.ets","directDependents":210,"coneSize":11550,
31+ "sampleDependents":["/src/pages/a.ets","..."]}]}
32+```
33+ 
34+字段说明：
35+ 
36+| 字段 | 含义 |
37+|---|---|
38+| `totalFiles` / `filesWithDependents` | 程序总文件数 / 至少被 1 个文件依赖的文件数 |
39+| `file` | 枢纽文件路径 |
40+| `directDependents` | 直接依赖该文件的文件数（第一层扇入） |
41+| `coneSize` | **传递依赖锥大小 = 修改该文件会拖入重查/重编的文件数**（核心指标） |
42+| `sampleDependents` | 锥内前 5 个文件，便于快速确认依赖形态 |
43+ 
44+## 3. 算法与成本
45+ 
46+两阶段，避免对全部文件做锥计算（朴素做法为 O(V·(V+E))，不可接受）：
47+ 
48+```mermaid
49+flowchart TD
50+ ENV{"ARKTS_LINT_HUB_RANK=1?"} -->|否| ZERO["直接返回 - 零开销"]
51+ ENV -->|是| P1["第一阶段 O(V+E)<br/>遍历 referencedMap 统计每个文件<br/>直接依赖者数量并降序排序"]
52+ P1 --> P2["第二阶段 仅 Top-K x O(V+E)<br/>对前 K 个候选做反向 BFS<br/>求精确传递锥大小"]
53+ P2 --> OUT["输出 LINT-HUB-RANK 单行 JSON"]
54+```
55+ 
56+- 第一阶段：`BuilderState.getReferencedByPaths`（`builderState.ts:550`）反查每个文件的引用者，一次遍历完成；
57+- 第二阶段：仅对直接依赖者最多的前 K 个文件（默认 20）沿 `referencedMap` 反向 BFS；
58+- 成本估算：4.3 万文件工程约 2~6 秒，一次性发生（仅显式开启时）；多模块工程每个模块的 linter 运行各输出一行。
59+ 
60+指标口径：基于 linter program 的依赖图。对 lint 重查是精确值；编译产物的增量重编沿同一依赖图传播，`coneSize` 是其等价代理值。
61+ 
62+## 4. 兼容性
63+ 
64+| 维度 | 影响 |
65+|---|---|
66+| 编译/lint 行为 | 无任何改变——纯只读打点，函数整体 try/catch 包裹，异常不影响构建 |
67+| 默认开销 | 零（环境变量未设置时函数立即返回） |
68+| 缓存（.tsbuildinfo） | 不读不写，无格式影响 |
69+| 对外 API | `runArkTSLinter` 签名不变，调用方零改动 |
70+| 日志 | 新增单行 JSON 输出，量 = 打点次数（每模块每 linter 一行） |
71+ 
72+## 5. 测试
73+ 
74+| 编号 | 场景 | 预期 | 结果 |
75+|---|---|---|---|
76+| T1 | 5 文件链式依赖 harness（a←b←c、a←d），设 `ARKTS_LINT_HUB_RANK=1` | `a.ts` directDependents=2、coneSize=3（b/c/d）；`b.ts` 1/1；无依赖文件的 `d.ts` 不入榜 | ✅ 已验证（打包产物实测，数值与拓扑一致） |
77+| T2 | 不设环境变量 | 零输出、零行为差异 | ✅ 已验证 |
78+| T3 | `ARKTS_LINT_HUB_RANK_TOP=1` | 仅输出榜首 1 条 | ✅ 已验证 |
79+| T4 | 大工程全量构建 | 输出行出现，耗时增加 2~6s 量级；榜首文件与工程认知的公共模块吻合 | 待 CI/真实工程验证 |
80+| T5 | tsc 类型检查（两个改动文件） | 0 错误 | ✅ 已验证 |
81+ 
82+日志分析：`developtools/ace_ets2bundle/analyze_lint_incremental.py --hub-rank-log build.log` 可合并多模块输出并按锥大小输出全局排名。
Msrc/linter/ArkTSLinter_1_0/LinterRunner.ts+52-0
@@ -46,6 +46,7 @@ buildInfoWriteFile?: WriteFileCallback, arkTSVersion?: string): Diagnostic[] {
46 const oldDiagnostics = programState.arktsLinterDiagnosticsPerFile;46 const oldDiagnostics = programState.arktsLinterDiagnosticsPerFile;
47 programState.arktsLinterDiagnosticsPerFile = new Map();47 programState.arktsLinterDiagnosticsPerFile = new Map();
48 const changedFiles = collectChangedFilesFromProgramState(programState, arkTSVersion);48 const changedFiles = collectChangedFilesFromProgramState(programState, arkTSVersion);
49+ logDependencyHubs(programState, 'ArkTS_1_0');
49 // Set arkTSVersion info for file .tsbuildinfo.50 // Set arkTSVersion info for file .tsbuildinfo.
50 // File .tsbuildinfo.linter dosen't need to set arkTSVersion because it dosen't contain linter diagnostics.51 // File .tsbuildinfo.linter dosen't need to set arkTSVersion because it dosen't contain linter diagnostics.
51 programState.arkTSVersion = arkTSVersion;52 programState.arkTSVersion = arkTSVersion;
@@ -116,6 +117,57 @@ function releaseReferences(): void {
116 clearTrueSymbolAtLocationCache();117 clearTrueSymbolAtLocationCache();
117}118}
118 119 
120+// One-shot dependency-hub ranking, enabled with ARKTS_LINT_HUB_RANK=1 (optionally
121+// ARKTS_LINT_HUB_RANK_TOP=N, default 20). Run it during one full build: it lists the
122+// files with the largest transitive dependent cones - the hub files whose edit would
123+// drag thousands of files into re-checking/re-linting. Ranking is two-phase: direct
124+// dependent counts for every file (O(V+E)), then exact transitive cones only for the
125+// top candidates (K x O(V+E)). Env-gated, zero cost when disabled.
126+function logDependencyHubs(state: ReusableBuilderProgramState, linterLabel: string): void {
127+ try {
128+ if (process.env.ARKTS_LINT_HUB_RANK !== '1' || !state.referencedMap) {
129+ return;
130+ }
131+ const entries: [Path, number][] = [];
132+ state.fileInfos.forEach((_info, path) => {
133+ const direct = BuilderState.getReferencedByPaths(state, path).length;
134+ if (direct > 0) {
135+ entries.push([path, direct]);
136+ }
137+ });
138+ entries.sort((a, b) => b[1] - a[1]);
139+ const topN = Number(process.env.ARKTS_LINT_HUB_RANK_TOP) > 0 ? Number(process.env.ARKTS_LINT_HUB_RANK_TOP) : 20;
140+ const top = entries.slice(0, topN).map(([path, direct]) => {
141+ const cone = new Set<Path>();
142+ const queue: Path[] = [...BuilderState.getReferencedByPaths(state, path)];
143+ while (queue.length) {
144+ const current = queue.pop()!;
145+ if (!cone.has(current)) {
146+ cone.add(current);
147+ for (const dependent of BuilderState.getReferencedByPaths(state, current)) {
148+ queue.push(dependent);
149+ }
150+ }
151+ }
152+ return {
153+ file: path,
154+ directDependents: direct,
155+ coneSize: cone.size,
156+ sampleDependents: arrayFrom(cone.keys()).slice(0, 5)
157+ };
158+ });
159+ console.log(`[LINT-HUB-RANK] ${JSON.stringify({
160+ ts: new Date().toISOString(),
161+ linter: linterLabel,
162+ totalFiles: state.fileInfos.size,
163+ filesWithDependents: entries.length,
164+ top
165+ })}`);
166+ } catch {
167+ // instrumentation must never break the build
168+ }
169+}
170+ 
119function collectChangedFilesFromProgramState(state: ReusableBuilderProgramState, arkTSVersion?: string): Set<Path> {171function collectChangedFilesFromProgramState(state: ReusableBuilderProgramState, arkTSVersion?: string): Set<Path> {
120 const changedFiles = new Set<Path>(state.changedFilesSet);172 const changedFiles = new Set<Path>(state.changedFilesSet);
121 173 
Msrc/linter/ArkTSLinter_1_1/LinterRunner.ts+53-1
@@ -14,7 +14,7 @@
14 */14 */
15 15 
16import {16import {
17- ArkTSLinterTimePrinter, arrayFrom, BuilderProgram, Diagnostic, DiagnosticCategory, getBaseFileName,17+ ArkTSLinterTimePrinter, arrayFrom, BuilderProgram, BuilderState, Diagnostic, DiagnosticCategory, getBaseFileName,
18 isOHModules, Map, normalizePath, Path, PerformanceDotting, Program, resolvePath, ReusableBuilderProgramState, ScriptKind, Set, SourceFile,18 isOHModules, Map, normalizePath, Path, PerformanceDotting, Program, resolvePath, ReusableBuilderProgramState, ScriptKind, Set, SourceFile,
19 TimePhase, WriteFileCallback, MemoryUtils19 TimePhase, WriteFileCallback, MemoryUtils
20} from "../_namespaces/ts";20} from "../_namespaces/ts";
@@ -74,6 +74,7 @@ buildInfoWriteFile?: WriteFileCallback, arkTSVersion?: string): Diagnostic[] {
74 compilerOptions.compatibleSdkVersionStage74 compilerOptions.compatibleSdkVersionStage
75 );75 );
76 PerformanceDotting.stopAdvanced('collectChangedFilesFromProgramState');76 PerformanceDotting.stopAdvanced('collectChangedFilesFromProgramState');
77+ logDependencyHubs(programState, 'ArkTS_1_1');
77 // Set arkTSVersion info for file .tsbuildinfo.78 // Set arkTSVersion info for file .tsbuildinfo.
78 // File .tsbuildinfo.linter dosen't need to set arkTSVersion because it dosen't contain linter diagnostics.79 // File .tsbuildinfo.linter dosen't need to set arkTSVersion because it dosen't contain linter diagnostics.
79 programState.arkTSVersion = arkTSVersion;80 programState.arkTSVersion = arkTSVersion;
@@ -201,6 +202,57 @@ function releaseReferences(): void {
201 LibraryTypeCallDiagnosticChecker.instance.clear();202 LibraryTypeCallDiagnosticChecker.instance.clear();
202}203}
203 204 
205+// One-shot dependency-hub ranking, enabled with ARKTS_LINT_HUB_RANK=1 (optionally
206+// ARKTS_LINT_HUB_RANK_TOP=N, default 20). Run it during one full build: it lists the
207+// files with the largest transitive dependent cones - the hub files whose edit would
208+// drag thousands of files into re-checking/re-linting. Ranking is two-phase: direct
209+// dependent counts for every file (O(V+E)), then exact transitive cones only for the
210+// top candidates (K x O(V+E)). Env-gated, zero cost when disabled.
211+function logDependencyHubs(state: ReusableBuilderProgramState, linterLabel: string): void {
212+ try {
213+ if (process.env.ARKTS_LINT_HUB_RANK !== '1' || !state.referencedMap) {
214+ return;
215+ }
216+ const entries: [Path, number][] = [];
217+ state.fileInfos.forEach((_info, path) => {
218+ const direct = BuilderState.getReferencedByPaths(state, path).length;
219+ if (direct > 0) {
220+ entries.push([path, direct]);
221+ }
222+ });
223+ entries.sort((a, b) => b[1] - a[1]);
224+ const topN = Number(process.env.ARKTS_LINT_HUB_RANK_TOP) > 0 ? Number(process.env.ARKTS_LINT_HUB_RANK_TOP) : 20;
225+ const top = entries.slice(0, topN).map(([path, direct]) => {
226+ const cone = new Set<Path>();
227+ const queue: Path[] = [...BuilderState.getReferencedByPaths(state, path)];
228+ while (queue.length) {
229+ const current = queue.pop()!;
230+ if (!cone.has(current)) {
231+ cone.add(current);
232+ for (const dependent of BuilderState.getReferencedByPaths(state, current)) {
233+ queue.push(dependent);
234+ }
235+ }
236+ }
237+ return {
238+ file: path,
239+ directDependents: direct,
240+ coneSize: cone.size,
241+ sampleDependents: arrayFrom(cone.keys()).slice(0, 5)
242+ };
243+ });
244+ console.log(`[LINT-HUB-RANK] ${JSON.stringify({
245+ ts: new Date().toISOString(),
246+ linter: linterLabel,
247+ totalFiles: state.fileInfos.size,
248+ filesWithDependents: entries.length,
249+ top
250+ })}`);
251+ } catch {
252+ // instrumentation must never break the build
253+ }
254+}
255+ 
204function collectChangedFilesFromProgramState(256function collectChangedFilesFromProgramState(
205 state: ReusableBuilderProgramState,257 state: ReusableBuilderProgramState,
206 program: Program,258 program: Program,