| fix(benchmark): address PR review feedback
- Run cargo test directly through testCmd instead of faking results
- Add 60s per-test timeout on vmExec to prevent hung suite
- Make report path configurable via --report flag (default: project root)
- Fix shell injection in testRewrite by escaping single quotes
Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>
| 1 个月前 |
| feat(benchmark): add multipass VM integration test suite
Bun/TypeScript orchestrator that creates an Ubuntu 24.04 VM via multipass,
installs all dev tools (Rust, Go, Node, Python, .NET, Terraform, etc.),
builds RTK, and runs 103 tests across 11 phases:
- Cargo quality (fmt, clippy, test)
- 47 Rust built-in commands (git, ls, grep, cargo, pytest, go, tsc...)
- 21 TOML filter commands (df, ps, shellcheck, hadolint, helm...)
- Hook rewrite engine (17 rewrite assertions)
- Exit code preservation
- Token savings verification (avg 81%)
- Pipe compatibility
- Edge cases (unicode, ANSI, empty output)
- Performance (memory < 20MB)
- Concurrency (10 parallel executions)
Usage:
bun run scripts/benchmark/run.ts # Full suite (~3 min)
bun run scripts/benchmark/run.ts --quick # Skip perf/concurrency
bun run scripts/benchmark/run.ts --phase 3 # Single phase
bun run scripts/benchmark/cleanup.ts # Delete VM
bun run scripts/benchmark/rebuild.ts # Fast rebuild
Prerequisites: brew install multipass, bun
Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>
| 1 个月前 |
| feat(benchmark): add multipass VM integration test suite
Bun/TypeScript orchestrator that creates an Ubuntu 24.04 VM via multipass,
installs all dev tools (Rust, Go, Node, Python, .NET, Terraform, etc.),
builds RTK, and runs 103 tests across 11 phases:
- Cargo quality (fmt, clippy, test)
- 47 Rust built-in commands (git, ls, grep, cargo, pytest, go, tsc...)
- 21 TOML filter commands (df, ps, shellcheck, hadolint, helm...)
- Hook rewrite engine (17 rewrite assertions)
- Exit code preservation
- Token savings verification (avg 81%)
- Pipe compatibility
- Edge cases (unicode, ANSI, empty output)
- Performance (memory < 20MB)
- Concurrency (10 parallel executions)
Usage:
bun run scripts/benchmark/run.ts # Full suite (~3 min)
bun run scripts/benchmark/run.ts --quick # Skip perf/concurrency
bun run scripts/benchmark/run.ts --phase 3 # Single phase
bun run scripts/benchmark/cleanup.ts # Delete VM
bun run scripts/benchmark/rebuild.ts # Fast rebuild
Prerequisites: brew install multipass, bun
Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>
| 1 个月前 |
| feat(benchmark): add multipass VM integration test suite
Bun/TypeScript orchestrator that creates an Ubuntu 24.04 VM via multipass,
installs all dev tools (Rust, Go, Node, Python, .NET, Terraform, etc.),
builds RTK, and runs 103 tests across 11 phases:
- Cargo quality (fmt, clippy, test)
- 47 Rust built-in commands (git, ls, grep, cargo, pytest, go, tsc...)
- 21 TOML filter commands (df, ps, shellcheck, hadolint, helm...)
- Hook rewrite engine (17 rewrite assertions)
- Exit code preservation
- Token savings verification (avg 81%)
- Pipe compatibility
- Edge cases (unicode, ANSI, empty output)
- Performance (memory < 20MB)
- Concurrency (10 parallel executions)
Usage:
bun run scripts/benchmark/run.ts # Full suite (~3 min)
bun run scripts/benchmark/run.ts --quick # Skip perf/concurrency
bun run scripts/benchmark/run.ts --phase 3 # Single phase
bun run scripts/benchmark/cleanup.ts # Delete VM
bun run scripts/benchmark/rebuild.ts # Fast rebuild
Prerequisites: brew install multipass, bun
Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>
| 1 个月前 |
| fix(benchmark): address PR review feedback
- Run cargo test directly through testCmd instead of faking results
- Add 60s per-test timeout on vmExec to prevent hung suite
- Make report path configurable via --report flag (default: project root)
- Fix shell injection in testRewrite by escaping single quotes
Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>
| 1 个月前 |