Configuration
Reference for site-config.toml, the Site's configuration file. This is the field reference (every field, its type, default, and code source). For a guided walk-through of setting up Site and webhooks see the deploying-site guide and the platform-integration guide.
Overview
- Location: the Site reads
site-config.tomlfrom the current working directory (site/src/load.rs:478-496). When the file is absent, an equivalent config is built from environment variables (see §7). - Format: TOML, parsed with
#[serde(deny_unknown_fields)]— unknown keys abort startup. Schema:Config(site/src/load.rs:226). - Reload: the config is loaded once at startup; restart Site to apply changes.
A complete annotated example lives at site-config.toml in the repo root.
1. [keys] — platform credentials
Per-platform API token and webhook secret. Each platform section ([keys.github], [keys.gitcode]) is optional:
- Absent → the site starts in degraded mode for that platform: platform API calls and webhook verification for repos on that platform will fail (logged as errors, not fatal) —
load.rs:299-349. - Present → both
tokenandsecretare required and non-empty; an empty value aborts startup —load.rs:327-346.
| Field | Type | Required | Description | Source |
|---|---|---|---|---|
token |
string | yes (when section present) | API token for outbound platform calls (post comments, fetch PR/commit, collaborator check). GitHub uses HTTP Basic auth with username rust-bench; GitCode uses Bearer. |
load.rs:47 |
secret |
string | yes (when section present) | Webhook secret for HMAC-SHA256 verification. Aliased as webhook_secret. |
load.rs:49 |
2. [site] — site URL
| Field | Type | Default | Description | Source |
|---|---|---|---|---|
url |
string (optional) | http://localhost:2346 |
Base URL used in benchmark-completion comments (per-tag and cross-machine comparison links). Must be reachable from the platform's webhook source. | load.rs:216-221, 258-260 |
3. [repos.<key>] — repositories
A map of repo keys to RepoConfig (load.rs:55). Default: a single rust repo pointing at github/rust-lang/rust. Each entry:
| Field | Type | Default | Description | Source |
|---|---|---|---|---|
repo |
string | (none) | Repository identifier <platform>/<owner>/<repo> (e.g. gitcode/xuanwu/rust). Determines platform, API/web base URLs. The DB stores the [repos.<key>] key, not this field. |
load.rs:65 |
default_branch |
string | (none) | Branch monitored for master requests and seed_master_commits. |
load.rs:66 |
benchmark_groups |
string[] | ["default"] |
Groups benchmarked when a request specifies none. | load.rs:67-68, 142-144 |
build_cmd |
string | "" |
Shell command to build the artifact (non-Rust repos). Run with the repo checked out at the request SHA. | load.rs:70 |
build_dir |
string | "" |
Directory cached as the build output (non-Rust repos). | load.rs:71 |
is_rust |
bool | false |
If true, the artifact is a Rust toolchain fetched via the toolchain machinery, not built locally. |
load.rs:73 |
post_build |
string | "" |
Shell command run after build_cmd with ARTIFACTS_DIR env pointing at the cached build output (e.g. pip install ./*.whl). |
load.rs:77-78 |
URL resolution helpers derive from repo (load.rs:93-122): GitHub → https://api.github.com/repos/<slug> / https://github.com/<slug>/pull/<pr>; GitCode → https://api.gitcode.com/api/v5/repos/<slug> / https://gitcode.com/<slug>/merge_requests/<pr>.
The default repo key is rust if present, else the first configured key (load.rs:240-252). repo_key_for_repo maps a full repo identifier back to a key (load.rs:268-275).
4. [collectors] — collector tags
| Field | Type | Default | Description | Source |
|---|---|---|---|---|
tags |
string[] | ["default"] |
List of collector names. Each must match a collector_config.name row in the DB. Drives job fan-out: the site enqueues one job per (benchmark_group, collector_tag) for every tag listed here. |
load.rs:201-214 |
Example: tags = ["Kunpeng 920B", "AMD EPYC 9654"] — every request produces, per benchmark group, one job for each of the two collectors.
5. [benchmark] and [benchmark.collectors.<tag>] — per-collector runtime overrides
The top-level [benchmark].collector table holds default RuntimeArgs applied to every collector; [benchmark.collectors."<tag>"] overrides per collector (load.rs:171-199). The merged result is serialized into each job's QueueRuntimeConfig (database/src/lib.rs:1627).
[benchmark.collectors."<tag>"] fields
| Field | Type | Description | Source |
|---|---|---|---|
iterations |
u32 (optional) | Measured iterations per benchmark (default DEFAULT_RUNTIME_ITERATIONS = 5). |
load.rs:176 |
no_isolate |
bool (optional) | If true, compile benchmark crates in-place (reuse artifacts) instead of isolated tempdirs. |
load.rs:178 |
exclude |
string[] | Benchmark-name prefixes to exclude. | load.rs:180 |
exclude_suffix |
string[] | Benchmark-name suffixes to exclude. | load.rs:182 |
include |
string[] | Benchmark-name prefixes to include (empty = all). | load.rs:184 |
exact_match |
string[] | Exact benchmark names to run (conflicts with include/exclude). | load.rs:186 |
[benchmark.collectors."<tag>".groups."<g>"]
Per-group overrides for the named benchmark group, applied on top of the collector's defaults:
| Field | Type | Description | Source |
|---|---|---|---|
iterations |
u32 (optional) | Per-group iteration override. | load.rs:155 |
no_isolate |
bool (optional) | Per-group isolation override. | load.rs:157 |
exclude / exclude_suffix / include / exact_match |
string[] | Per-group filter overrides. | load.rs:159-166 |
extra_args (alias bench_args) |
string[] | Trailing args passed after -- to the benchmark binary's run subcommand. |
load.rs:167-168 |
The [benchmark.collectors."<tag>".groups."<g>"] table is the mechanism for passing dataset scale, data folders, or bind configs to benchmarks like the Daft suite (see site-config.toml).
6. Full annotated example
# [keys.github]
# token = "" # GitHub API token
# secret = "" # X-Hub-Signature-256 webhook secret
# [keys.gitcode]
# token = "" # GitCode API token
# secret = "" # X-GitCode-Signature-256 webhook secret
[site]
url = "http://localhost:2346" # base URL used in completion comments
[repos.rust]
repo = "gitcode/xuanwu/rust"
default_branch = "main"
benchmark_groups = ["alloc", "core", "std"]
build_cmd = "echo 'profile = \"dist\"\nllvm.download-ci-llvm = false' > bootstrap.toml && ./x dist"
build_dir = "build/dist"
is_rust = true
[repos.daft]
repo = "gitcode/xuanwu/Daft"
default_branch = "0.7.5_dev"
benchmark_groups = ["daft"]
build_cmd = "export CI=1 && bash -i +x ./daft_build.sh"
build_dir = "target/wheels"
is_rust = false
post_build = "pip install ./*.whl"
[collectors]
tags = ["Kunpeng 920B", "AMD EPYC 9654"]
[benchmark.collectors."Kunpeng 920B"]
iterations = 5
no_isolate = false
[benchmark.collectors."Kunpeng 920B".groups.daft]
extra_args = ["--tpcds-scale-factor", "50", "--tpch-scale-factor", "100",
"--data-folder", "/root/data/daft", "--output-dir", "/root/data/daft_output",
"--bind-config", "default=3"]
[benchmark.collectors."AMD EPYC 9654"]
iterations = 5
no_isolate = false
7. Environment-variable fallback
When site-config.toml is absent, the config is built from env vars (load.rs:478-496). A platform's keys section is considered configured only when both its token and secret env vars are set:
| Platform | Token env | Secret env |
|---|---|---|
| GitHub | GITHUB_API_TOKEN |
GITHUB_WEBHOOK_SECRET |
| GitCode | GITCODE_API_TOKEN |
GITCODE_WEBHOOK_SECRET |
Site runtime env vars (always read, regardless of config source):
| Env | Default | Description | Source |
|---|---|---|---|
DATABASE_URL |
results.db |
DB connection URL; also the first CLI arg if given. | main.rs:24-30 |
PORT |
2346 |
HTTP listen port. | main.rs:31-34 |
QUEUE_UPDATE_INTERVAL_SECONDS |
30 |
Job-queue tick interval. | main.rs:35-38 |
SELF_PROFILE_STORAGE_S3 |
(unset) | If set, load self-profile data from S3; else local directory. | main.rs:39-49 |
8. Benchmark module parameters
Each runtime benchmark module accepts its own arguments after -- on the bench_runtime_local command line. In CI mode these are passed via extra_args in [benchmark.collectors."<tag>".groups."<g>"].
monoio
TCP ping-pong throughput benchmark. No external dependencies.
| Parameter | Type | Default | Description |
|---|---|---|---|
--monoio_cores |
u8 | 160 | Server CPU core |
--client_cores |
u8 list | auto (4 cores after server core) | Client CPU cores, space-separated |
--client_target |
string | localhost:40000 |
Client target address |
--conns_per_core |
usize | 150 | Connections per core |
--warmup_minutes |
u32 | 7 | Warmup duration in minutes |
--measure_seconds |
u32 | 10 | Measurement duration in seconds |
--output_dir |
path | ./output |
Output directory for results |
Example: -- --warmup-minutes 0 --measure-seconds 10 --conns-per-core 150
daft
SQL benchmark (TPC-H + TPC-DS). Requires Python.
| Parameter | Type | Default | Description |
|---|---|---|---|
--tpch_scale_factor |
u32 | 100 | TPC-H data scale factor |
--tpcds_scale_factor |
u32 | 1 | TPC-DS data scale factor |
--data_folder |
path | ./data |
Data folder (contains tpch-dbgen, tpcds-dbgen) |
--bind_config |
string list | (none) | NUMA bind config: name=numa_node[,num_cores]. Repeatable. |
--perf_freq |
u32 | 249 | Perf record sampling frequency (Hz) |
--output_dir |
path | ./output |
Output directory (perf data, CSV) |
--num_cpus |
u32 | (auto from bind_config) | CPU count for native runner thread pool |
Example: -- --tpch-scale-factor 100 --data-folder /root/data/daft --bind-config default=3
auron
Spark TPC-DS accelerator benchmark. Requires Python, JDK, Spark. Supports --auto-setup to auto-clone and build.
| Parameter | Type | Default | Description |
|---|---|---|---|
--auto_setup |
flag | off | Auto clone auron repo + download Spark + clone TPC-DS data |
--auron_dir |
path | (none) | Path to pre-built auron project (required if not --auto_setup) |
--spark_home |
path | (env SPARK_HOME) |
Spark installation path |
--data_location |
path | (none) | TPC-DS data path (required if not --auto_setup) |
--auron_version |
string | v7.0.0 |
Auron git tag/branch to checkout |
--spark_version |
string | spark-3.5 |
Spark version for Maven profile |
--scala_version |
string | 2.12 |
Scala version |
--spark_download_url |
string | (Spark 3.5.6 URL) | Spark download URL |
--rest_seconds |
u64 | 5 | Rest seconds between queries |
--max_retries |
u32 | 3 | Max retries per query on failure |
--benchmark_dir |
path | benchmark |
Directory for cloned repos and downloaded files |
--spark_conf |
string list | spark.auron.tokio.worker.threads.per.cpu=1 |
Extra Spark config (repeatable) |
--split_queries |
flag | off | Run each query in a separate Spark session |
Example: -- --auto-setup
firecracker
microVM boot time benchmark. Requires Docker, KVM, root. Supports --auto-setup to auto-clone firecracker repo.
| Parameter | Type | Default | Description |
|---|---|---|---|
--auto_setup |
flag | off | Auto clone firecracker repo |
--fc_dir |
path | (none) | Path to firecracker project (required if not --auto_setup) |
--fc_version |
string | "" (main) |
Firecracker git tag/branch |
--benchmark_dir |
path | benchmark |
Directory for cloned repo |
--test_filter |
string | test_boottime |
pytest test name filter |
Example: -- --auto-setup
wasmtime
WebAssembly runtime benchmark (criterion + libtest). Requires Rust nightly. Supports --auto-setup to auto-clone wasmtime repo and install wasm targets.
| Parameter | Type | Default | Description |
|---|---|---|---|
--auto_setup |
flag | off | Auto clone wasmtime repo + install wasm targets |
--wasmtime_dir |
path | (none) | Path to wasmtime project (required if not --auto_setup) |
--wasmtime_version |
string | "" (main) |
Wasmtime git tag/branch |
--bench_filter |
string | "" (all) |
Comma-separated bench names to run (e.g. compile,call) |
--benchmark_dir |
path | benchmark |
Directory for cloned repo |
Example: -- --auto-setup --bench-filter "compile,call"
tikv
Distributed KV database benchmark (criterion + libtest). Requires cmake, RocksDB. Supports --auto-setup to auto-clone tikv repo.
| Parameter | Type | Default | Description |
|---|---|---|---|
--auto_setup |
flag | off | Auto clone tikv repo |
--tikv_dir |
path | (none) | Path to tikv project (required if not --auto_setup) |
--tikv_version |
string | "" (main) |
TiKV git tag/branch |
--stages |
string | p0 |
Comma-separated benchmark stages: p0, p1, p2, p3 |
--configs |
string | native |
Comma-separated build configs: portable, native |
--benchmark_dir |
path | benchmark |
Directory for cloned repo |
Example: -- --auto-setup --stages "p0,p1" --configs native