Benchmark Report

Comprehensive Benchmarking Across All 10 Semantic Keyword Families

This page consolidates every benchmark corpus in the repository, including macro CSV and TXT outputs, plus the Criterion cross-language matrix generated from target/criterion. Each section lists direct artifact paths, measured values, and caveats where datasets are noisy or timer-limited.

1.70x

Macro Geometric Mean Speedup

Published project-level headline versus Clang -O3.

2.364x

Criterion Geomean (Keyword)

Computed from target/criterion keyword runtime means (x vs C++).

1.549x

Criterion Geomean (Non-Keyword)

Computed from paired non-keyword variants across all 10 families.

15.665x

Best Multi-Dispatch Speedup

X-Lang multi versus C++ visitor implementation.

13,274x

Largest Memoise Speedup

x_normal/x_memo on fib(40), 50-trial benchmark set.

01. Methodology

Reproducible Measurement Setup

Criterion Harness Configuration

Sampling mode: SamplingMode::Flat
Sample size: 10
Warm-up time: 1 second
Measurement window: 8 seconds
Families: 10 keywords x 2 variants (keyword and non-keyword) x 2 languages (X-Lang and C++)

Source: crates/x-bench/benches/keyword_benchmarks.rs

Scope of Included Benchmarks

The report merges macro benchmark outputs under benchmarks/full with runtime means extracted from Criterion JSON estimates under target/criterion.

Macro datasets cover unique, invariant, layout, memoise, multi, parallel, pure, static, throws, and become.
Criterion compares keyword and non-keyword variants for both X-Lang and C++ in each family.
Every value shown below is sourced from repository artifacts listed in the appendix.

02. Macro Results

Per-Keyword Benchmark Outcomes

@layout(SoA)

macro

2.083x keyword gain within X-Lang on Criterion layout workloads

x layout non-keyword mean 9.093984s to keyword mean 4.365582s.

Source: target/criterion/keyword_nonkeyword_runtime_x_vs_cpp/layout_*/x/new/estimates.json

Caveat: Raw FDTD artifact runs in benchmarks/full/layout/fdtd_layout_results.csv contain widespread non-zero return codes, so this page treats Criterion values as the stable comparison source.

@unique

macro

10.686% faster in unique_vs_normal summary (1.1196x ratio)

normal mean 1.837912s, unique mean 1.641507s, paired p=0.1804, n=7.

Source: benchmarks/full/unique/summary_results.csv

@invariant

macro

1.494x faster at -O3 with invariants enabled

normal 0.026795s to invariant 0.017939s median_time_s.

Source: benchmarks/full/invariant/invariant_bench_results.csv

@pure

macro

1.385x speedup on pure_bench_50 and 3.137x on 3x_pure stress run

C_raw/x_pure ratios: 1.385x (pure_bench_50) and 3.137x (3x_pure).

Source: benchmarks/full/pure/pure_bench_50.txt and benchmarks/full/pure/3x_pure.txt

@memoise

macro

13,274.464x x_normal/x_memo speedup on fib(40) benchmark

x normal median 0.185843s to x memo median 0.000014s in 50-trial run.

Source: benchmarks/full/memoise/memo_50_fib(40).txt

@static

macro

Static and specialised variants collapse toward timer floor

cpp_runtime 0.031774s, cpp_specialised 0.0s, x_runtime 1.00001e-07s, x_static 1.00001e-07s.

Source: benchmarks/full/static/regex_results.csv

Caveat: Near-zero readings imply timer-resolution limits; treat this as directional evidence of successful specialisation, not precise microsecond-level ranking.

@parallel

macro

10.140x speedup from sequential to 16-thread parallel configuration

sequential median 16.330447s, parallel(16) median 1.610582s; total_cpu_mean 1344.618%.

Source: benchmarks/full/parallel/parallel_bench_results.csv

@become

macro

Tail-call variant completes depths where baseline crashes

baseline returns 3221225725 for depths 10k to 1M; tco returns 0 and completes in 0.015s to 0.037s.

Source: benchmarks/full/become/tco_bench_results.csv

@multi

macro

15.665x faster than C++ visitor and 3.186x faster than C++ manual

X-Lang Multi mean 0.00215s versus C++ Visitor 0.03368s and C++ Manual 0.00685s; all marked Significant=True.

Source: benchmarks/full/multi/statistical_test_summary.csv

no @throws (nounwind)

macro

1.090x faster than throws mode at -O3

throws median 0.010523s, nounwind median 0.009653s with identical call_count_ll=14.

Source: benchmarks/full/throws/throws_bench_results.csv

03. Criterion Matrix

Raw Runtime Means (Seconds)

Keyword	X (Keyword)	C++ (Keyword)	X (Non-Keyword)	C++ (Non-Keyword)	x/cpp Keyword	x/cpp Non-Keyword
become	0.5717s	0.5458s	0.5971s	0.5770s	0.955x	0.966x
invariant	7.320e-3s	9.695e-3s	0.0172s	6.142e-3s	1.325x	0.358x
layout	4.366s	3.969s	9.094s	6.816s	0.909x	0.749x
memoise	6.625e-3s	0.0334s	0.2411s	0.1577s	5.036x	0.654x
multi	9.712e-3s	0.0358s	7.239e-3s	0.0136s	3.687x	1.883x
parallel	1.422s	3.797s	28.65s	48.84s	2.670x	1.705x
pure	0.1830s	0.2000s	0.1885s	1.115s	1.093x	5.914x
static	0.0102s	0.0465s	0.0136s	0.0247s	4.550x	1.813x
throws	1.707s	1.157s	1.674s	1.025s	0.678x	0.612x
unique	7.797e-3s	0.2210s	9.250e-3s	0.2065s	28.341x	22.326x

04. Derived Ratios

Keyword Effect and Cross-Language Comparison

Aggregate

2.364x

Geometric mean of x/cpp runtime ratio on keyword variants across all families.

Aggregate

1.549x

Geometric mean of x/cpp runtime ratio on non-keyword variants across all families.

Keyword	X Keyword Gain	C++ Keyword Gain	x/cpp Keyword	x/cpp Non-Keyword
become	1.045x	1.057x	0.955x	0.966x
invariant	2.346x	0.634x	1.325x	0.358x
layout	2.083x	1.717x	0.909x	0.749x
memoise	36.394x	4.727x	5.036x	0.654x
multi	0.745x	0.381x	3.687x	1.883x
parallel	20.143x	12.864x	2.670x	1.705x
pure	1.030x	5.574x	1.093x	5.914x
static	1.334x	0.532x	4.550x	1.813x
throws	0.981x	0.885x	0.678x	0.612x
unique	1.186x	0.935x	28.341x	22.326x

05. Caveats

Data Quality and Known Gaps

Layout and Timer Caveats

The raw FDTD layout CSV includes multiple non-zero return codes. Static regex data includes near-zero timings that suggest clock-floor effects. These results are still surfaced for completeness, with interpretation guidance.

Artifacts: benchmarks/full/layout/fdtd_layout_results.csv, benchmarks/full/static/regex_results.csv

Harness Gap in Repository Notes

Repository memory for x-bench records a known compile gap for the multi non-keyword X source during some runs. Existing Criterion artifacts still provide complete matrix values for this page.

Note source: /memories/repo/x-bench.md

06. Sources

Artifact Index

Files Used to Produce This Report

benchmarks/full/unique/summary_results.csv
benchmarks/full/invariant/invariant_bench_results.csv
benchmarks/full/layout/fdtd_layout_results.csv
benchmarks/full/parallel/parallel_bench_results.csv
benchmarks/full/static/regex_results.csv
benchmarks/full/throws/throws_bench_results.csv
benchmarks/full/become/tco_bench_results.csv
benchmarks/full/become/become_peak_memory.csv
benchmarks/full/multi/bench_compare.csv
benchmarks/full/multi/dot_compare.csv
benchmarks/full/multi/statistical_test_summary.csv
benchmarks/full/memoise/memo_50_fib(40).txt
benchmarks/full/memoise/memo_50_10k_fib(40).txt
benchmarks/full/pure/pure_bench_50.txt
benchmarks/full/pure/3x_pure.txt
crates/x-bench/benches/keyword_benchmarks.rs
target/criterion/keyword_nonkeyword_runtime_x_vs_cpp/*/(x|cpp)/new/estimates.json