Benchmarks

Every number on this page is reproducible from zap-proto/bench. If a claim isn't traceable to a benchmark file checked into that repo, it doesn't live here. We've burned trust with fabricated tables before; this page exists as an antidote.

What's measured today

The current harness (bench_test.go) measures HTTP/1.1+JSON vs ZAP-binary across four workload sizes: tiny (16 B body, 4 headers), small (256 B, 8 headers), medium (4 KiB, 12 headers), large (64 KiB, 8 headers). Source: bench_test.go. Raw output: bench-results.txt.

Single-client throughput

5000 sequential requests, single TCP connection, keep-alive.

Concurrent throughput (32 in-flight)

32 concurrent workers × 1000 requests each.

Wire bytes

How big is the request on the wire?

What the numbers say

The honest read:

• At small payloads ZAP is mildly faster and uses ~10% fewer wire bytes than HTTP+JSON. Wins are modest.
• At medium payloads it's a tie. Both are dominated by socket and OS overhead.
• At large payloads (64 KiB) ZAP loses by 2× — the current ZAP-HTTP path has higher allocations and an extra copy in the hot loop. Optimization work is queued; results will land here when re-measured.
• Under concurrency 32, ZAP loses 10% uniformly. The hot path doesn't yet take advantage of arena pooling that the encoder is capable of.

The hand-wavy "10× faster than gRPC" / "1438× faster consensus" tables that lived here before were not measured. They were aspirational claims dressed as data. Removed.

Where ZAP actually wins today

Performance is not the strongest argument for ZAP in 2026. The stronger arguments — supported by mechanism, not by µs — are:

1. One wire format, many protocols. HTTP, MCP, A2A, FIX all ride the same framing. One observability story, one auth story, one cipher suite to audit.
2. PQ at the transport. IETF X-Wing is the default. Browsers got X25519MLKEM768 in 2025; ZAP gives you the same KEM directly inside your application protocol, not just terminated at the edge.
3. Capabilities, not tokens. The capability mailbox example shows what's structurally impossible to express in HTTP: a reference that the holder can call but cannot forge, transferable to a third peer, revocable without rotating keys.
4. PQ-RNS. Name resolution that doesn't trust a CA. The KEM key in the resolved Record is the key the client dials — no separate cert chain.

These are the levers. When the harness produces wins on µs/req, they'll land here, sourced from the run.

Run it yourself

Go harness (HTTP-shaped workloads)

git clone https://github.com/zap-proto/bench
cd bench
go test -v -run Test 2>&1 | tee my-bench-results.txt

Tweak the workloads in bench_test.go. Hardware will move the numbers; the ratios should be reproducible across machines.

Rust harness (carsales / catrank / eval)

The canonical workload suite — three workloads (carsales, catrank, eval), three modes (object / bytes / pipe), two compressions (none / packed), two scratch policies (reuse / no-reuse). 20 distinct measurements per run.

The benchmark uses only the zap crate (the renamed runtime) and zapc crate (the renamed codegen). Schema files are .zap; pre-compiled *_zap.rs is committed in-tree so cargo build -p benchmark invokes zero external schema tools. cargo tree -p benchmark shows no capnp dependency anywhere.

git clone https://github.com/zap-proto/rust
cd rust
bash ./benchmark/bench.sh quick     # ~10 sec sanity
bash ./benchmark/bench.sh default   # original iteration counts, ~1 min
bash ./benchmark/bench.sh full      # 10× iterations, ~10 min

Each run writes one JSON file to bench-results/$(hostname)-$(timestamp).json. The schema is documented in zap-proto/rust/benchmark/README.md:

{
  "schema_version": 1,
  "host": {"hostname":"...","os":"...","arch":"..."},
  "toolchain": {"rustc":"..."},
  "iters": {"carsales":10000,"catrank":1000,"eval":200000},
  "runs": [
    {"case":"carsales","mode":"object","compression":"none",
     "scratch":"reuse","iters":10000,"elapsed_secs":0.42,
     "throughput_ops_per_sec":23809.5},
    ...
  ]
}

Twenty runs per file (carsales × 2 scratch × 5 modes/compressions + catrank × 5 + eval × 5).

Contribute results across hardware

The whole point of producing structured JSON is so we can collect numbers across hardware classes — server CPUs vs laptops vs embedded vs cloud VMs. Open a PR against zap-proto/rust dropping your bench-results/*.json into the bench-results/ directory. The schema is stable; future aggregation will pull from whatever's in there.

A sample run from this dev machine (Apple M-series, macOS aarch64, quick preset, eval case only):

Object-mode wins because there's no encode/decode step — the in-memory layout is the wire layout. Pipe is slowest because it crosses a process boundary. Packed adds compute for compression in exchange for fewer bytes on the wire. The shape of these numbers should hold across machines; absolute throughput will swing with CPU and cache size.

Parallel saturation

benchmark/saturate.sh runs N worker copies of the encoder workload in parallel and computes real parallel efficiency against a recorded single-worker baseline. Honest measurement, not aggregate-divided-by-per-core (which is tautological).

# 1. record the single-worker baseline
PRESET=full bash ./benchmark/saturate.sh baseline

# 2. saturate with N=ncpu workers (default), get real speedup
PRESET=full bash ./benchmark/saturate.sh

# 3. eyeball CPU usage during the run
tail bench-results/saturate-*/cpu.log

Sample numbers on the dev machine (Apple M1 Max, 10 cores: 8 P + 2 E):

single-worker      : 6.003s for 2,000,000 iters = 333,183 ops/s
10-worker wall     : 11.46s
aggregate ops/s    : 1,745,201
REAL speedup       : 5.24× (= single × N / wall)
effective cores    : 5.24 of 10
parallel efficiency: 52%

52% efficiency on a 10-core M1 Max is normal: macOS QoS scheduler defaults subprocesses to efficiency cores, the system allocator serializes across processes, and unified memory bandwidth is shared. The CPU sampler captures the actual peak (~557% in ps -o %cpu, ≈5.5 cores busy), which matches the speedup math.

If you submit a bench-results/ JSON from a different machine, the matching saturate-*/cpu.log is welcome too — it's the proof-of-life that the workers actually ran in parallel.

Methodology

• Single host, loopback (127.0.0.1)
• Both HTTP and ZAP use bufio readers/writers sized 64 KiB
• TCP NoDelay = true on both
• Persistent keep-alive connections (no per-request TCP establishment)
• 5000-request warm window, allocs/GC measured via runtime/metrics
• Both servers are minimal echo handlers — what you're measuring is framing overhead, not application logic

This is not a comprehensive perf suite. It's a single, honest harness in-tree. Native-ZAP-vs-gRPC, MCP-bridge-vs-stdio, FIX-vs-FIX-over-TCP — each needs its own harness, each is roadmap work, each will be added to this page when the harness file exists in zap-proto/bench.