Hairspring

Find concurrency bugs in your own program, then replay them exactly.

Hairspring uses deterministic simulation. It explores many executions of a system, finds the rare interleaving that breaks a property, shrinks it to a short trace, and gives you a bundle that replays the failure.

Experimental, open source (Apache-2.0), and it runs on your laptop or your own servers.

Two ways to run it

Determinism depends on how a system is run, so Hairspring has a backend for each case and says plainly what each one needs.

Simulated and WebAssembly models

You describe the system as a state machine (Rust) or a WebAssembly module. The state is plain data, so runs are deterministic by construction.

  • Works on macOS and Linux.
  • No special hardware.
  • You test a model of your system, not the program itself.

Real Linux programs, under Hermit

Point Hairspring at an unmodified Linux program. It runs the program under a pinned fork of Hermit with a different scheduler seed for each trial.

  • Needs Linux on x86-64: bare metal, or a VM that passes hardware performance counters through.
  • Does not run on macOS or Apple silicon.
  • Hermit is a young third-party project. scripts/install-hermit.sh builds a pinned fork of it from source.
  • Experimental. Runs are roughly an order of magnitude slower than native.

Which languages have passed the acceptance test under Hermit, and on exactly what configuration, is in the support matrix. That page is the source of truth.

What you get

Replayable bundles

A failure is saved as a one-file text bundle. Replaying it is checked against the program version it was recorded with.

Shrinking

Delta-debugging reduces a failing run to a 1-minimal trace that keeps the same failure.

Same-seed replay and a determinism check

Every run is driven by a seed. hairspring doctor runs the same seed twice and reports the first divergence.

Sweeps with intervals

A sweep runs the whole trial budget and reports violations out of rated trials with a Wilson 95% interval. Timeouts are reported as inconclusive and are never counted as passes.

A confirmation ladder

A failure moves from candidate to rerun repeated, confirmed and verified as checks are recorded. Confirming needs the process backend. None of the steps says a failure is fixed.

Web UI

Runs, failures, targets and workers in a browser, served by hairspring serve, with light and dark themes.

REST and live events

The same service the CLI and UI use is available over HTTP, with server-sent events for live runs.

MCP server

An MCP server for AI agents. Text that comes from the system under test is sanitised and framed as untrusted data.

GitHub Action

A GitHub Action for simulation and WebAssembly targets, with SARIF findings and the bundles kept as an artifact. Its steps were tested in a shell, not yet on GitHub; see the CI guide.

Try it on your machine

This uses the built-in simulation targets, so it works without Hermit or special hardware. It installs the v0.1-RC2 pre-release binary for macOS on Apple silicon or Linux on x86-64, and checks its sha256 against the release’s SHA256SUMS before installing it. You do not need Rust or Node.js for this route.

1. Install

git clone https://github.com/spiculedata/hairspring && cd hairspring
bash scripts/install.sh --version v0.1-RC2
export PATH="$HOME/.local/bin:$PATH"

The installer then runs hairspring doctor and hairspring doctor --process-backend. Part of what it prints (abridged: the plan:, downloading, installed:, sha256: and note: lines, and the output of the second command, are left out):

sha256 verified: c19e94fe40e0139dbff9ebfcc00908066a89b462f01e82dbe22ad1ca53e88ccb
version: hairspring 0.1.0 (c2687b8 2026-11-05)
running: hairspring doctor
lock-lease: deterministic over 8 seeds x 200 steps
pubsub-atomic: deterministic over 8 seeds x 200 steps
pubsub-buggy: deterministic over 8 seeds x 200 steps

On a Mac the second command reports that the process backend cannot run there. That is expected, and the simulation and WebAssembly backends are not affected. The hash above is the macOS arm64 binary.

To build from source instead, for example to follow main, use bash scripts/install.sh --with-ui. That needs a stable Rust toolchain, 1.96 or newer (rustup keeps it current), with a C compiler and linker, and Node.js 20.19 or newer (or 22.12 or newer) with npm. The first build compiles a few hundred crates and can take several minutes. Without --with-ui the server shows a placeholder page instead of the web UI. Upgrading says which build you have.

2. Find a bug and shrink it

pubsub-buggy is a built-in model of an at-least-once consumer that can apply a message twice.

$ hairspring test pubsub-buggy --seed 5
hairspring: exploring pubsub-buggy (seed 5)
FAILED  pubsub-buggy (sim)
  invariant violated; shrunk from 23 to 8 steps
  steps (8):
      1. Deliver(1, 0)
      2. Tick
      3. Tick
      4. Deliver(0, 0)
      5. Step(1)
      6. Step(0)
      7. Step(1)
      8. Step(0)
  bundle:
    hairspring-bundle v2
    seed=5
    build=0.1.0
    target=pubsub-buggy
    backend=sim
    path=aa28ac6355e750e5,81d23fd7003c2305,81d23fd7003c2305,5b2a969b42d238a4,80237c8667fadf86,cf21924e7b0ff7c7,80237c8667fadf86,cf21924e7b0ff7c7
  bundle written: .hairspring/bundles/pubsub-buggy-3893b41f.hairspring  (replay with `hairspring replay .hairspring/bundles/pubsub-buggy-3893b41f.hairspring`)

The command exits 1 because a property was violated. The bundle file name includes a hash, so yours may differ from this one.

3. Replay it

$ hairspring replay .hairspring/bundles/pubsub-buggy-3893b41f.hairspring
replaying 8 steps on target `pubsub-buggy` (backend sim)
    1. Deliver(1, 0)
    2. Tick
    3. Tick
    4. Deliver(0, 0)
    5. Step(1)
    6. Step(0)
    7. Step(1)
    8. Step(0)
reproduced: the property is still violated

It exits 1 while the bug exists. Run hairspring serve and open http://127.0.0.1:7878 to see the web UI. It listens on 127.0.0.1 by default and refuses any other address unless tokens are configured.

That output came from the v0.1-RC2 macOS binary. To run your own software, see the wiki (Installation, Your first run, Running your own software) and the quickstart, which covers the Hermit steps for Linux.

The web UI

Screenshots of a build of main, against a scratch server holding runs of the built-in simulation targets. The v0.1-RC2 pre-release serves the same web UI build as main does at the time of writing. They follow your light or dark setting.

The failure page for pubsub-buggy: the shortest failing steps (8 steps), Replay and Download bundle buttons, and a Confirmation section that reads Candidate, no checks recorded.
A failure page: the shrunk 8-step trace, the bundle, and a Confirmation section. This one is a candidate because no check has repeated it; simulation targets cannot be confirmed.
The Runs page with the Start run form set to a sweep of pubsub-atomic with 5 trials, the budget the server will enforce, and the filter bar above the list of runs.
The Runs page. The start form shows the budget the server will enforce before you start a search or a sweep.
A run page for pubsub-buggy, Failed, with a panel listing what the run shows and does not show, four tabs, an empty property table and a coverage section saying every count stayed at zero.
A run page. Simulation targets declare no named properties, so the property table is empty; a process target using the guest SDK fills it.

Honest limits

  • Experimental. APIs and the bundle format will change.
  • Not a multi-machine simulator. It does not simulate several machines or a network between them.
  • Real-program runs need hardware performance counters and Linux on x86-64. macOS has no system-level determinism and Apple silicon cannot run Hermit.
  • Hermit is a third-party project we do not control. Review it before running it on anything important.
  • The host is part of the program. Under Hermit, host files and environment can act as hidden inputs; hairspring doctor helps detect this, and it is probabilistic for in-process leaks, so check several seeds.
  • A pass is not a proof. It says only that the seeds you ran did not fail. Rates are reported with intervals, and inconclusive trials are never counted as passes.
  • Languages: the support matrix says what has been tested and on what. We make no claim beyond it.
  • Security: if you run a server, read the security policy first. The remote worker protocol and the database speak plain TCP and HTTP, so use a private network or a TLS proxy.

The measurements behind these statements are in docs/determinism.md.