11. Record, query, reproduce: the run registry

Every run can be found again, compared, re-run to the bit, and reused only when you ask.

Goal. Record a small parameter scan, certify the runs that pass a closed-form check, query them, compare two of them, reproduce one bit for bit, and see the cache reuse an identical computation only when cache=True is given. Expected. Heat equation \(u_t = \kappa u_{xx}\) with \(u(x,0) = \sin 3x\): the measured decay rate equals \(9\kappa\) (closed form) to better than \(10^{-9}\) relative for every run; the query returns exactly the two certified runs; reproduce() is bit-identical; with the cache off an identical call is computed again, with cache=True it takes no step and returns the recorded end state to the bit. Time: a few seconds of CPU. Script: t11_runs.py · Tests behind it: tests/runs/ (including several processes writing at once, and an unwritable registry leaving the run bit-identical). Guide: The run registry.

The problem

import os
import tempfile
import warnings

import numpy as np

import conatus as cn

# this tutorial's own registry, in a new temporary directory: nothing of yours is read or deleted
# (your own work: point CN_RUNS at a scratch disk, or give root= to cn.runs.track)
os.environ["CN_RUNS"] = tempfile.mkdtemp(prefix="conatus_t11_registry_")

The tutorial keeps its registry in a new temporary directory, so it never reads or deletes a registry of yours. Point CN_RUNS at a scratch disk for your own work: a registry holds states, and states can be large.

T_END, DT = 0.2, 1e-3
NSTEPS = round(T_END / DT)


def simulation(kappa: float):
    """Heat equation with diffusivity kappa (a model parameter: it becomes a run parameter)."""
    m = cn.Model(cn.Box(x=cn.periodic(2 * np.pi)), name="heat sin(3x)", kappa=kappa)
    m.scalar("u")
    m.equations("dt(u) = kappa*lap(u)")
    sim = m.simulate(N=32, scheme="RK443", dt=DT)
    return sim.initial(u=lambda x: np.sin(3 * x))


def amplitude(sim) -> float:
    """Coefficient of sin(3x) in u (projection on the grid, exact for this resolution)."""
    g = sim.grid("u")
    X = np.linspace(0, 2 * np.pi, g.size, endpoint=False)
    return 2 * float(np.mean(g * np.sin(3 * X)))

The heat equation is a native model on a periodic line; kappa is a model parameter. Its value is not part of the problem’s structural hash, but it is recorded as a run parameter and enters the cache key (two runs at different \(\kappa\) are never confused).

Record and certify

runs = {}
for kappa in (0.1, 0.2, 0.4):
    sim = simulation(kappa)
    with cn.runs.track(f"heat_k{kappa}", tags=["tutorial"]) as run:
        sim.solver.run(NSTEPS, record=run)             # numerics, problem hash, start/end state
        rate = -np.log(amplitude(sim)) / sim.t         # measured decay rate
        run.result(rate=rate, rel_error=abs(rate - 9 * kappa) / (9 * kappa))
    runs[kappa] = run
    print(run.explain().splitlines()[0])
for kappa in (0.2, 0.4):                               # certify with the evidence, never bare
    runs[kappa].certify("decay rate = 9 kappa (closed form)",
                        evidence={"rel_error": runs[kappa].results["rel_error"]})

• sim.solver.run(n, record=run) records into the open run: problem hash, parameter values, numerics and explain(), the executed code, host, GPU, Slurm job, and the start and end states. sim.solver is the time stepper the simulation wraps; recording and the cache are options of its run.

• run.result(...) stores scalars you can query. certify needs a reason, and here carries the evidence (the relative error against the closed form); the history of statuses is kept.

The three recorded runs, sampled every five steps: each decays as exp(−9κt) (grey), the closed form the registry’s results are checked against.

Query and compare

found = cn.runs.find(kappa__gt=0.15, status="certified", tags="tutorial", order="created")
print(found.explain())
cmp = cn.runs.compare(runs[0.2], runs[0.4])
print(cmp.explain())

kappa__gt=0.15 filters on a parameter; results, tags, statuses and index columns (host, gpu, git_head, slurm_job…) work the same way. An unknown key is an error with a suggestion, never an empty list.

Reproduce

rep = runs[0.2].reproduce(simulation(0.2).solver)      # same problem and numerics, checked
print(rep.explain())

The solver you pass must be the same problem with the same parameter values and numerics (checked, never adapted); the recorded start state is loaded and the end state compared bit for bit.

The cache, only when asked

plain = simulation(0.4).solver
again = plain.run(NSTEPS, record="heat_k0.4_again")     # identical call, cache off: computed
cached = simulation(0.4).solver
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    hit = cached.run(NSTEPS, record="heat_k0.4_cached", cache=True)
print(caught[0].message)
print({k: hit.cache_report[k] for k in ("hit", "reused_from", "computed")})
print(cn.runs.lineage(again).explain())               # the cached run is a descendant

Check

Every decay rate equals \(9\kappa\) to \(< 10^{-9}\) relative (RK443, \(\Delta t = 10^{-3}\), \(t = 0.2\)); the query returns the two certified runs in order; the reproduction and the cache hit give the recorded end state bit for bit, and the identical call without cache=True was computed again.

for kappa, run in runs.items():
    assert run.results["rel_error"] < 1e-9, (kappa, run.results)      # exp(-9 kappa t), closed form
assert [r.id for r in found] == [runs[0.2].id, runs[0.4].id]
assert rep.identical and rep.max_abs_diff == 0.0
assert again.rec["reused_from"] is None and again.cache_report is None
assert hit.cache_report["hit"] and hit.cache_report["reused_from"] in (runs[0.4].id, again.id)
assert np.array_equal(cached.X, plain.X) and cached.iteration == NSTEPS
print("OK: decay rates exact, query, bit-for-bit reproduce, cache only on request")

Output (as executed)

# t11_runs: PASS (exit 0) in 26.6 s | 2026-10-20 | script sha256 8a4e7db6a6b083fd
Run (heat_k0.1): state completed, status provisional, 0.679 s
Run (heat_k0.2): state completed, status provisional, 0.039 s
Run (heat_k0.4): state completed, status provisional, 0.0382 s
  completed certified   rate=1.8000000002184402, rel_error=1.2135564158515738e-10
  completed certified   rate=3.600000003494189, rel_error=9.706080383864446e-10
compare (a) vs (b)
  problem: same structural hash
  params kappa: 0.2 -> 0.4
  numerics problem.parameters.kappa: 0.2 -> 0.4
  result rate: a=1.8000000002184402 b=3.600000003494189 (rel. diff 5.000e-01)
  result rel_error: a=1.2135564158515738e-10 b=9.706080383864446e-10 (rel. diff 8.750e-01)
  code: identical executed tree; git e9cbfabb96 -> e9cbfabb96
reproduce -> bit-for-bit identical
  code: identical to the recorded code
  steps: 200
  compared: end state: X, multistep history (), t, iteration
  differ: nothing
conatus.runs cache hit: nothing computed; state after 200 steps read from run (took 0.037866 s); key 1a1fd1de77be, report in run.cache_report
{'hit': True, 'reused_from': '', 'computed': 'nothing: 0 steps taken'}
 (this run)
  └─ (cache reuse)
OK: decay rates exact, query, bit-for-bit reproduce, cache only on request

Going further

• @cn.runs.track("name", cache=True) records every call of a function (arguments as parameters, return value as results) and reuses identical calls on request.

• In a Slurm job the record also carries the job id and the nodes.

• The run registry describes every query key, statuses and lineage.