Design: the regression ladder in CI, PnR traces and critical-path animations¶

3.1 Principles¶

• Observational. The engine never reads a trace; no decision depends on one.

• Opt-in. PNR_TRACE_DIR unset: each hook costs one environment lookup, hoisted out of loops (a tracer object that is None); the recorder module is not even imported. No hook uses a random number generator (Python, numpy or torch), adds a torch operation, or mutates engine state; ids are counters.

• Stdlib only and 3.9-parseable, like pnr.live, so future KiCad-side hooks can use it (listed in pnr_kicad_srcs then).

• Never fails the engine. The first recorder error disables recording for the process and writes errors.json (the same stance as cost capture: diagnostics are “unavailable, never a score”).

• Bounded (§3.8), path-free (names and run-relative paths only, never absolute ones) and deterministic (no wall-clock value in any field the renderer reads; sorted keys; integer micrometres).

3.2 Layout¶

<trace>/
  run.json               written by the orchestrator (ladder runner): subject, stage order, config
  header.json            board, stack-up, components with pads, nets, total connection count
  streams/<lane>.jsonl   one writer per file; events in emission order
  blobs/<sha256>.json    geometry, content-addressed (a re-added identical route costs nothing)
  errors.json            only if the recorder disabled itself

A lane is a writer process: engine (the place-route process) and native (the runner’s native stages). A child process gets its lane from PNR_TRACE_LANE; a second writer on an existing lane file gets <lane>.<n>.jsonl (exclusive create), so no two processes ever append to one file. Halving and synthesis give each evaluation its own trace directory inside its candidate or native directory (the parent sets PNR_TRACE_DIR for the child); pnr.provenance links them (§4).

3.3 Frame and units¶

The engine frame of pnr.graph: millimetres, y up, origin at the outline’s bottom-left corner, stored as integer micrometres. Angles are degrees counter-clockwise (engine convention). Layers are indices into header.copper_layers, outer to outer (F.Cu, In1.Cu, In2.Cu, B.Cu).

3.4 header.json¶

Written once, by the first trace.begin_board(graph, constraints, rules) (called at the start of route_and_place), with exclusive create; a later call checks that its component set matches.

{
  "schema": "pnr-trace-v1",
  "recorder": 1,
  "outline": { "w": 30000, "h": 24000, "polygon": null },
  "copper_layers": ["F.Cu", "B.Cu"],
  "plane_layers": {},
  "components": [
    {
      "ref": "U1",
      "footprint": "Package_SO:SOIC-8_3.9x4.9mm_P1.27mm",
      "value": "TLC555",
      "courtyard": [7400, 5400],
      "fixed": false,
      "pads": [
        {
          "name": "1",
          "net": "GND",
          "offset": [-2475, 1905],
          "size": [1950, 600],
          "shape": "roundrect",
          "corner": 150,
          "drill": null
        }
      ]
    }
  ],
  "nets": [
    {
      "name": "GND",
      "pins": [
        ["U1", "1"],
        ["C1", "2"]
      ],
      "width": 400,
      "plane": null
    }
  ],
  "connections_total": 17
}

Pad shapes come from the graph (size, land_corner, through_hole, drill_size); the ladder runner refines them (shape name, round-rect ratio, pad angle) from source.kicad_pcb with the reader of §3.11. connections_total is the sum over nets of (pads − 1): the ratsnest edge count of the unrouted board, the same quantity KiCad’s unconnected_items counts down.

3.5 Events¶

Every event carries seq (per-stream counter), kind, scope (a path-like id such as initial-pool/start-05 or round-02/route, never a file path), phase (from a fixed list the renderer maps to display text) and, where known, progress: {done, total, source}.

The sequence of net events with op in (commit, recover) and provisional: false is the per-connection commit order; add/rip with provisional: true are the negotiation passes (overlaps allowed), and rip in the final rip-up-and-reroute pass is a real rip-up.

3.6 Blobs¶

Canonical JSON (sorted keys, no spaces), named by its SHA-256:

copper: {"tracks": [[layer, x0, y0, x1, y1, width]],
         "vias":   [[x, y, diameter, drill]],
         "zones":  [[layer, net, [outer ring, hole rings...]]]}
poses:  [[ref, x, y, rot, side]]

A net’s copper blob is complete for that net at that moment: grid segments, vias (plated pad transitions excluded exactly as route_board excludes them) and the pad escapes of the escape plan (_emit_escape into a scratch object), so what is drawn is what the router would hand to writeback.

3.7 Progress: “% routed”¶

done / total connections, with total = connections_total (§3.4) throughout:

• engine steps (source: router, or router-provisional during negotiation): per net, pads − groups, where the groups are the net’s pads joined by the route’s own edges (a union-find over the route edges, projected to pads through the escape plan’s access cells). Nets the grid router does not route (plane nets, deferred electrical nets) count as fully open. This matches the router’s remaining_connections bookkeeping.

• native steps (source: kicad): total − len(unconnected_items) from KiCad DRC on the saved board.

• placement (source: none): 0.

The bar is honest, not monotone: rip-ups make it dip. The final value is KiCad’s.

3.8 Bounds¶

• Global placement: a snapshot every ceil(iters / 24) optimizer steps and at the last step (PNR_TRACE_PLACEMENT_EVERY overrides).

• Router detail is recorded only inside a route scope opened by a caller (rounds, pool finalists, halving screens). route_board calls elsewhere (relocation probes, holdout routing) record only a route_end summary, so probe storms cannot flood a trace.

• Size: PNR_TRACE_MAX_MB (default 64). Past half of it, negotiation passes are written as one keyframe per pass instead of per-net events; past 90 %, only scopes, selections, board and result events and one truncated event (10 % is reserved for them). Expected size of a ladder case: well under 5 MB (to be measured).

3.9 Hook points (all small, guarded, at stable anchors)¶

The displayed rotation during global placement is the arg-max of the rotation probabilities, as cost capture does; positions are the optimizer’s current pos (read with detach()).

3.10 Native stages in the ladder¶

With --trace, the runner sets PNR_TRACE_DIR=<case>/trace for the place-route stage only (explicitly, like the pool flags; ambient PNR_* variables stay stripped), writes run.json, and after writeback, planes and refill it:

1. copies the board, its .kicad_pro and fp-lib-table to <case>/trace/native/<stage>/;

2. runs kicad-cli pcb drc on the copy (bounded by --timeout), except after refill, where the runner’s own final DRC report is used;

3. parses the board (§3.11) and appends a board event to the native lane.

That is two extra kicad-cli runs per case, sequential, never on the saved board itself. The final result event repeats the acceptance fields of result.json. provenance.json records trace: true.

3.11 The .kicad_pcb reader (pnr.trace_board)¶

A stdlib S-expression tokenizer that extracts: the frame (the Edge.Cuts extent, as pnr.ingest._board_frame uses the physical contour), the net table, segment, arc (tessellated), via, the filled_polygons of zones, and footprints (position, side, pads with local position, size, shape, round-rect ratio, drill). It accepts nets written as codes or as names. It needs no KiCad process, so it runs in the controller and in CI outside the image. Tests use small hand-written snippets; a manual requires-kicad test checks it against pcbnew on one ladder board (coordinates within 1 micrometre, same counts, same poses as placed.json).

3.12 The no-behaviour-change guarantee¶

• trace_noop_test runs a synthetic 5-part board through route_and_place with detailed routing, with and without the initial pool, three times: tracing unset, set, unset again, and requires byte-identical placed JSON and routes (timing fields removed).

• On the ladder (development Mac and the container), a traced and an untraced run of the same seed must give byte-identical placed.json, routes.json and pnr-report.json (minus elapsed seconds) and equal audit and DRC metrics. Board files are compared by metrics only: writeback gives new items fresh UUIDs. The measured overhead goes into the commit body (AGENTS.md: new behaviour is default-off, with a measured A/B).

4.1 Model (pnr-provenance-v1)¶

• Nodes: artifact (a pose set, a route result, a board, a DRC report), experiment (a global placement, a legalization, a detailed route, a native stage, a block evaluation) and selection (a ranking that picks one candidate). Each has a stable, path-free id (for example ladder:round-02/route, pool:start-05/place, halving:p017/deep, block:<template>/<layout-tag>), a label, a status, metrics, an order key, and either a trace scope or run-relative artifact paths.

• Edges: derive (input artifact to experiment, experiment to output artifact) and select (candidate to selection, selection to chosen).

4.2 The critical path¶

The critical path of a final artifact is its derivation ancestry, where a selection contributes only its chosen input; the other inputs of a selection are that selection’s competitors.

def critical_path(dag, final):
    order, competitors, seen = [], {}, set()

    def visit(node):
        if node in seen:
            return
        seen.add(node)
        if node.kind == "selection":
            visit(node.chosen)
            competitors[node] = [c for c in node.candidates if c is not node.chosen]
        for source in sorted(dag.derive_inputs(node), key=lambda n: n.order):
            visit(source)
        order.append(node)  # post-order: inputs before their consumer

    visit(final)
    return order, competitors

Post-order keeps each input’s whole ancestry contiguous, so independent inputs (block layouts that an assembly consumes) become consecutive sub-sequences before their consumer, in time order.

Consequences for the ladder: round k+1’s placement derives from the congestion history of rounds 1 to k, so every round up to the best round is on the path; rounds after the best round are competitors of the “best round” selection (named on the end card, not replayed). Illegal placement attempts are competitors of the round’s “first legal attempt”. With the pool, the chosen start’s global placement, legalization and finalist route are on the path (the winner’s finalist route is reused as round 1’s route); the other starts and finalists are competitors.

4.3 Adapters¶

Coarse mode animates an untraced directory from what it saved: placed.json poses, a round’s routes.json drawn net by net in file order, phases/*/diagnostic.kicad_pcb as native keyframes, evaluation.json and DRC. The overlay then says “reconstructed from saved results”. The animator only reads its sources and refuses an output path inside one (AGENTS.md: never write into experiment directories).

4.4 Storyboard (pnr-storyboard-v1)¶

critical_path becomes an ordered list of scenes; pnr.animate --storyboard FILE writes it for inspection and golden tests.

Ladder, baseline configuration:

1. Title card: case title, circuit description, parts, nets, layers.

2. Source: the generated board (parts in the generator’s naive row, partly outside the outline), full ratsnest, 0 %.

3. Round 1: failed placement attempts flash (if any); global placement (parts fly in and spread); legalization (parts snap to their slots one by one).

4. Round 1 routing: negotiation (provisional routes, dashed, overlaps allowed), commit passes (copper settles into layer colours), rip-up and reroute (red fades), recovery.

5. Per further round on the path: congestion heat overlay with the inflated parts named, the new placement, its routing.

6. Native: writeback (engine copper cross-fades to the KiCad board), planes (zone pour revealed by a wipe), refill, DRC.

7. End card: the saved board, KiCad’s verdict and the metrics.

Ladder with the initial pool replaces step 3’s first half with two montages (the 8 legal starts with their proxy scores, the shortlisted 3 lit; then the 3 routed finalists with their objectives, the winner lit), a zoom into the winner, and the winner’s own global placement and legalization replayed in full.

Halving and hierarchical runs: per assembled block, a montage of its template’s ranked layouts (chosen one lit, with its rank) and the chosen layout’s native sequence on its sub-board (at most 3 s per block; beyond 6 blocks, one combined montage); then the rung montages (place, screen, native, deep; promoted candidates lit at each rung), the winner’s generation lineage as a strip of its ancestors’ final boards, the macro placement, the assembly (block copper moves from the sub-board into the macro pose), and the winner’s native phases to its DRC verdict.

4.5 How competitors are shown¶

• A selection with at least two candidates on the path becomes a montage placed before the winner’s replay; consecutive selections (shortlist, then finalists) form one montage group in stage order.

• At most 8 tiles (4 × 2): up to 7 competitors (the best by the selection’s own criterion) plus the winner; more candidates show as a “+N more” tile.

• Each tile is the candidate’s end state at that stage (legal placement, or routed copper) with its score in the selection’s criterion; losers are dimmed to 35 %, the winner is outlined in the accent colour and labelled “chosen”. Timing: 0.4 s in, 1.0 s hold, 0.6 s zoom into the winner.

5.1 Pipeline¶

sources → provenance DAG → critical path → storyboard → timeline (frames with durations) → raster frames (Pillow) → encoders. Pure Python and Pillow (no numpy, no torch, no KiCad), so it runs on any host through Bazel. Modules: storyboard.py, timeline.py, render.py, encode.py, theme.py, __main__.py.

5.2 Visual specification¶

Colours are the viewer’s (yapnr/viewer/static/app.js in PR4), copied into theme.py with a comment naming the source; two new tokens are marked.

• Draw order: background, substrate, zones (bottom layer first), tracks (B.Cu, In2.Cu, In1.Cu, F.Cu), vias, pads, courtyards, reference labels (only when at least 7 px tall), ratsnest, highlights, outline, overlay.

• Ratsnest: per net, a minimum spanning tree between its pad groups (§3.7), nearest pad pair per group pair. Native frames draw KiCad’s open_pairs instead.

• Camera: the first scenes frame the source row and the outline together; the view eases to the outline (plus 4 % margin) as global placement starts. The y axis is flipped for the image.

• Overlay: a header with the case title (left) and “10 parts · 9 nets · 2 layers” (right); a footer with the phase text (left), the “% routed” bar with its number, and a small step counter “step 412/980” (right). The end card adds “KiCad DRC: 0 unconnected · 0 violations”, “7 vias · 120.1 mm copper · seed 0” and the engine revision. The bar is drawn lighter during provisional negotiation.

• Text is whitelisted: titles and descriptions from the ladder’s title map, reference designators, net names, counts, and phase strings from a fixed table. A validator rejects any overlay string that looks like a path (/ between word characters, ~, a drive letter) or an e-mail address; the manifest lists every overlay string, so the privacy scan sees them as text.

• Font: Pillow’s embedded default TrueType font at a fixed size (ImageFont.load_default(size), Aileron, CC0), so no system font is involved.

5.3 Timing¶

Frames carry their own durations, so holds cost one frame. WebP frames are 60 ms (16.7 fps); GIF frames 80 ms (GIF delays are in 10 ms units). Motion uses ease-in-out cubic.

Routing events are batched per frame (k = ceil(events / frames)); a committed net flashes for two frames, a ripped net fades over three. Expected lengths: about 8 to 10 s for the small cases and 18 to 22 s for the chasers; --max-seconds (default 22) scales scenes down proportionally. Rigid bodies (line groups, block macros), the hierarchical lift and the holds of a side-by-side comparison are specified in the constraint and hierarchy design, section 6.1, which lists everything a frame may interpolate.

5.4 Outputs and budgets¶

Frames are drawn at twice the size and reduced with a Lanczos filter (Pillow draws without anti-aliasing). If an output exceeds its budget, the encoder steps down deterministically: WebP quality 80, 70, 60, then a longer frame interval, then width 720, 640; GIF 128, 96, 64 colours, then 100 ms frames, then width 560. The chosen settings are recorded in the manifest; an output still over budget is an error. ffmpeg runs with a deadline (pnr.proc).

5.5 Determinism¶

The output bytes are a function of the trace content, the options and the Pillow version: no time, no environment, no path, iteration always in sorted order, a fixed palette and font. Tests render twice, and from a copy of the trace at another path, and compare hashes. Across platforms the output is expected, not promised, to be identical (the Pillow wheels bundle their own FreeType, zlib and libwebp); the manifest records the Pillow version. The engine results themselves are reproducible per platform only (docs/decisions.md), which is why the manifest records the platform and image the ladder ran on.

5.6 Performance¶

A static layer (substrate, outline, courtyards of fixed parts) is cached; the copper layer is redrawn only when copper changed; placement frames redraw parts only. Target: all 8 ladder cases rendered in a few minutes on one core (to be measured).

7.1 Workflow ladder.yaml¶

• Triggers: every pull request (a plan step skips the ladder unless engine inputs changed: hardware/pnr/, hardware/tools/scan_via_proximity.py, requirements*, docker/, the workflow), a nightly schedule on main, and workflow_dispatch (inputs: cases, seeds, animate). A final ladder job always reports, as image.yaml does, so the check can become required later without a path filter leaving it pending.

• Permissions: contents: read, packages: read. Concurrency per PR, cancelling superseded PR runs.

• Runner: ubuntu-24.04-arm (the image’s arm64 build; the same architecture as the test job).

7.2 The image and the runtime guard¶

1. docker/login-action (pinned by commit, as in image.yaml) with GITHUB_TOKEN. While the package is private, the repository needs read access to it (packages pushed by this repository’s workflows are linked to it; otherwise Package settings, Manage Actions access). Once the owner makes the packages public, the login stays harmless. Fork PRs skip the ladder until then.

2. Resolve ghcr.io/studio-fug/yapnr:edge to a digest and use the digest (recorded in the step summary and the manifest).

3. Runtime guard. edge is built from main; a PR can change the runtime pins. A step compares the image’s /opt/venv distributions with the checkout’s docker/yapnr/runtime-arm64.lock. If they differ, it creates an overlay venv inside the container from the image’s own CPython (/opt/python) and installs the checkout’s lock with pip --require-hashes --no-deps: the same packages the PR’s image would get, in a minute or two, without building an image. This is the simplest reliable option; building the app image in the job (Bazel wheel plus docker buildx) was rejected as slow for no gain, since the ladder runs the checkout’s engine sources, not the image’s wheel.

4. If docker/yapnr-kicad/ changed (a new KiCad base that is not published yet), the ladder is skipped with a notice: image.yaml smoke-tests the new base on the PR, and the nightly run on main covers the ladder after the merge.

7.3 Running the ladder in the container¶

timeout --kill-after=60 50m docker run --rm --init \
  -u "$(id -u):$(id -g)" -v "$PWD":/work -w /work --shm-size 1g \
  --entrypoint /usr/local/bin/yapnr-kicad-env "$IMAGE" \
  "$PY" hardware/pnr/regression/run.py --repo /work --out /work/out/ladder \
    --python "$PY" --kicad-python /usr/bin/python3 --kicad-cli /usr/bin/kicad-cli \
    --library /usr/share/kicad/footprints --timeout 600 $CASES $SEEDS

$PY is /opt/venv/bin/python or the overlay venv’s. yapnr-kicad-env gives the runner’s UID a writable home with KiCad’s library tables. The KiCad-side stages run under the image’s KiCad Python (3.12) from the frozen sources on PYTHONPATH, as the runner already arranges.

7.4 What runs when¶

The PR subset is revisited after the first CI timings (all 8 cases at seed 0 took about 3 min on the Mac, so the full seed-0 ladder may fit a PR too).

7.5 Reporting and artifacts¶

• A stdlib script (tools/ci/ladder_summary.py) turns summary.json into the step summary: case, seed, pass, opens, violations, vias, copper, seconds, and the failure reasons. junit.xml is uploaded as is (no third-party reporter action needed).

• if: always() uploads ladder-results (summary, junit, per-case result.json, drc.json, stage logs; boards of failed cases) for 14 days. Case directories in these files are run-relative (§2).

• Timeouts at three levels: run.py --timeout per stage, timeout around docker run (so the upload still runs after a hang), and the job’s timeout-minutes.

• Animations (nightly and animate: true): a second job on ubuntu-24.04-arm sets up Bazel with the same caches as ci.yaml, downloads the traced run, runs bazel run //hardware/pnr:ladder_animations -- --render-only ..., uploads ladder-animations, and compares the trace hashes with docs/animations/manifest.json; a drift is a notice, never a failure. Refreshing the committed animations stays a deliberate change (§8.4).

• The lane is informational at first; after about a week of green nightly runs the owner can make ladder a required check.

8.1 Files¶

docs/animations/<case>.webp     one per ladder case (8)
docs/animations/07-chaser-20.gif README sizzle
docs/animations/manifest.json   provenance, hashes, sizes, settings, overlay strings
docs/regression-ladder.md       the "Regression ladder" page

The folder is docs/animations/, not docs/static/animations/: docs/build_docs.py stages only Markdown from docs/, and its GitHub Pages step rewrites every static reference in the HTML, which would break raw <img> paths. build_docs.py gets one more step: copy docs/animations/ to _extra/docs/animations/, which html_extra_path places at docs/animations/ in the site. Then the same relative paths work on GitHub and on the site, for README.md (staged at the site root) and for docs/regression-ladder.md. Both use raw HTML <img> inside a <p> or <figure> (not a standalone <img>, which MyST’s html_image would turn into a Sphinx image node).

8.2 The page and the README¶

• docs/regression-ladder.md: what the ladder tests and its gate (from the regression README), how to run it (Mac with the headless KiCad copy, container, Bazel), the CI lanes, how to read an animation (colours, bar, montages), then one section per case: its animation, a caption from the manifest (parts, nets, layers, vias, copper, KiCad result, seed, configuration, engine revision, platform). Linked from docs/index.md (“Start here”) and the “Project” toctree, and from hardware/pnr/regression/README.md.

• README.md, right after the first paragraph under the title:

<p align="center">
  <img src="docs/animations/07-chaser-20.gif" width="640"
    alt="yapnr placing and routing a TLC555 + CD4017B LED chaser, from an unplaced board
    to a KiCad-DRC-clean result">
</p>
<p align="center"><sub>TLC555 + CD4017B five-stage LED chaser (20 parts), from the
  <a href="docs/regression-ladder.md">regression ladder</a>: initial placement pool,
  placement, routing and KiCad DRC.</sub></p>

8.3 Size budget instead of the large-file hook¶

• .pre-commit-config.yaml: check-added-large-files gets exclude: ^(third_party/|docs/animations/).

• tests/unit/repo/test_animations.py (a repository check, stdlib only, tagged like the others) enforces: every file in docs/animations/ is in the manifest with a matching SHA-256 and size; extensions are .webp, .gif and .json only; WebP at most 2.5 MB, GIF at most 5 MB, folder total at most 20 MB; widths between 480 and 960 px (read from the file headers); no EXIF, XMP or GIF comment chunks; every ladder case (from designs.py) has an animation; the README and the page reference only files that exist.

Manifest shape:

{
  "schema": "yapnr-animations-v1",
  "generated": {
    "date": "2026-10-01",
    "engine_revision": "<commit>",
    "renderer": 1,
    "pillow": "<version>",
    "platform": "linux-aarch64",
    "image": "ghcr.io/studio-fug/yapnr@sha256:<digest>",
    "kicad": "10.0.6"
  },
  "animations": [
    {
      "case": "05-timer-led-10",
      "title": "TLC555 blinker",
      "seed": 0,
      "config": ["--initial-pool", "--initial-starts", "8", "--initial-finalists", "3"],
      "file": "05-timer-led-10.webp",
      "bytes": 812345,
      "sha256": "<hash>",
      "width": 800,
      "frames": 240,
      "seconds": 16.2,
      "trace_sha256": "<hash>",
      "result": { "passed": true, "opens": 0, "violations": 0, "vias": 7, "copper_mm": 120.1 },
      "settings": { "quality": 80, "frame_ms": 60 },
      "captions": ["TLC555 blinker", "10 parts · 9 nets · 2 layers", "Global placement"]
    }
  ],
  "readme": { "case": "07-chaser-20", "file": "07-chaser-20.gif" }
}

8.4 Where the committed animations come from, and how often they change¶

• Preferred: the ladder in the published arm64 image (locally with colima, or the nightly artifact), so CI can reproduce them; the manifest records the image digest and platform. Fallback: the development Mac (headless KiCad copy, the experiment runtime), recorded as darwin-arm64.

• Only boards that pass KiCad’s gate are committed; the renderer refuses to write into docs/animations/ for a failed case unless --allow-failed, and then the end card says FAIL.

• Each refresh adds about 10 to 16 MB to the history. Refresh deliberately (a notable engine change, or a release), never on every PR; the total budget bounds each refresh. Git LFS and Pages-only hosting were considered and rejected for now: the request is to commit them, LFS complicates the Pages checkout and the README.