Ortova — Architecture

The intelligence layer for the built world. One spine holds every property's complete truth — permit history, ownership, valuation, physical change — joined on the matricule (Québec's universal parcel ID). An engine + an AI fleet compose intelligence products from that spine.

This is the single container-level map: what runs, what it talks to, and the invariants that hold across those boundaries. Component-level shape lives in each unit's README.md and in the named Zod schemas (src/schemas.ts) — jump-to-definition terminates there, not here.

• What shape is this data? → the named schema in a schemas.ts (colocated-docs Layer 1; docs/architecture/2026-07-22-boundary-schema-table.md).
• What does a unit do / how do I run it? → that unit's README.md.
• How does it connect / where does a fix go? → this file.
• Why is it built this way? → ## Decisions below; git log --follow for the timeline.

Container map

flowchart TD
    subgraph external["External feeds"]
        FEEDS["ArcGIS / Socrata / CKAN / ODS / CARTO / STAC / bulk-geo"]
        RF["Registre foncier (browser)"]
        PDFS["Municipal PDFs (minutes, permits)"]
        LIDARSRC["QC LiDAR / imagery tiles"]
    end

    subgraph runners["ortova-mac self-hosted runner fleet"]
        CRON["launchd watch-heartbeat cron -> watch-drive.yml"]
        WORKERS["apps/worker/* sidecars: geo-ingester, geo-trinity, permit-ingester,
minutes-harvester, permit-doc-parser, governance-parser,
lidar-ingester, tile-builder"]
    end

    subgraph queue["Watch executor"]
        BULLMQ["BullMQ / Redis queue (WatchJobSchema)"]
    end

    subgraph engine["Engine (in-process library)"]
        PLATFORM["packages/platform (@ortova/platform)
six define* factories + four fixed dispatchers"]
        INNGEST["Inngest dispatchers:
source -> canonical-upsert -> signal -> product"]
    end

    subgraph data["Data plane"]
        SUPA["Supabase Postgres
raw / canonical / serving / api / aerial / ops"]
        R2["Cloudflare R2
PMTiles / GeoParquet / COPC / COG objects"]
    end

    subgraph surfaces["Serving surfaces (Vercel)"]
        DEV["apps/dev (@ortova/dev)
operator scenes + app/api/* serving edges"]
        LANDING["apps/landing (@ortova/landing)
ortova.io marketing"]
    end

    subgraph sidecars["HTTP sidecars"]
        LIBPOSTAL["libpostal-sidecar (pydantic)"]
        RFSTAGE["rf-stagehand (Zod)"]
    end

    subgraph taps["External read taps"]
        QCTAP["apps/worker/qc-mcp-tap (Cloudflare Worker)
remote MCP server, read-only"]
    end

    CLIENT["Outside collaborator's AI client"]

    FEEDS --> WORKERS
    PDFS --> WORKERS
    LIDARSRC --> WORKERS
    RF --> RFSTAGE
    CRON --> BULLMQ
    BULLMQ --> WORKERS
    WORKERS --> SUPA
    WORKERS --> R2
    PLATFORM --> INNGEST
    INNGEST --> SUPA
    WORKERS -. use .-> PLATFORM
    DEV --> SUPA
    DEV --> R2
    DEV --> LIBPOSTAL
    DEV --> RFSTAGE
    RFSTAGE --> SUPA
    CLIENT -->|MCP over HTTPS| QCTAP
    QCTAP -->|api.* only, api_reader envelope| SUPA

Containers

Invariants (hold across every boundary; survive all refactors)

1. Every property fact lives in a typed canonical.property_* satellite that owns exactly one fact and carries its own provenance (source_key + external_id + raw_row_hash + SCD-2). canonical.property is identity-only.
2. Identity is normalized once, at ingest. matricule_id is the province-wide join key; derive over store (generated columns / views), never re-store a derivable value.
3. Drawers evolve additively. A canonical satellite in production is never renamed, rebuilt, or superseded — new source classes generalize into the existing drawer so context compounds.
4. Every hand-shaped boundary is a named Zod schema in a schemas.ts, with types derived via z.infer. Verbatim api.* / canonical rows stay to_jsonb passthrough (DB owns that shape).
5. The api.* read-contract is PII-free by construction — SECURITY DEFINER functions, and api_reader holds zero grant on canonical.*.
6. Postgres schemas are restricted to {raw, canonical, serving, aerial, ops, api} — anything else build-fails.
7. All engine data flows through the six define* factories and the four fixed dispatchers. Bypassing a factory throws at module load.
8. Serving is open-format end-to-end. MapLibre + deck.gl project COPC / COG / PMTiles as range-readable R2 objects; compute and UI meet only through those objects plus DB facts.
9. Batch compute runs headless on the ortova-mac fleet / runners, writing objects + facts; the browser only projects them. The machine compounds even with the UI down.
10. DB writes are Felix-gated through scripts/apply-migration.ts --i-am-felix; reads are free.
11. Every data source carries an attribution line everywhere it is shown or delivered.
12. Every served response states its own COMPLETENESS and its own AS-OF. On every published surface — the /api/v0 contract, the change feed, the MCP tap — a success carries a positive completeness assertion (meta.complete), any caveat travels as a list (meta.warnings[], never a growing set of booleans), and the response carries meta.as_of: the minimum freshness stamp across the relations that answered it, because a response is only as fresh as its stalest input. A failure is a classified non-200 carrying a reason and a retryability flag, never an empty success. Rationale: an empty result and a failed query render oppositely — one is an answer to show a person, the other must suppress the number — and a valuation shown without its own as-of reads as a current market price, which it is not. Prior art: GraphQL errors[] beside a partial data, Prometheus/Thanos warnings[], Elasticsearch _shards + timed_out, OGC API Features numberMatched/numberReturned; RFC 9111 §5.5 retired the HTTP Warning header, so this belongs in the body. Built 2026-07-30 in apps/dev/src/lib/api-v0/handler.ts: meta.complete, meta.warnings[], and now meta.as_of + meta.request_id (migration 0478, DRAFT). as_of is null — with a warning naming the relation — whenever no source relation carries a refresh stamp, never now(); the route→relation map lives in api.route_as_of and is held to the live function bodies by validate:arch's api-route-asof, because an undeclared join makes a response silently OVERSTATE its freshness.

Module boundaries

• packages/platform exposes the define* contract that apps/core relies on — change the exposed surface deliberately and note it in the commit.
• apps/dev/src/schemas.ts is the only home for hand-Zod'd serving-edge contracts (curated column subsets, GeoJSON envelopes); it never mirrors a canonical table.
• scripts/ops/watch/executor/schemas.ts owns WatchJobSchema — the single cross-worker queue message — and the per-workflow arg bags (WorkflowArgsSchemas).
• Worker input boundaries are stringly-typed env/flag bags; their named schemas live in the executor schema file, not in each worker.

Decisions

Irreversible, architecture-shaping decisions, newest first. Append new entries here (context / decision / consequences, 3–6 lines); split to standalone ADR files only past ten entries. Full archive: decisions/decisions.md; current truth: decisions/canon-now.md.

2026-07-31 — The municipal document corpus lives in R2, and its address is derived, never parsed

• Context. apps/worker/minutes-harvester/storage.ts banked every document to Vercel Blob, and its header argued the choice on a stated estimate: the corpus "even at full 35-municipality/ full-history scale, is a few GB … nowhere near the S3-vs-Blob pivot point (province-scale, hundreds of GB)". Measured against raw.qc_minutes_documents on 2026-07-31 it is 191,262 PDFs / 179 GB plus 225,776 cached-text siblings — 417,038 objects, three orders of magnitude past the estimate and past the very pivot the header named as the reason to move. The estimate was never wrong when written; it was never re-measured, and the corpus outgrew it silently because nothing in the system compares a stated capacity assumption against the live number. Two further facts decided the shape. Blob is egress-billed and R2 is not, on a corpus the OCR/redate/ metadata repair passes re-read whole. And the timing was the last cheap moment: today's pull roughly doubles the corpus, so every document it banks to Blob would be a second document to move.
• Decision. R2 is the store of record for the corpus — the house object store already (invariant 8), reached through the same four env vars (R2_ENDPOINT / R2_ACCESS_KEY_ID / R2_SECRET_ACCESS_KEY / R2_BUCKET) lidar-ingester, geo-trinity, apps/dev, and this worker's own cloud/r2-heartbeat.ts already read. Three choices carry the weight. (1) The key does not change. documentKey() still yields qc-minutes/<municipality_code>/<sha256>.<ext>, so the Blob key is the R2 key — verified 191,262/191,262 and 225,776/225,776 against the live table. That single decision makes the backfill a pure copy with nothing to remap, makes the DB rewrite invertible, and makes a dual-read exact rather than a guess: readObjectBytes() accepts either URI shape and falls back to the other store at the same key, in both directions, so the object copy and the URI rewrite — two operations that cannot be atomic with each other — may happen in either order. *(Corrected 2026-07-31, in production: that last clause was written as if it covered the whole design and covered only READS. The migrator's WORK LIST keyed on URI shape, so applying 0486 first made it match nothing — run 30637592942 exited success on settled 0 / 0 while all 179 GB sat in the store we were about to retire. Fixed by deriving the work list from durable facts (sentinel-or-not, has-text-sibling, and the key from (municipality_code, content_hash)) and pinning it with a red-before test. The generalization is the entry's own last line arriving a day early: an order-independence claim is a capacity assumption in disguise — it must name WHICH operations it covers, and each one needs its own test.)* (2) The stored address is r2://<bucket>/<key>, the shape canonical.lidar_tile.our_store_uri already uses, and deliberately not a public HTTPS URL: R2 buckets are private and staying private is the point — 179 GB of municipal documents on an anonymously-fetchable host is the exact class the 2026-07-30 exposure decision exists to prevent. Readers resolve bytes through the seam, never by handing the URI to fetch(). (3) The store is chosen explicitly (MINUTES_OBJECT_STORE=r2|blob, default r2) and logged by the run, never inferred from whichever credential happens to be on the box.
• Consequences. A stored object address is now derived, never parsed: every URI on both sides is a total function of (municipality_code, content_hash), which is what lets migration 0486 (DRAFT) re-prove losslessness at apply time and abort rather than rewrite a value it could not reconstruct. The corpus lands under ortova-geo/qc-minutes/ and not a dedicated bucket because the live R2 token is bucket-scoped (ListBuckets → AccessDenied) — a corpus bucket needs Felix to create it and mint a token, so MINUTES_R2_BUCKET exists to make that later move a one-value change rather than a key rewrite. Blob is retired for this corpus only when the copier reports mismatch = 0, source-missing = 0, errors = 0, settled = corpus total — a positive assertion, not the absence of complaints, and after 30637592942 the copier now refuses to report a settled corpus it has not sampled in R2: "nothing to do" is a claim that has to be earned, because a batch job that can succeed without looking at anything is worse than one that crashes. Generalizes: a capacity assumption written into a design comment is a claim with an expiry date, and the 81,062 index-only:// sentinel rows (bytes never fetched) stay untouched, because inventing an address for absent bytes is the same confident-label-for-an-absence error the comp-class decision named the day before.

2026-07-30 — Anonymous exposure is DECLARED in one registry, never emergent from a URL's shape

• Context. apps/dev/src/middleware.ts gated the whole app on a config.matcher chain of negative lookaheads containing .*\..* — "any path with a dot skips auth". It is the stock Next.js docs recipe standing in for "don't run middleware on /public files", but a matcher cannot tell a static file from a route handler, so what it encoded was the unratified policy file-looking = world-readable. It fired twice. (1) 2a2298d0 walled /ui/* mockups carrying real sample spine data; the bruise closed, the hole stayed. (2) On 2026-07-30 the verdun-twin pyramid moved to private R2 behind /api/atlas/verdun-twin/[...path], whose every asset path is dotted — so in ONE deploy it went from absent-in-prod to anonymously readable, while the undotted sibling /api/atlas/ortho-tile/[z]/[x]/[y] stayed walled. Measured on prod before the fix: _vector/buildings.geojson 200 / 5.2 MB, plate-fullres.webp 200 / 7.06 MB, manifest.json 200 / 10 KB; and the siblings the class predicted — /data/verdun-permits.geojson 200 / 1.63 MB of permit data, /verdun-imagery-2023-2026/partner-2026-04-25.jpg 200 / 867 KB of licensed WV-2 imagery, /briefing/atlas-briefing-for-marc.pdf 200 / 328 KB.
• Decision. Deny-first, one expression, one writer (canon P4.c). apps/dev/src/lib/exposure-registry.ts is the ONE declaration of what an anonymous caller may read: every route under apps/dev/src/app/api/ carries an entry classed walled / public-with-own-auth / public-static / public-pre-auth, with a required reason and — for public-with-own-auth — a required ownAuth naming the credential the handler itself checks and its fail-closed behaviour. The matcher catches everything ('/((?!_next/static|_next/image).*)'; those two are framework build output the login page itself needs) and the allowlist is applied inside the middleware, because config.matcher must be a static literal and a regex copy of the registry would be the second writer this removes. The twin proxy is walled per Felix's ruling; the logged-in scene reaches it same-origin on the session cookie.
• Consequences. A new API route cannot ship undeclared: validate:arch's route-exposure fails the build (RULE 1), rejects dead entries (2), demands a reason and a named credential (3), and forbids policy in the matcher — the dot-arm can never return (4). Proven red-before on the pre-fix file and on the twin route left undeclared. Live behaviour is verified rather than assumed: pnpm probe:exposure fires anonymous GETs and compares them to the registry, wired as step [3/3] of scripts/ops/release-deploy.sh. Cost, stated: the middleware now also runs on public/ assets, so a request with no Supabase cookie is redirected without a getUser() round-trip — no cookie can be a session, so the call could only confirm the redirect.

2026-07-30 — A published read-contract bounds what ONE CALL may COST before it bounds how many

• Context. api.nearest_comps accepted radius_m to 25 000 with lim to 200 on a Bearer-walled but publicly-reachable path about to sit behind a paid advertisement, and no rate limiting existed anywhere in apps/dev/src. Measured server-side 2026-07-30: that maximum call costs 12 986 ms of database time against 32 ms for the 700 m / lim 50 call the surface actually makes — on the pooler the harvest fleet shares. Two further measurements moved the design. Radius, not lim, is the cost driver (505 / 506 / 512 ms at lim 50 / 200 / 500 on one radius), so a cap list leaning on lim would look prudent and protect nothing. And the same parameters cost ~3x more in dense urban than suburban (2 000 m: 505 ms on the Montréal Plateau, 166 ms in Châteauguay), which corrects the work spec's own 5 000 m recommendation — 5 000 m on the Plateau is 1 915 ms, worse than the 1.3-1.6 s that raised the item.
• Decision. Migration 0477 (DRAFT, supersedes the applied 0476). A COST CEILING comes first and a COUNT LIMIT second, because a request-count limit on an unbounded unit of work bounds nothing: with the ceilings one key's worst case is 120 x 336 ms ~ 40 s of database time per minute; without them the same 120 calls would draw ~26 minutes per minute. Two ceilings, answering two different questions — the FUNCTION's (5 000 m / lim 200) is the physical bound on every caller, enforced at the seam api_reader cannot go around, and the ROUTE's (1 500 m / lim 100) is the v0 consumer contract, read off the dense curve where it crosses ~300 ms. Both reject with a named ceiling; neither clamps, because a silent clamp is a default that changes the answer without saying so. The limit itself is a per-key COLUMN (app.api_key.rate_limit_per_min / _burst, default 120/40) spent from a continuously-refilling token bucket in one Postgres row — not process memory, which on Fluid Compute would enforce (limit x instances) and weaken as traffic grew, and not a new infra service.
• Consequences. No api.* function may publish a parameter range it has not measured the cost of. LANGUAGE sql keeps a raising guard in its MATERIALIZED origin CTE rather than converting to plpgsql — proven read-only to fire before the scan (23 ms vs 11 290 ms) and so leaving the measured plan untouched. The limiter fails open, loudly: a limiter is an availability protection, and failing closed converts a limiter fault into an outage of the surface it protects — the opposite of auth.ts, which answers a different question and fails closed.

2026-07-30 — A committed data file is a PROJECTION of the database, never a second answer

• Context. apps/worker/minutes-harvester/target-registry.json is committed so git archive HEAD can ship the target map to a box, and the resolver reads it FIRST (a DB touch costs 1,667 ms per child on a process-per-muni fleet). It was WRITTEN by --reconcile, which re-derives every field from the seven legacy stores — so it was an eighth store with its own opinion. Measured: the file said 338 rows were confidence='verified', the table said 76. Both derivations were internally correct; having two was the defect the 2026-07-29 ONE-registry decision exists to end.
• Decision. The DB is authoritative and the file is a deploy artifact: regenerated by discover-targets.ts --project-from-db from raw.qc_minutes_target_registry (the map) + serving.qc_target_state (0473's ledger-derived state), carrying a projection header (source / from[] / row_count / harvest_eligible / regenerate_with). It derives nothing. Corollary, and it inverts an earlier rule: a derived verdict (harvest_eligible) is now KEPT in the file rather than stripped — a projection has no independent derivation to go stale against, and the fleet reads the file, so stripping the verdict just moved the derivation onto every box.
• Consequences. Three mechanisms, not discipline: the header marker; a free load-time audit that shouts on stderr when the file is unmarked, reconciled, self-inconsistent, or older than 7 days (MINUTES_REGISTRY_MAX_AGE_DAYS); and a CI ratchet asserting the committed file is a source:'db' projection. The DB comparison is deliberately NOT on the default load path — it would cost more than the drift it detects — and runs in the generator, --report, and under MINUTES_REGISTRY_VERIFY=1. This generalizes: any committed file beside a table is a projection or it is a second answer.

2026-07-30 — One question, one answer, per table: the registry holds identity, a view holds state

• Context. raw.qc_minutes_target_registry — the table created on 2026-07-29 to end the seven-answers problem — grew to 36 columns, ~14 of them in one day, and rebuilt that problem inside itself: SIX columns answered "did it work?" (confidence, probe_status, completeness_state, live_proof_outcome, map_defect, harvest_eligible), five answered "why do we believe it?", four answered "when did we check?", three answered "what vendor?". column-redundancy passed throughout because it guards two NAMED FACTS and never asks whether one table holds four ways to say one thing. The cost was measured, not aesthetic: the stored live_proof_outcome snapshot and raw.qc_minutes_location_probe (the ledger) disagreed on 200+ rows — 123 locations the ledger proved reachable read NULL, while 76 robots-refusals existed ONLY as a registry column, so deriving from the ledger alone would have made forbidden hosts crawlable. vendor is populated on 1 row of 1,205 against site_family's 400, and the map gauge projects the dead one, so its work queue lost its cluster axis.
• Decision. A QUESTION EARNS A STORED COLUMN ONLY IF IT CANNOT BE ANSWERED FROM THE LEDGER. raw.qc_minutes_location_probe is the append-only event log of every fetch; migration 0472 (DRAFT) backfills the 77 verdicts that lived only as registry columns so the ledger is complete; 0473 creates serving.qc_target_state, the ONE derived view of reachability / eligibility / map defect / as-of, with terminal STICKY and tested first; 0474 marks 13 superseded columns DEPRECATED and drops none. (Numbers corrected 2026-07-30: this entry was written as 0471/0472/0473 and the set was renumbered in f5249aeb off the applied 0471 without the prose following. 0471 is applied and is the api.nearest_comps migration, so the original citation pointed at a different, already-applied migration. The recurrence is ratcheted as migration-citation in the spec below.) confidence and completeness_state are KEPT — they answer different questions (which tier supplied the URL; is the corpus complete) — and re-commented.
• Consequences. Eligibility comes from evidence rather than a snapshot: 554 locations where the stored answer reached 428, with the 428 a strict subset (0 lost) and 0 of the 131 terminal locations eligible. Ratchet: validate:arch's same-question-columns (one evidence payload, one we-checked timestamp, no stored copy of a named ledger's answer), cleared by a DEPRECATED column comment so invariant 3 still forbids drop-and-stop-writing in one migration.

2026-07-30 — A published read-contract may not apply a silent default, and states what is IN its answer

• Context. api.nearest_comps applied no class filter when the caller omitted property_class, which the seller-evaluation path does on every call. Measured on prod: the comp universe is 2 777 691 rows of which 323 736 (11.7%) are class other, averaging $1 568 083 against single-family's $458 459; over a deterministic 209-subject sample at that surface's own call shape (700 m, lim 50), 108 of 209 subjects received at least one other comp, inflating the comp mean by up to $468 870. That mean is the one number a homeowner is shown. Separately, the class expression sent every row with a NULL property_type — 89.1% of all properties — to ELSE 'other': a confident label for an absence.
• Decision. Migration 0476 (DRAFT, supersedes 0471). (1) When the caller passes no class, the function resolves the subject at the origin — typed civic+street match, else the nearest property within 50 m — and defaults the filter to its class; unresolved falls back to today's unfiltered behaviour, so the change can only narrow, never widen. An explicit argument always wins: a default, not a policy. (2) An unknown property_type yields a NULL class; 'other' becomes a claim we can defend. (3) The response declares what was applied — property_class_applied / property_class_source per row, because a caller cannot distinguish "filtered to X" from "unfiltered and every row was X". The v0 envelope adds a two-part honesty channel: meta.counters (set-level tallies that ARE derivable from data, so they are computed once in the envelope rather than repeated per row) and meta.filters (what the database decided, which is NOT derivable at all).
• Consequences. No api.* function may apply a default that changes its answer without naming it in the payload. A derivable set-level aggregate belongs in the envelope, never as a column (derive-over-store applies to responses, not only to tables). Cost is stated, not hidden: measured A/B, the product path got faster (52 ms → 44 ms at 700 m / lim 50) while the route's declared maximum (25 km / lim 200, which no consumer calls) went 2 686 ms → 7 447 ms, because filtering on class forces a join the unfiltered path could defer past LIMIT. Evidence: docs/architecture/2026-07-30-comp-class-contamination-sample.sql.

2026-07-30 — The publication registry is LOCATION-grained, and doc_type stays on the document

• Context. raw.qc_minutes_target_registry landed as one row per BODY with one minutes_url. Québec municipal documents are statute-governed, so all 1,098 municipalities produce the same KINDS (procès-verbaux, règlements, avis publics, budget, PTI, états financiers, rôle d'évaluation — 19 doc_type values already held); only ACCESS varies. A publication location is usually kind-agnostic, but some bodies publish different kinds at different locations — which one minutes_url cannot express. Changing the grain was free while the table had no reader keying on it; after the ~74k-document pull it would not be.
• Decision. Migration 0458 (DRAFT) rekeys the drawer to (target_code, location_key), with location_key a GENERATED column (URL minus scheme/trailing slashes, or the (none) sentinel so a dark target keeps one queryable row) and serves_kinds[] declaring which kinds a location serves (EMPTY = kind-agnostic). doc_type stays a property of the DOCUMENT; a column-per-kind would violate the one-fact law. The drawer is NOT renamed — qc_minutes_* is a legacy string like overwatch (invariant 3). Same migration retires website / website_source_tier: canonical.entity_governance.official_url owns that fact and it is now joined, not copied.
• Consequences. In code every registry store is target_code -> TargetRecord[], and a target's N verified locations become the N seeds of ONE MunicipalityConfig, so ingest.ts was unchanged. Body facts (name, kind, entity_id, completeness_state) now repeat per location row; the migration asserts they agree, and the mechanical guard (a btree_gist EXCLUDE, or a parent target table) is the named follow-up.

2026-07-30 — The api.* read-contract states its ORDER and its COMPLETENESS

• Context. The seller-evaluation surface renders one number to a homeowner from api.nearest_comps and nothing else, so that payload is the product. It could not carry two things it needed. (1) The function ORDERed by the <-> sphere operator while RETURNing the spheroid ST_Distance: at a real 700 m Châteauguay origin those rank 756 of 1 167 rows differently (up to 1.96 m apart), and equal distances had no tiebreak at all, so lim cut an underdetermined set and a top-N valuation inherited the instability. (2) A failed query and a genuinely empty result both reached the caller as an empty array, and the honest responses are opposite — one is an answer to show, the other must suppress the number.
• Decision. A published api.* read-contract states its ordering and its completeness, and both are enforced rather than implied. Ordering is a total order ending in a unique NOT NULL key and sorting on the value the function actually RETURNS (migration 0471, DRAFT: distance_m ASC, property_id ASC, documented in the route + README). Completeness is a positive assertion: every v0 200 carries meta.complete: true, there is no partial-success path, and every failure is a classified non-200 carrying a reason and a retryable flag (contract_mismatch / upstream_timeout / db_unavailable / query_failed). Disclosure fields travel with the fact they qualify — a comp row carries its own roll_year / roll_market_date and its municipality_code / municipality_name, read from the same canonical rows api.property reads, so the surfaces cannot disagree.
• Consequences. "Nearest-first" is no longer a contract any surface may rely on without a documented tiebreak; a KNN operator may bound a candidate set but may not define a returned order. Any DROP+CREATE in api.* must re-grant every role read from pg_proc.proacl first (api_reader and ortova_preview_reader today) — the unapplied 0394 re-grants only the former and would silently revoke the latter. Visibility is not policy: the function exposes the municipality boundary it crosses and deliberately does not filter on it. Caveats travel as meta.warnings[] (a list, per GraphQL/Prometheus/OGC prior art), never as a growing set of booleans. Ratchet: validate:arch's new api-route-arity compares every v0 route's argument count against the live pg_proc signature — it currently FAILS on nearestComps (routes.ts passes 7, the database publishes 5), which is the HX-019 outage, and it goes green the moment 0471 is applied. A route may not ship against a signature only a DRAFT migration provides.

2026-07-29 — ONE target table for the QC-minutes harvest

• Context. "Where does this body publish its procès-verbaux?" was answered in seven places (qc_minutes_coverage_census.minutes_url/.website, ..._seed.minutes_url, qc_municipality_directory.website, canonical.entity_governance.official_url, municipalities.ts, discovered-archives.json) with no rule about which won. Measured cost: the fleet idled on "no resolvable config" for 240 municipalities whose URL we already stored, and an agent went scraping the MAMH directory to reproduce a column we already hold.
• Decision. raw.qc_minutes_target_registry (migration 0450, DRAFT) is the single source of truth: one row per target for the whole universe (1,205 = 1,098 municipalities + 107 bodies), ONE authoritative minutes_url, with minutes_url_source_tier / evidence / confidence / last_verified_at beside it. The seven stores stay as read-only INPUTS to one reconciler (target-registry.ts's TIER_PRECEDENCE); they stop being independent answers the resolver consults in parallel. Only confidence='verified' is harvested.
• Consequences. "Which targets do we know / what is still dark" is a SELECT, not a script. A new source of minutes URLs is a new tier in one precedence list, never a new answer.

2026-07-22 — Colocated documentation architecture

• Context. Shape and prose kept drifting from code; docs lived far from what they described.
• Decision. Schemas are the source of shape (named Zod in schemas.ts, types via z.infer); one README.md per deployable unit; one root ARCHITECTURE.md; the same files double as CLAUDE.md agent context. Order: docs/ops/orders/2026-07-21-colocated-docs-architecture.md.
• Consequences. A new boundary means a new named schema in the same commit; a new deployable unit means a new README in the same PR; irreversible decisions land here before the implementing commit.

2026-07-22 — Serving is open-format, never platform

• Context. A stateful tile server / vendor GIS middleware would insert a moat into the serving path.
• Decision. MapLibre + deck.gl over COPC / COG / PMTiles as range-readable static R2 objects; no tile servers, no GIS middleware. The formats are the contract between compute and UI.
• Consequences. Every heavy pipeline runs headless writing objects + facts; the browser only projects. Compute and UI never meet directly, so the machine compounds even with the UI down.

2026-07-22 — Drawers evolve additively (never supersede a production satellite)

• Context. Ratifying the imagery-pipeline architecture; supersede-and-rebuild was the tempting shortcut.
• Decision. A canonical satellite in production is never renamed / rebuilt / replaced; new source classes generalize into the existing drawer (added columns, widened enums).
• Consequences. Context compounds on one spine long-term; migrations widen, never fork, a fact.

2026-07-21 — Sub-metre optical is THE detector; 10 m Sentinel-2 retired

• Context. The 10 m path was carried under a "rural-reserved" reservation for a tier never built.
• Decision. Sub-metre optical (WV-3 × WV-2, 0.5 m) is the sole detector. The Sentinel Lambda, sentinel2-stac / sentinel1-rtc-mpc sources, ndvi-zscore signal, and 10 m dispatchers are excised.
• Consequences. image-change is a pre-production shell awaiting a sub-metre source; no rural tier exists. Tombstone: docs/ops/2026-07-21-repo-staleness-audit.md.

2026-07-13 — GitHub Actions / the mac fleet is the batch cloud-execution path

• Context. Long-running ingests should not sit on the HQ laptop's critical path, and stale-code builds were a recurring failure.
• Decision. Batch jobs run via gh workflow run <wf> -f ... off merged main on the ortova-mac self-hosted fleet (mac-fleet-first, 2026-07-21). Railway is retired; Fly.io is the always-on fallback.
• Consequences. Runners check out main, so jobs always run merged code. Home: docs/architecture/cloud-execution-paths.md; ratchet: validate:wiring cloud-exec-path-wired.

2026-06-25 — Generated-satellite auto-DDL lane (zero-touch onboarding)

• Context. Onboarding a new source required hand-writing the satellite DDL each time.
• Decision. A Rule-6 extension auto-generates the typed satellite DDL for zero-touch source onboarding. Design: docs/architecture/zero-touch-source-onboarding.md.
• Consequences. Adding a source is scaffold-driven; the satellite shape stays catalog-complete by construction. defineService / civic-service was retired the same window (never built).

2026-06-24 — api.* is the published PII-free read-contract (system of record)

• Context. The agency was duplicating the platform's read surface, causing drift.
• Decision. api.* (consumed as @nexod/property) is the sole published reader — SECURITY DEFINER + an api_reader role with zero canonical.* grant.
• Consequences. PII is unreachable by construction; the agency drops its rf_* / agency.* copies. Migrations 0102–0118 applied.

2026-06-23 — Target data model v1: typed temporal logs + reversible identity + as-of layer

• Context. property_observation conflated facts and could not answer as-of questions.
• Decision. Typed canonical.property_* temporal logs (SCD-2), reversible identity normalized at ingest, and an as-of derivation layer. property_observation / recordObservation() / resolveToProperty() retired (migration 0075).
• Consequences. Each fact owns its satellite + provenance; derive-over-store is enforced by validate:arch. Canon: decisions/canon/data/data-shape-v1.md.

2026-04-27 — Six primitives + four fixed dispatchers

• Context. Ad-hoc pipeline code had no enforced topology.
• Decision. Five pipeline primitives (subject / source / signal / bundle / product) + defineAction, all via define* factories; the four-dispatcher topology is fixed and throws at module load on bypass.
• Consequences. The generative loop is uniform and mechanically enforced; new sources/signals/ products slot into fixed seams. Home: decisions/canon-now.md (§Engine paradigm).

Deployable units

README.md

Ortova

The intelligence layer for the built world. Ortova is a property system-of-record: it joins, on one spine, every property's permit history, ownership, assessed/market value, and physical change detected from sub-metre imagery — keyed to the matricule (Québec's universal parcel ID). The spine is the moat; the products are equal views of it, each green/red-lit by what the Atlas holds. There is no single "lead product": roofing leads (pre-production experiment), a municipal compliance worklist (designed), and the further config-products (insurer feeds, lender monitors, pre-closing reports, contractor leads) are all configs on the same spine — the stack and each product's honest maturity live in PRODUCT.md. Entry scope: the Verdun arrondissement of Montréal, with Laval second.

(The company/product is Ortova; the repo, packages, and paths are still named overwatch — legacy, not a second thing.)

• Full product description — vision, the open wedge, kill list, data sources, status: PRODUCT.md.
• The controlling vision + roadmap: docs/founder/the-machine.md.
• The agent operating contract — conventions, per-area guidance, DB/git discipline, execution protocol: CLAUDE.md.

Engine paradigm

The six primitives (subject / source / signal / bundle / product + defineAction) built through the define* factories in @ortova/platform, and the fixed four-dispatcher topology, are canon — single home: decisions/canon-now.md (§Engine paradigm) + decisions/enforced-index.md (build-enforced). defineService was proposed and retired, never built. Not restated here.

This file is the authoritative repo-structure map: when this layout and CLAUDE.md's per-area guidance overlap, this file wins for "what is where," and CLAUDE.md wins for "how to behave there."

Repository layout

apps/

The runtime. Six apps:

• apps/core/ -- the engine implementation: the signal/source/subject/product/bundle declarations and their producers. Subdirectories under apps/core/src/ are signals/, sources/, subjects/, products/, bundles/, canonical/ (reconcilers), atlas/, and inngest/ (dispatcher wiring). This is where domain work lands.
• apps/dev/ -- the Next.js internal operator surface (audit dossier rendering, signal/source/product views, the atlas, the ops cockpit, demo screens). Vercel-deployable. Local dev: pnpm --filter @ortova/dev dev. The survivor knowledge home lives inside it at apps/dev/src/brain/ (see below).
• apps/landing/ -- the public marketing site, live at ortova.io (Next.js, Vercel). Deployed via scripts/ops/release-deploy.sh --i-am-felix landing.
• apps/worker/ -- cloud-side sidecars, run next to the DB off the network-bound laptop (default batch trigger is GitHub Actions, not Railway — Railway is retired, trial exhausted): libpostal-sidecar/ (address canonicalization), rf-stagehand/ (Registre foncier browser scraping, bounded query budget), geo-ingester/ + geo-trinity/ (cloud-side bulk geo ingestion), tile-builder/, minutes-harvester/ (municipal meeting-minutes harvest), governance-parser/ + permit-doc-parser/ (document extraction), and permit-ingester/.
• apps/score-api/ -- a small Next.js app serving property-level score lookups (no current surface routes to it; live-vs-fossil status is open — don't delete pending that call).

packages/platform/

@ortova/platform -- the internal infrastructure: the six define* factories (incl. defineAction), the four-dispatcher topology, storage abstractions, observability with Slack alerts, and the dashboard scanner. apps/core/ is built on this package's contract. Changes to its exposed surface are consequential; make them deliberately and note the contract change in the commit.

pilots/

pilots/verdun-tier1-gee/ -- the Verdun Tier-1 Google Earth Engine pilot (Python/GEE). Validation and exploration surface, separate from the production engine.

data/

data/ -- the atlas: reference material for source and signal design, not runtime-loaded. Subdirectories: atlas/ (sensors, enrichments, transformations, vocabulary), scripts/, seed/, sql/, src/.

migrations/

migrations/ -- sequenced SQL migrations, numbered 0001... Migration order is the file numbering. Applied-migration manifests live in migrations/applied/. The sanctioned apply path is scripts/apply-migration.ts --i-am-felix <file> (Felix-only).

decisions/

The decision surface:

• decisions/canon-now.md -- what's true now (the active canon). Read on cold boot.
• decisions/enforced-index.md -- the canon rules mechanically enforced in the build (generated).
• decisions/decisions.md -- the full decision log (archive: why, not what's true now).
• decisions/source-priority.md -- source onboarding priority.
• decisions/canon/ -- the live canon set (architecture + data-shape).

docs/

• docs/founder/ -- the vision + operating layer. the-machine.md (North Star + roadmap), ortova-ai-native-org.md (how we build), buildout/fleet-zones.md (the 9 standing zones HQ dispatches against), founder-mode.md (the position room).
• docs/briefs/ -- the live process spine; work is dispatched as briefs (the model that replaced the order/receipt ceremony, docs/briefs/2026-05-13-process-rectification.md).
• docs/architecture/ -- architecture notes, incl. enforcement-contract.md (the rules binding agent + human work).
• docs/research/, docs/design/, docs/dossiers/, docs/validation/, etc. -- supporting research, design, and deliverable libraries (reference, not the context layer).
• archive/2026-06-02/ -- cold-archived ceremony (the order/receipt loop + its validators), retained as historical record. Never delete.

Survivor knowledge home

apps/dev/src/brain/ holds the load-bearing institutional knowledge -- inline knowledge cards plus MAP.md. Source-capability truth, compliance/legal notes, cost-model floors, the DR191 NDVI rationale. Treat it as load-bearing, not docs cruft.

Getting oriented

1. PRODUCT.md -- what Ortova is.
2. docs/founder/the-machine.md -- the controlling vision + roadmap.
3. run pnpm state -- prints the live "where we are right now" (branch, migrations, zones); fresh from the system, not a committed file.
4. decisions/canon-now.md + decisions/enforced-index.md -- what's true now + the build-enforced rules.
5. CLAUDE.md -- conventions, per-area guidance, execution discipline.

Build and validate

pnpm install
pnpm typecheck
pnpm validate:arch   # architecture invariants (incl. the wiring guardian)
pnpm validate:wiring # every doc claim / path / command resolves; boot docs are loaded
pnpm state           # regenerate STATE.md

apps/dev/README.md

@ortova/dev

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: the Next.js operator surface — three chrome-free scene surfaces (atlas map /, permit coverage, live /monitor) + the app/api/* serving edges.
• Input/output boundaries (Layer-1): every hand-shaped HTTP boundary is a named Zod schema in apps/dev/src/schemas.ts — request params (LeadScoresRequestSchema, FootprintsRequestSchema, SignalsRequestSchema, PilotRequestSchema, AtlasNeighborsRequestSchema, ScoreRequestSchema, BrainQueryRequestSchema, …), serving-edge responses (LeadScoreCollectionSchema, FootprintCollectionSchema, SignalCollectionSchema), envelopes (V0OkEnvelopeSchema / V0ErrEnvelopeSchema), and emitted events ({Source,Signal,Product}RunEventDataSchema). Product egress InsuranceScoreSchema lives in @ortova/core/products/insurance-score/shape. v0 request params are in src/lib/api-v0/routes.ts. Full map: boundary table §A. Verbatim api.* rows are to_jsonb passthrough (DB-owned, deliberately not hand-Zod'd).
• Run locally: pnpm --filter @ortova/dev dev (port 3030; needs .env.local with SUPABASE_POSTGRES_POOLER_URL).
• Deploy: Vercel — scripts/ops/release-deploy.sh --i-am-felix dev (Release Agent, gated). Live ortova.nexod.ca.

Operator dashboard for the Nexod Platform. Dark Palantir-style UI reading live state from Postgres (dytmsttyixbkltadddok, ca-central-1).

Routes

• / — pipeline overview
• /sources, /sources/[key] — source adapter health, runs, config
• /signals, /signals/[key] — signal producers + model versions
• /products, /products/[key] — products + delivery clients + recent leads
• /workers — Inngest function inventory grouped by primitive
• /bundles — scoring bundle versions
• /health — adapter health + recent failures
• /report — sprint data report with live KPIs + print-to-PDF
• Demo routes under (demo)/*: /gis, /permis, /use-cases, /opportunities, etc.

Operational scripts

# Local dev (requires .env.local with SUPABASE_POSTGRES_POOLER_URL)
pnpm --filter @nexod/overwatch-dev dev

# Smoke-test every route against a running URL
SMOKE_BASE_URL=https://overwatch.nexod.ca \
  pnpm --filter @nexod/overwatch-dev test:smoke

# Assert the Verdun/Laval cascade baselines
SUPABASE_POSTGRES_POOLER_URL=... \
  pnpm --filter @nexod/overwatch-dev cascade:replay

# Fire a synthetic alert to verify Slack wiring
SUPABASE_POSTGRES_POOLER_URL=... SLACK_WEBHOOK_URL=https://hooks.slack.com/... \
  pnpm --filter @nexod/overwatch-dev alert:smoke

Slack observability setup

1. Create a Slack incoming webhook in the target channel.
2. Set SLACK_WEBHOOK_URL on the Vercel project (Settings → Environment Variables → Production).
3. Redeploy (vercel --prod) — env vars are snapshotted at build time.
4. Run pnpm --filter @nexod/overwatch-dev alert:smoke to confirm the pipe works.

Once wired, the observability-digest-cron Inngest function (runs 05:00 UTC daily) scans raw.observability_alerts for unposted rows and ships them to Slack.

CI

• .github/workflows/dashboard-smoke.yml — Playwright smoke runs on every successful Vercel production deploy.
• .github/workflows/cascade-replay.yml — cascade baseline check runs nightly 04:00 UTC.

apps/landing/README.md

@ortova/landing

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

The public marketing site (ortova.io). Next.js, static/marketing-only.

• What it is: the outward brand + narrative site. No spine data, no operator surface.
• Input/output boundaries (Layer-1): none. This unit has no runtime data boundary — no HTTP API, no DB read, no queue message. Nothing here resolves to a schemas.ts name by design (marketing content, not a serving edge). Any future form/lead-capture endpoint gets a named schema in a local src/schemas.ts in the same commit (standing rule).
• Run locally: pnpm --filter @ortova/landing dev (port 3032).
• Deploy: Vercel — scripts/ops/release-deploy.sh --i-am-felix landing (Release Agent, gated). Live domain ortova.io.

apps/worker/geo-ingester/README.md

geo-ingester

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: cloud-side bulk-ingest of a national/regional geo source into raw staging, off the HQ laptop's critical path.
• Input boundary (Layer-1): WorkflowArgsSchemas['geo-ingest'] = { source, phase? } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema when Watch-driven. External payloads validated per adapter-kind schema.ts (boundary table §D).
• Output: raw geo staging rows → canonical upsert. DB-owned shape (not hand-Zod'd, §C).
• Run locally: pnpm --filter @ortova/geo-ingester plan (dry) / ingest.
• Deploy: gh workflow run geo-ingest.yml -f source=<key> (runs-on ortova-mac).

Reusable bulk-geo ingestion job: pulls a large external vector layer into a raw Postgres table, province-wide. Built to run on a cloud box with a fat pipe to the DB — not a laptop. Province-scale geometry (cadastre ≈ 3.7 GB, footprints similar) is upload-bandwidth-bound; from a home uplink (330 KB/s) the cadastre took ~3 h, with the DB sitting idle in ClientRead the whole time. Cloud-to-cloud in/near ca-central-1 (where Supabase lives), the DB write speed is the floor instead of the pipe — **20–30 min for the full cadastre**.

What it does (per INGEST_SOURCE)

1. Reads the source from sources.ts (ArcGIS FeatureServer URL, OID field, fields, target table, scope).
2. Harvests the layer in OID-range chunks — orderByFields=<oid> + FEATURE_SERVER_PAGING (stable paging; without it ArcGIS silently dupes+gaps), validating every chunk COUNT(*) == COUNT(DISTINCT oid) == range size, with 503 backoff retries.
3. Loads each chunk → staging via ogr2ogr COPY (-append), statement timeout disabled, one ~3 MB chunk on disk at a time.
4. Applies staging → target in WAL-safe ogc_fid batches (a single multi-M-row INSERT can PANIC the instance), scope-tagged.
5. Verifies the target row count equals the source count.

Idempotent: the apply refuses if the scope is already loaded.

Lessons baked in (so we never re-fight them)

• ArcGIS pagination is unstable without orderByFields → silent dupes + gaps. Always ordered.
• GlobalID is not unique in some QC layers — validate on OBJECTID.
• The public service 503s under sustained pulls → per-chunk retries + backoff.
• The pooler statement_timeout kills long COPYs → disabled on the connection.
• A single province-size INSERT/COPY blows WAL → chunked apply, chunk-at-a-time load.
• ogr2ogr -overwrite on the first chunk DROPs + recreates the staging table, wiping any table/column comments a prior catalog seed left there — which used to leave the staging table uncataloged after every run and trip validate:catalog red until someone re-ran scripts/seed-dataset-catalog.py by hand. Fixed (Felix decision 2026-07-08): the worker owns its catalog — stampCatalog() in ingest.ts re-applies the meta.dataset_catalog row + native COMMENTs for both the staging table and the target table at the end of every successful harvest, sourced from the stagingCatalog/targetCatalog fields on each sources.ts entry. Best-effort/non-fatal by design — a catalog-stamp failure never fails an otherwise-successful harvest. A new source's catalog content lives with its other declarative fields in sources.ts.

Run it

Sanctioned cloud-execution tiering — full home: docs/architecture/cloud-execution-paths.md. Every trigger honours the same principle: never run stale laptop code; run merged main (the 2026-07-08 build-model decision). What changed 2026-07-13: the free GitHub Actions runner, not the paid Railway image, is the DEFAULT.

PRIMARY (default): GitHub Actions — .github/workflows/geo-ingest.yml. A run-to-completion job on a free GitHub runner with a fat pipe to the DB (~20–30 min vs ~3 h from a laptop). actions/checkout@v4s main, so the code that runs is always merged main — the same never-run-stale guarantee, for free. Dispatch:

gh workflow run geo-ingest.yml -f source=cadastre_qc
gh run watch                                # tail to completion; read the final VERIFY row-count line

INGEST_PHASE — split harvest from apply (resumable, INV-4)

The pipeline is harvest → stage → apply. INGEST_PHASE (workflow input, or the env var locally) splits it so a long run can't strand a full stage table before the apply. timeout-minutes is 300 (a 414-tile province Overture harvest overran the old 120 cap at tile 28 and left 2.8M rows stranded in raw.*_stage with a zero target — the failure this mode exists to make un-repeatable).

• full (default) — harvest + apply, one run. Unchanged behaviour.
• harvest — stage only, apply deferred. Use to keep a huge harvest under a timeout, then finish with an apply run.
• apply — promote ALREADY-STAGED rows → the target without re-harvesting. Idempotent + resumable: the apply is ON CONFLICT (<target PK>) DO NOTHING in 200k-row fid batches, so a re-run (or a crash mid-apply) converges to the full set without double-writing — the fid-ordered batching is the watermark. Only for sources that declare applyConflictTarget in sources.ts (e.g. footprints_qc).

gh workflow run geo-ingest.yml -f source=footprints_qc -f phase=apply   # finish a stranded stage in minutes

source is a key from sources.ts (cadastre_qc, footprints_qc, …). The workflow runs ingest.ts --plan (read-only) then ingest.ts --i-am-felix (the Felix-gated write) with SUPABASE_POSTGRES_POOLER_URL from repo secrets. gh workflow run executes the workflow as it exists on the default branch (main), so a workflow edit must LAND on main before a dispatch can use it. Felix-gated: the Release Agent dispatches on Felix's approved set. Dry-run locally first if you like:

INGEST_SOURCE=cadastre_qc npm run plan      # prints the plan, no writes

LEGACY / fallback: Railway (scripts/ops/release-worker.sh). The git-connected Railway image still exists and still runs merged main, but its trial is effectively exhausted (2026-07-13) — do not provision new Railway services. Build model (decided Felix 2026-07-08): this service builds its Railway image directly from GitHub main (railway up's upload-from-cwd model is what let a merged fix run stale in prod once already; see scripts/ops/release-worker.sh's header). One-time per service, railway service source connect --repo <owner>/<repo> --branch main --service geo-ingester, Root Directory = apps/worker/geo-ingester, Dockerfile path = Dockerfile (mirrors railway.json's build.watchPatterns: ["apps/worker/geo-ingester/**"]), region ca-central-1. Triggering: bash scripts/ops/release-worker.sh --i-am-felix geo-ingester INGEST_SOURCE=cadastre_qc. Raw railway up / railway redeploy are not an allowlisted agent path. Use only an already-connected service; prefer GitHub Actions above.

FALLBACK for an always-on job: Fly.io (ca-central-1, scale-to-zero) — for a standing service rather than a run-to-completion dispatch. Not needed for batch geo (that is GitHub Actions).

After cadastre_qc lands (raw.qc_cadastre_lot, scope qc), run scripts/reconcile-parcel-qc.ts --scope qc --apply --i-am-felix to resolve canonical.property.parcel_id province-wide.

Adding a layer

Add an entry to SOURCES in sources.ts — no code change for another ArcGIS FeatureServer. (Overture footprints need a second kind — DuckDB over S3 GeoParquet by tile; the staging→apply→verify stages here are reused.)

US metro footprints (tile 401) — one table, many scopes

footprints_us_<metro> (20 sources, US_METRO_AOIS in sources.ts) is the sibling of footprints_qc generalized to disjoint AOIs instead of one province bbox: one national raw table (raw.us_overture_building, migration 0245), scope-tagged per metro (us_nyc, us_austin, …) — mirrors raw.qc_overture_building being "one table per DATASET KIND, scope-tagged," not "one table per city." Each metro source gets its own staging table (raw.us_overture_building_stage_<metro>), unlike footprints_qc's single shared stage — that's deliberate: 20 distinct sources sharing ONE stage table would let two concurrent dispatches clobber each other's in-flight -overwrite; isolated per-metro stage tables make concurrent metro dispatch safe. Adding a metro = add a row to US_METRO_AOIS (bbox + tileSize + measured building count) + its scope_code to migration 0245's reconciler list — no new code path. AOI bboxes are derived from the US Census TIGERweb "Incorporated Places" extent for each metro's GEOID (canonical.property_permit.municipality_code), padded 0.03°; see the US_METRO_AOIS header comment for the full citation and the measured canonical.property.centroid join-feasibility caveat (7 of the 20 metros have near-zero centroid coverage today — an upstream geocoding gap, not an ingest defect).

apps/worker/geo-trinity/README.md

geo-trinity

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: the release-sync geo pipeline — bakes GeoParquet + PMTiles and attaches footprints.
• Input boundary (Layer-1): WorkflowArgsSchemas['geo-trinity'] = { job: 'release-sync', release, shard? } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema when Watch-driven.
• Output: GeoParquet/PMTiles → R2, plus canonical.property_footprint rows. DB/R2-owned (§C).
• Run locally: pnpm --filter @ortova/geo-trinity release-sync (needs R2_* in .env.local).
• Deploy: gh workflow run geo-trinity.yml (runs-on ortova-mac).

The continental buildings trinity (tile 431, Felix's architecture decision 2026-07-14): three jobs that pin, join, and serve Overture Maps' NA buildings layer at national scale — off the laptop, cloud-side, next to R2 and the DB. Supersedes apps/worker/geo-ingester's per-city AOI approach for buildings (footprints_us_<metro>, tile 401): instead of ~20+ hand-declared metro bboxes landing rows straight into Postgres, this pins the WHOLE continent to object storage once per release, and only SELECTIVELY joins matched buildings onto properties — a new city's buildings are already in the pin (its grid tile is already covered), no new source entry, no unmatched-building row ever lands in the DB at all.

The three jobs

1. release-sync.ts — extracts Overture's theme=buildings/type=building for one release, tiled over a 5°×5° North-America grid (lib/na-grid.ts, 288 tiles), and publishes partitioned GeoParquet + MANIFEST.json (release tag, per-partition row counts, byte sizes, sha256s) to R2 (or a local dir for testing). Read-only against Overture's public S3 (unsigned).
2. attach-join.ts — exports property centroids + QC parcel geometries from canonical.* (read-only), spatial-joins them against a pinned buildings partition in DuckDB (QC rows: parcel-contains-building-centroid; everyone else: 0249's centroid-in-building rule), and upserts MATCHES ONLY into canonical.property_footprint + canonical.building_height — same idempotent ON CONFLICT keys as migration 0249 ((property_id, footprint_id, source) / (toit_id, lidar_source)), so a monthly re-run of a newer release is a pure GERS-stable diff, not a rebuild.
3. tile-bake.ts — bakes ONE buildings.pmtiles from the pinned NA parquet via tippecanoe (see tile-bake.ts's header for why tippecanoe over planetiler) and publishes it to R2 alongside the release.

Every job is --plan (read-only, real counts) verified, --local-only (writes locally, never touches R2), and Felix-gated on the actual R2/DB write (--i-am-felix) — same discipline as apps/worker/geo-ingester and apps/worker/tile-builder, whose patterns this worker reuses (DuckDB spatial+httpfs preamble, PGOPTIONS pooler-timeout override, the psql-COPY-to-stdout-into-tippecanoe pipe).

Destination: R2-or-local (lib/dest.ts)

resolveDest() picks R2 when R2_ENDPOINT + R2_ACCESS_KEY_ID + R2_SECRET_ACCESS_KEY + R2_BUCKET are all set, else falls back to --local-dir <path> (or LOCAL_DIR env, or a WORK_DIR-relative default) — so every job runs and proves itself with zero R2 credentials. As of 2026-07-14 the R2 env DOES exist (Felix's Cloudflare account, bucket ortova-geo, region auto, in .env.local + GitHub Actions secrets), but this worker's local proofs (below) were deliberately run against --local-dir — the actual R2 --i-am-felix publish is a Felix/Release-Agent op (Rule 6 / the Blob-upload class of write), not something this drop executes itself, per the SCALE agent's own draft-and-prove mandate.

Local proof (2026-07-14, run against the real overturemaps-us-west-2 S3 bucket + the real SUPABASE_POSTGRES_POOLER_URL)

release-sync, partition atlantic_test (bbox [-64.5,45.9,-61.9,47.1], PEI + eastern Nova Scotia — chosen small/fast, sits over Ortova's not-yet-covered geography so it proves the mechanics without touching real spine data):

• --plan: 112,181 buildings (COUNT pushdown, no data read) — reconciled against a hand-run duckdb query against the same S3 path (bfnu85sft): identical count.
• Real extract (--local-only): 112,181 rows, 14.0 MB parquet, 2.6 min. Independent shasum -a 256 on the output file matched the sha256 the script wrote into MANIFEST.json (95618d3a...513502e).

release-sync, partition verdun_test (bbox [-73.6044,45.423,-73.5158,45.4816], added specifically because it's the ONE tile that overlaps live canonical.property rows — 10,080 Verdun properties, queried live via arrondissement ILIKE '%verdun%'): 26,082 buildings (--plan).

tile-bake, fed the atlantic_test parquet (--local-only): 112,181 buildings → 12.5 MB mbtiles → 12.2 MB PMTiles, 6 seconds wall-clock. pmtiles show confirms a valid tileset: zoom 2–12, 366 addressed tiles, bounds matching the source bbox.

attach-join fixture test (attach-join.test.ts, pnpm test / tsx --test attach-join.test.ts): builds 3 synthetic buildings + 2 synthetic properties (one with a QC parcel whose geometry contains a building's centroid while the property's OWN centroid is deliberately OUTSIDE that building — proving the parcel-contains rule is a genuinely different match path, not a redundant special case of centroid-in-building; one with no parcel_id, centroid-in-building only) and runs the EXACT joinSql() used in production through the real DuckDB binary. Asserts: 1 QC match, 1 centroid match, 2 distinct buildings, the untouched 3rd building excluded. Passing.

attach-join live reconciliation: canonical.property_footprint's constraint (UNIQUE (property_id, footprint_id, source)) and canonical.building_height's (UNIQUE (toit_id, lidar_source)) were read LIVE via pg_get_constraintdef and confirmed to match this worker's loaderSql() byte-for-byte against migration 0249's already-proven INSERT shape (0249 committed 2,661,560 US property_footprint rows this exact way). A full DuckDB-join-to-DB-write dry-run against a real metro's staged data (the tile's ask) needs one pinned buildings partition covering that metro — verdun_test's partition is the first one that overlaps live spine data; attach-join.ts --scope verdun --release 2026-06-17.0 --buildings <verdun_test.parquet> --dry-run is the exact ready-to-run command (see --dry-run output shape below) once that partition finishes extracting.

Sharding + resume (tile 431 follow-up, 2026-07-14)

Why: atlantic_test's measured tile (112,181 buildings, 2.6 min) and verdun_test's (26,082 buildings, 3.3 min) are both BELOW a full 5°×5° grid cell's average density (~150M NA buildings / 288 land tiles ≈ 520k/tile) — neither is even a full-size grid cell (both are smaller custom test bboxes, lib/na-grid.ts's TEST_PARTITIONS, not the production grid). A full-NA release-sync sizes to somewhere between "14-24 runner-hours" (if wall-clock is dominated by DuckDB's per-tile S3-scan overhead, which my two samples' near-constant 2.6-3.3 min regardless of row count hints at) and "40-60 runner-hours" (if wall-clock scales with row count at average density, scaling my faster sample's 43,146 buildings/min up to 520k buildings/tile ≈ 12 min/tile × 288 tiles ≈ 58 hours) — either way, one unsharded dispatch cannot fit a single GitHub-hosted job's 360-minute hard cap.

• --shard <index>/<count> (release-sync.ts, lib/na-grid.ts's shardTiles()) slices the 288-tile grid into count disjoint, order-stable slices — tiles[i] goes to shard i % count. Fixture-tested (lib/na-grid.test.ts): every shard 0..count-1 is disjoint, their union is the full grid, and the split is stable across re-computation (no hidden randomness). At count=12, 288/12 = exactly 24 tiles/shard (verified live: --shard 3/12 reports tiles=24).
• Resume: before extracting a tile, release-sync.ts does a cheap destination check (lib/dest.ts's existsAt/fetchJson — a HEAD/GET on a tiny per-partition sidecar JSON, NEVER a re-download of the parquet itself). If the tile is already pinned, it's skipped and its recorded metadata is reused. Proven live: ran --partition atlantic_test twice against the same local destination — first run: 4.7 min (real extract, sidecar published); second run (identical command): 0.0 min, logged already pinned (resume) — 112,181 rows ... skipping re-extract, Total: 1 non-empty partitions (1 resumed from a prior pin, 0 newly extracted). This is what turns a cap-death/eviction on shard N into a fast no-op tail on re-dispatch, not a redo of already-finished tiles.
• Manifests, sharded: a global MANIFEST.json assumed ONE writer; N concurrent shards writing it directly would race the same object key. Each --shard <i>/<m> run instead publishes its OWN partial manifest to overture/<release>/manifest/shard-<i>-of-<m>.json. lib/merge-manifests.ts reads every shard's partial manifest and combines them into the final overture/<release>/MANIFEST.json — it refuses a partial merge (non-zero exit, unless --plan) if any shard 0..count-1 is missing, rather than silently publishing a manifest that under-claims coverage. Proven live with synthetic 2-shard fixtures: merged 2 partitions/30 rows correctly; a 3rd-shard-missing run correctly listed all 3 as MISSING and refused. An UNSHARDED run (no --shard) is unchanged — it's the sole writer, so it still publishes MANIFEST.json directly.

Per-shard wall-clock estimate (honest range, not false precision): at the CONSTANT-overhead-dominated floor (my two samples' 2.6-3.3 min/tile regardless of row count), 24 tiles/shard ≈ 79-96 min/shard — comfortably inside the 300-min hosted cap. At the ROW-COUNT-scaled ceiling (43,146 buildings/min applied to the 520k/tile NA average), 24 tiles/shard ≈ **290 min/shard** — inside the 300-min cap but with almost no margin, and a single denser-than-average tile could push it over. Recommendation: dispatch the first 1-2 shards on ubuntu-latest and WATCH the real wall-clock (gh run watch); if any shard approaches the cap, either re-dispatch the remaining shards on a self-hosted runner (-f runner=ortova-mac, see the caveat below) or raise shardCount (more, smaller shards) — resume makes either recovery path free (already-pinned tiles never re-extract).

Runner input — correcting the record: the workflow's runner choice (ubuntu-latest default, ortova-mac self-hosted option) is a NEW input in this drop, not a mirror of an existing apps/worker/permit-ingester GitHub Actions pattern — permit-ingest.yml has no runner input today (verified: runs-on: ubuntu-latest is hardcoded there). ortova-mac is listed as a CANDIDATE label only; it does nothing unless a self-hosted runner is actually registered under that exact label (repo Settings > Actions > Runners) — registering/operating that runner is infrastructure ops this drop does not perform or verify.

Adding a partition / release

• New NA tile: nothing to add — lib/na-grid.ts's 5° grid already covers the continent; a new city's buildings land automatically on the next release-sync run for a new release tag.
• New release: RELEASE=<new-tag> — no code change; MANIFEST.json namespaces every partition under overture/<release>/.
• New test partition (for a local proof over a specific metro before a full NA run): add a { key, bbox } to TEST_PARTITIONS in lib/na-grid.ts.

The canon wiring

• meta.dataset_catalog's external-parquet locator shape + the drafted companion-table migration for pinned-layer rows: see migrations/0363_external_layer_catalog.sql (DRAFT, not applied).
• pnpm validate:wiring's external-layer-pinned check (scripts/validate-arch/wiring.ts): asserts a MANIFEST.json exists at the configured destination and its checksums match, skippable-with-warning when no R2 env / no manifest yet exists (so CI stays green pre-first-run).
• The retirement note for raw.us_overture_building / raw.qc_overture_building is drafted in tile 431's outcome, not executed — their drop is a LATER gated op, after the trinity is verified end-to-end in production.

Deploy (Felix-gated — drafted, not run)

Primary: GitHub Actions, .github/workflows/geo-trinity.yml (mirrors geo-ingest.yml's shape — free runner, ca-central-1-adjacent AWS/Cloudflare edge, always runs merged main). Dispatch:

gh workflow run geo-trinity.yml -f job=release-sync -f release=2026-06-17.0 -f shard=0/12
gh workflow run geo-trinity.yml -f job=attach-join -f release=2026-06-17.0 -f scope=us_nyc -f buildings=<r2-key-or-local-path> -f dryRun=true
gh workflow run geo-trinity.yml -f job=tile-bake -f release=2026-06-17.0 -f buildings=<r2-key-or-local-path>
gh workflow run geo-trinity.yml -f job=merge-manifests -f release=2026-06-17.0 -f shardCount=12

The full 12-shard NA release-sync dispatch (one gh workflow run per shard, all disjoint, each independently resumable — the Release Agent runs these, not this drop):

for i in 0 1 2 3 4 5 6 7 8 9 10 11; do
  gh workflow run geo-trinity.yml -f job=release-sync -f release=2026-06-17.0 \
    -f shard=${i}/12 -f runner=ortova-mac
done
# then, once all 12 have completed (watch each with `gh run watch`):
gh workflow run geo-trinity.yml -f job=merge-manifests -f release=2026-06-17.0 -f shardCount=12

(-f runner=ortova-mac per the coordinator's ask — see the caveat above: this only works once that label is a real registered runner; drop the flag, or set -f runner=ubuntu-latest, to dispatch on the GitHub-hosted default instead, watching the first shard or two before trusting the 300-min cap at scale.)

R2 secrets (R2_ENDPOINT, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET) are already in .env.local + GitHub Actions secrets (Felix, 2026-07-14) — this workflow is ready to dispatch on Felix/Release-Agent's go. No new Railway service (trial exhausted, no new services per docs/architecture/cloud-execution-paths.md); a Dockerfile is still provided here for the always-on-fallback Fly.io path if a standing service is ever warranted, but GitHub Actions run-to-completion is the primary path for all three jobs (none of them is a long-running service).

apps/worker/governance-parser/README.md

governance-parser — QC council résolutions + règlements, parsed from held PVs

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: extracts council résolutions/règlements from held minutes PVs (LLM + rules).
• Input boundary (Layer-1): CLI --muni / --doc-types / --limit / --plan / --rehearse / --apply. Hand types PvMeta / Extracted* in types.ts — no Zod schema yet (FLAGGED, boundary table §C; Zod is follow-up).
• Output: governance facts → DB. DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/governance-parser dry-run (also classify-dry-run, extract-join-dry-run). To size up a municipality or a doc_type before firing anything: tsx ingest.ts --plan --muni <code> --doc-types <list> --limit 20 — read-only, prints yield per doc_type + the number-scheme histogram, and writes a sample of the rows to out/ to be OPENED.
• Deploy: .github/workflows/governance-parse.yml (runs-on: ortova-mac, sharded one job per municipality). Not a WORKER_SPECS entry — that workflow IS its deploy config.

DEPTH play of the QC Municipal Governance Corpus (master map Tier 1: docs/research/qc-minutes-lane/2026-07-18-qc-municipal-governance-corpus-map.md). Turns the 66,617 procès-verbaux we already hold (raw.qc_minutes_documents, plain text at text_object_uri) into the higher-value résolution (numbered council decisions) and règlement (by-law lifecycle) folders — ZERO new scraping.

• Decision it informs: are the résolution + règlement folders fillable at quality from the PV text we already have, and for how many munis?
• Action it unblocks: filling the resolution + reglement folders (foundation canon 2026-07-19-qc-governance-corpus-foundation.md) for the 635 PV-covered municipalities, feeding canonical.entity_resolution / canonical.entity_reglement (migrations 0386 / 0387).

Lives in apps/worker/ (self-contained, non-workspace — like minutes-harvester) because the work is bulk text parsing + fetch. Install standalone: pnpm install --ignore-workspace.

Files

• parse-sommaire.ts + patterns-sommaire.ts — the sommaire-décisionnel extractor (a FORM, not a procès-verbal — see the section below). ingest.ts dispatches to it on doc_type='sommaire-decisionnel'.
• sommaire-random-audit.ts — read-only, random-sample field-yield measurement for that extractor + N verbatim extractions for hand-check. Exists because --plan --limit N slices one contiguous harvest era, not the corpus.
• patterns.ts — the French council-minutes regexes (résolution number schemes, proposé/appuyé, adoption markers; règlement number + stage keywords).
• parse-pv.ts — pure, deterministic core: parsePv(text, meta) → résolutions + règlements. Idempotent (same text → same external_id + content_hash).
• types.ts — the record shapes (mirror the raw tables 0386/0387).
• run-dryrun.ts — read-only measurement (samples PVs, parses, reports yield/precision + a hand-check sample). No writes.
• ingest.ts — Felix-gated writer + reconcile (--apply --i-am-felix); default dry-run. UPSERTs raw + reconciles to canonical on the MAMH code.
• parse-pv.test.ts — unit test (three schemes, stages, precision guard, idempotency).

How résolutions are anchored

A résolution is a block led by a resolution-number token at the start of a line. Numbering schemes vary by municipality; the parser matches (in order): slash_nnnn_mm_yy (0064/02/21), alpha_yyyy_n (VS-AJ-2020-1, CM-2026-123), spaced_alpha (CM26 02 064), yyyymmdd_seq (20250811-01), yyyy_mm_dd_dot (2021-11-22.01), date_seq (2025-11-17-29), dot_seq_yymmdd (150504.094), yyyy_mm_seq (2022-03-44, 2025-07-202), yy_mm_nnn (21-11-901, 24-04-11265), keyword (RÉSOLUTION 2024-80). Precision guard: a block is accepted only if it carries a resolution signal (il est résolu / sur proposition / attendu que / considérant / an adoption marker).

How règlements are gated

A règlement number is emitted as a lifecycle event only when a STAGE keyword (avis de motion / adoption du règlement / projet de règlement / second projet / abrogation) sits with it. A bare reference — dérogation mineure au règlement de zonage no 4300 — is NOT an event and is excluded. That gate is the drawer's precision guard (verified: 30/30 hand-checked règlements were real proceedings, zero bare references leaked).

Measured (dry-run, 40 munis × 6 PVs, 2026-07-19, extractor v1)

• Résolutions: 2,135 across 240 PVs = 8.9 / PV avg (median 1; the mean is carried by full council munis, tens per PV); 33 / 40 munis yielded ≥1. Precision: 30/30 hand-checked = 100% real numbered council resolutions.
• Règlements: 243 events = ~1.0 / PV; 31 / 40 munis yielded ≥1; stages avis_motion/projet/projet_2/adoption/abrogation. Precision 30/30 = 100% real proceedings; number field exact 25/30 (5 truncate a compound number or grab a referenced by-law — a v2 item).
• Muni → spine join: 635/635 = 100% (deterministic MAMH-code resolve).

The four feed cities — measured 2026-07-27 (--plan), and the tile-513 borough verdict

The extractor was built and precision-verified on small/mid-municipality proces-verbal documents. The feed cities mostly hold OTHER doc_types, and they are not interchangeable.

Why the drawer held 8 rows for these cities: not a borough problem — ingest.ts hardcoded doc_type='proces-verbal', and only 520 of the ~81,000 feed-city documents carry that type. --doc-types now parameterizes it.

sommaire-decisionnel WAS a HARD HOLD, not a low yield. Opening the plan sample showed the extracted "resolution numbers" were the PDF's own print timestamp out of the page header (2026-07-15 14:44:49 Page : 1 de 2 sommaire décisionnel), and the remainder were citations of earlier résolutions the summary refers to. Firing parse-pv.ts over Québec's 73,656 sommaires would have injected on the order of 25–30k false municipality-level résolution facts.

The hold is lifted for this doc_type only, by its own extractor — see below. The 0.4 res/doc in the table above is the corrupt figure, kept so the history stays legible.

The sommaire-décisionnel extractor (parse-sommaire.ts, 2026-07-27, tile 521)

A sommaire décisionnel is not a procès-verbal. A PV is a stream of numbered council decisions anchored by a number at a line start; a sommaire is one administrative decision-summary sheet, laid out as a form, keyed by its own structured number. The two do not share an extraction model, and pretending they do is the defect above. So ingest.ts dispatches on doc_type and this extractor anchors on the form's own labels, never on a bare number at a line start.

One sheet → ONE row in raw.qc_governance_resolutions:

The Québec City number scheme is verified against all 73,662 held sommaires via their source_url basename: 73,662/73,662 match, 0 misses (census 2026-07-27).

What it refuses to emit (each one is a tested red fixture)

1. Page furniture. Every candidate passes isPrintFurniture() first — the print timestamp, the GPD1101R form id, Page : 1 de 2. No override path. A page header carrying no IDENTIFICATION line yields zero rows.
2. Citations. Numbers under DÉCISION(S) ANTÉRIEURE(S) are prior decisions being referenced. In Laval every CM-YYYYMMDD-NNN in the sheet lives there — complete with IL EST PROPOSÉ PAR / APPUYÉ PAR / ADOPTÉ, i.e. every signal the PV parser reads as proof of a real decision. The sheet is a recommendation to a council that has not sat.
3. A guessed date. Laval's raw rows carry session_date on 65 of 2,763 documents (2.4%), and the only in-text dates are Date CE/CM souhaitée — desired sitting dates. Those never reach session_date; it stays NULL. A NULL is a gap someone can close; a plausible wrong date is corruption nobody will ever find.
4. Règlement lifecycle events. A sommaire recommends an AVIS DE MOTION; it does not enact one. Dating a stage from a recommendation sheet would be wrong, so reglements is always empty.

Measured — TWO independent 300-document RANDOM samples, 2026-07-27

tsx sommaire-random-audit.ts --n 150 --show 6 --emit-linked-resolutions. Random, not a --limit slice (that is one contiguous harvest era; Québec City's sommaires span 2005–2026). Two independent draws are reported as a RANGE because a single draw's tail figures move by ~10 points.

The audit exits non-zero on any page-furniture row, so it is a ratchet and not just a report: re-run it after any change to patterns-sommaire.ts.

Linked council résolutions (--emit-linked-resolutions, DEFAULT OFF)

A Québec City sheet often records the council résolution it produced:

CE-2008-0185
  Résolution(s)
2008-02-06Date:
https://gpddocs.ville.quebec.qc.ca/gpdblob/CE-2008-0185.pdf

That is a genuine, high-value fact — the adopted decision with its real date. It is triple-anchored (the number token, the Résolution(s) label, and the gpdblob/<NUM>.pdf URL must all agree, and a real YYYY-MM-DD must be present); an unfilled block (bare Date:) and a block inside DÉCISION(S) ANTÉRIEURE(S) are both dropped. It is opt-in because the PV corpus can carry the same résolution, so emitting it is a deliberate choice about duplication — never a silent default.

session_date — the order of trust, and nothing else

1. the raw row's own session_date (Québec City: 73,662/73,662 populated, and equal to the linked résolution's adoption date on every sample that carried one);
2. the linked résolution's date (Québec City, triple-anchored);
3. the sheet's own French-month date (01 Février 2008Date :) — the creation date, last resort;
4. NULL.

Tile 513 — the Montréal borough landing-place gap: NO MIGRATION

raw.qc_minutes_documents has no borough column (it does have council_body, but it is NULL on all 469 eligible Montréal PVs and its vocabulary is conseil/comite/commission — a body class, not a borough identity). The question is whether that corrupts. It does not:

1. No false municipality-level fact. Boroughs are administrative divisions of Ville de Montréal, not separate MAMH municipalities. municipality_code='66023' is the correct, true code for a borough-council résolution. What is lost is granularity, not truth.
2. No collision, no overwrite. external_id is doc-scoped (<doc_id>:<number>), so two boroughs that happen to use the same résolution number land as two distinct rows. Nothing silently overwrites anything.
3. The body is recoverable — from the number itself. Montréal numbers every decision with a body prefix: CM conseil municipal, CA conseil d'arrondissement, CE comité exécutif, CG conseil d'agglomération. The body_yy_seq scheme (v5) captures that prefix into resolution_number, so the deciding body — and for CA, the borough digits — derive from the stored value. Derive over store; no new column, no migration, drawer evolves additively.

The real defect behind tile 513 was that the parser did not match Montréal's numbering at all. The keyword fallback caught a bare 02-0333 on the rare line reading "Résolution …", dropping the body prefix and ~99% of the yield: 8 Montréal PVs hold 67–69 line-start anchors each, of which the old extractor found 4 in total. body_yy_seq is the fix, and it is what makes the borough question answerable later without re-parsing.

Accepted v1 posture: borough granularity is DERIVED from resolution_number, not stored. If a consumer ever needs it as a first-class column, it is a downstream generated column over a value we already hold — never a re-harvest.

Honest limits (the ~18% of munis that yield little/nothing)

1. Misclassified raw docs — some doc_type='proces-verbal' rows are actually lab-quality certificates or avis-publics filed under the PV type (e.g. East Angus 41060). The parser correctly yields zero; this is a corpus-quality issue (ZONE 1 Pristine / harvester classifier), not a parser gap.
2. Narrative-style minutes without a per-item number at the margin (résolutions embedded inline) — genuinely low-yield, mostly small munis.
3. OCR-degraded scans — the dry-run sampled text_method='text-layer'; OCR munis parse worse and are excluded from the numbers above.
4. Number-field truncation on compound règlement numbers — a documented v2 fix.

Projection (order-of-magnitude, caveated)

8.9 rés/PV × 66,617 held PVs ≈ ~450k–600k résolution records (lower than the naive product: OCR munis + the misclassified/narrative tail parse below the text-layer sample average). 1.0 règl/PV ≈ **50k–70k règlement events**. Of the 635 PV-covered munis, an estimated ~500–525 gain a populated résolution folder and ~480–500 a règlement folder (from 33/40 and 31/40 in-sample).

Attribution: Municipal council minutes (procès-verbaux), municipality of origin.

apps/worker/libpostal-sidecar/README.md

libpostal HTTP sidecar

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: a Python/FastAPI address-parsing sidecar wrapping libpostal.
• Input/output boundary (Layer-1): HTTP POST /parse — ParseRequest / ParseResponse defined as pydantic models (Python). FLAGGED (boundary table §C): out of Zod scope — pydantic is this unit's schema authority.
• Run locally: docker build then docker run (see body); container serves /parse.
• Deploy: container image (was Railway — retired; runs as a container next to the DB).

FastAPI wrapper around libpostal + pypostal bindings. Deployed on Railway Hobby as a long-running process shared across all Nexod platform source adapters that need address canonicalization.

API

• POST /parse — body { "address": "<string>" } returns { "components": { "road": "...", "house_number": "...", ... } }.
• GET /health — Railway health check endpoint.

Auth: set LIBPOSTAL_API_KEY on Railway; clients pass it in X-API-Key. Without the env var the endpoint accepts any caller (local dev only).

Running locally

cd apps/overwatch-worker/libpostal-sidecar
docker build -t nexod-libpostal .
docker run --rm -p 8080:8080 nexod-libpostal
# Smoke:
curl -X POST http://localhost:8080/parse \
  -H 'content-type: application/json' \
  -d '{"address":"100 boulevard Saint-Laurent Montréal"}'

Expected: status 200 with a components dict containing road, house_number, and city.

Deploying to Railway

1. railway login
2. railway init in this directory and link it to a new service.
3. Add the env var LIBPOSTAL_API_KEY (any random 32+ char string). Keep it matched with LIBPOSTAL_API_KEY configured for the Nexod platform.
4. railway up — the build will compile libpostal from source (~8 minutes first time, cached thereafter). Health check at /health must return 200 before Railway marks the deploy ready.
5. Copy the public domain into LIBPOSTAL_URL on the Nexod dashboard env.

Wiring into the platform

import { createHttpLibpostalClient, canonicalizeAddress } from '@nexod/platform/adapters';

const libpostal = createHttpLibpostalClient({
  baseUrl: process.env.LIBPOSTAL_URL!,
  apiKey: process.env.LIBPOSTAL_API_KEY!,
});

const addr = await canonicalizeAddress('100 boulevard Saint-Laurent', {
  cityKey: 'montreal',
  libpostal,
});
// addr.method === 'libpostal_hybrid' when sidecar reachable
// addr.method === 'rule_fallback' when sidecar down or env vars absent

The client returns null on any network/auth failure so the adapter falls through to rule_fallback cleanly. Sidecar downtime degrades accuracy from ~96% to ~93% on the 50-address fixture; it does NOT break ingestion.

Memory footprint

libpostal loads its ~2 GB trained model into memory on process start. The Railway Hobby plan allocates 8 GB RAM; a single sidecar replica fits comfortably. Cold start is ~5–8 seconds while the model hydrates from disk cache (first start after deploy is slower — the healthcheck timeout is set to 300 s to accommodate this).

apps/worker/lidar-ingester/README.md

@ortova/lidar-ingester — the LiDAR index-drawer loader

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: acquires + stamps the QC LiDAR tile index (the height/point-cloud substrate).
• Input boundary (Layer-1): CLI --i-am-felix / --plan; reads canonical.lidar_tile. No Zod schema (FLAGGED, boundary table §C): env-only input, low shape surface.
• Output: LiDAR index rows → canonical.lidar_tile. DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/lidar-ingester acquire (also stamp-tiles).
• Deploy: gh workflow run lidar-acquire.yml (runs-on ortova-mac).

The metadata loader for migration 0397 (canonical.lidar_acquisition + canonical.lidar_tile

• their raw.* landings). It seeds the index drawer that answers "what LiDAR covers parcel X" — LiDAR mission P0a (docs/ops/2026-07-21-lidar-raw-layer-mission.md). Metadata only. No point-cloud bytes are ever downloaded or stored (INV-0; the drawer carries the fetch URI, never the tile).

The three doors (source_program)

Contract

• --dry-run is the DEFAULT. It parses the source, builds the raw + canonical UPSERT rows, prints the counts + a sample, and writes the full .sql to --out. It writes nothing to the DB — safe to run before migration 0397 is applied.
• --apply --i-am-felix pipes the generated SQL to psql over SUPABASE_POSTGRES_POOLER_URL in keyset batches (idempotent ON CONFLICT on the natural keys). Felix-gated (Rule 6); requires 0397 applied. Sub-agents never run this.
• Idempotent. Natural keys — acquisition (source_program, project_key), tile (source_program, project_key, tile_name) — so a re-run is an UPSERT, never a duplicate.
• After a bulk load: refresh the coverage gauge (Felix-gated): REFRESH MATERIALIZED VIEW CONCURRENTLY serving.lidar_coverage_by_acquisition;

Usage

# federal dry-run (uses/caches the two small index files under --cache-dir):
python3 load_index.py --source federal --dry-run --out /tmp/lidar_federal.sql

# provincial acquisition roster (live WFS), dry-run:
python3 load_index.py --source provincial --dry-run --out /tmp/lidar_provincial.sql

# apply (Felix only, after 0397 applied):
python3 load_index.py --source federal --apply --i-am-felix

Fidelity notes (carried into the drawer)

• Density is two numbers. Federal LDR_AGGREGATE_DENSITY = all-returns aggregate (CMM 2023 = 20.83 pts/m²); provincial DENSITE = nominal spec (CMM = 15). Both stored; density_basis says which. Never compare across bases as equal.
• Same flight, two doors. A CMM tile exists under both federal-copc (COPC) and mrnf-laz (LAZ). The drawer holds both, keyed by (source_program, ...); not collapsed.
• Federal tile footprints live in the 407 MB Index_LiDARtiles_tuileslidar.gpkg (over the metadata fence) — federal tile rows land with geom NULL; the acquisition footprint carries coverage. Full provincial per-tile footprint ingest is the lidar-acquire front's bulk job.

P1 ACQUIRE — acquire.ts (federal COPC → R2)

Tile 467 (Felix, 2026-07-21): R2 approved (~$15/mo metro tier). acquire.ts copies federal COPC tiles from the free anon S3 bucket into our R2 bucket, byte-for-byte (no format conversion — federal tiles are already COPC). Provincial (LAZ, not COPC) needs an untwine/PDAL convert step first — the seam for that is documented in acquire.ts's header but not built; only federal-copc tiles are acquired today.

Key layout (forever)

lidar/<source_program>/<project_key>/<basename-of-source_uri>
e.g. lidar/federal-copc/600023_52_CMM_2024/24_2495039f08_dc.copc.laz

source_program first (a CMM tile exists as both a federal-copc COPC row and an mrnf-laz LAZ row — the prefix keeps the two doors from ever colliding), project_key next (mirrors the federal bucket's own layout 1:1 — eyeball-mappable, prefix-listable per acquisition), tile filename last, unchanged. Full justification in acquire.ts's header comment.

Usage

# read-only plan (DB read + R2 HEAD only, no transfer):
tsx acquire.ts --project 600023_52_CMM_2024 --limit 10 --plan

# execute (Felix-gated — an R2 PUT is an outbound spend, Rule 6):
tsx acquire.ts --project 600023_52_CMM_2024 --limit 10 --i-am-felix

# bypass the DB (proof/testing against tiles a still-loading drawer doesn't hold yet):
tsx acquire.ts --manual /tmp/tiles.json --i-am-felix

# stage (never apply) the canonical.lidar_tile UPDATE that stamps the ledger's outcome:
tsx stage-db-stamp.ts --out /tmp/lidar-tile-stamp.sql

Resumable (a HEAD check against R2 skips a tile already landed at the expected size), paced at ≤3 concurrent transfers (hard cap in acquire.ts), per-tile ledger at .acquire-ledger.json (gitignored, runtime state).

Proof (2026-07-21, 3 CMM_2024 tiles, R2 bucket ortova-geo)

All three confirmed present in R2 at the exact source byte count (aws s3 ls against the R2 endpoint). HTTP range-read proof (the actual point of landing COPC in our own bucket — a presigned GET against the R2-hosted object, Range: bytes=0-1023 and a mid-file Range: bytes=100000-100511):

{ "status": 206, "contentRange": "bytes 0-1023/2867985", "bodyBytes": 1024 }
{ "status": 206, "contentRange": "bytes 100000-100511/190510", "bodyBytes": 512 }

Real 206 Partial Content + correct Content-Range, confirming R2 serves COPC octree range-reads without any public-read/custom-domain change — a presigned SigV4 GET (via the aws CLI's own s3 presign, not a new SDK dependency — see lib/r2.ts's header) is enough.

The one DB write this pipeline stages (never applies)

acquire.ts never touches canonical.lidar_tile. stage-db-stamp.ts reads the local ledger and writes an idempotent UPDATE ... SET our_store_uri, checksum, ingest_status='acquired' statement per acquired/verified tile to a .sql file for the release lane to review and run separately (Rule 6 — DB writes are Felix-gated).

Bulk numbers (2026-07-21 measurement + projection)

Measured this session (single-stream, this laptop, under heavy load tonight — not representative of a clean environment): 57.87 MB in 100.2 s ≈ 0.58 MB/s; a smaller 11.3 MB tile ran at 0.38 MB/s (fixed per-tile overhead dominates on small files). At the pipeline's 3-concurrency cap, and assuming roughly linear scaling, call it ~1.5–1.7 MB/s aggregate on this machine tonight.

Recommendation: run the bulk transfer cloud-side via GitHub Actions, not the self-hosted ortova-mac runner. The ortova-mac runner is this same physical laptop (.env.local access is its only advantage) — exactly the machine measured as bandwidth/load-constrained tonight. A hosted GH Actions runner has no direct .env.local, but the only credentials this pipeline needs (R2_ENDPOINT / R2_ACCESS_KEY_ID / R2_SECRET_ACCESS_KEY / R2_BUCKET) are easy to hold as encrypted repo secrets — the federal S3 read is anonymous, no credentials at all. Per docs/architecture/cloud-execution-paths.md, GH Actions is the default batch-geo-ingest path. Built (2026-07-21, tile 467 follow-up): --tier cmm-metro --shard <i>/<m> on acquire.ts (deterministic tiles[i] -> shard i % m partitioning, lib/shard.ts — geo-trinity's release-sync.ts/lib/na-grid.ts precedent mirrored verbatim; fixture-tested in lib/shard.test.ts against a synthetic 4,920-tile set: every shard 0..count-1 disjoint, union = full set, stable re-computation, both even (4920/8) and uneven (4920/7) splits proven) + .github/workflows/ lidar-acquire.yml (workflow_dispatch, inputs tier/shard_index/shard_total).

Per-shard wall time (8 shards, cmm-metro tier)

4,920 tiles / 1.17 TB (exact, measured) ÷ 8 shards = 615 tiles/shard, ≈146.5 GB/shard. At the README's own conservative cloud-runner rate (30 MB/s aggregate, 3-way concurrency, ~10 MB/s/stream — the same number the "Bulk numbers" table above used for the full-tier 11-hour estimate):

146,488.6 MB / 30 MB/s = 4,883 s ≈ 81 minutes/shard

Comfortably inside the workflow's 200-minute hard cap (the spend ceiling by construction — GitHub Actions bills per runner-minute) with wide margin for real-world variance (S3 throttling, transient retries, per-tile fixed overhead on the smaller tiles). Total wall time for all 8 shards run in parallel: ≈81-100 minutes (not 8x serial — every shard is an independent job). Total GitHub Actions runner-minutes billed: ≈8 × 90 min ≈ 720 min ≈ $5.76 at the $0.008/min Linux overage rate (cloud-execution-paths.md) — comfortably inside the $15 cap Felix approved (tile 467), with margin for a slower-than-projected real run or a retry.

Fire commands (the Release Agent runs these on Felix's go — NOT fired by this drop)

for i in 0 1 2 3 4 5 6 7; do
  gh workflow run lidar-acquire.yml -f tier=cmm-metro -f shard_index=$i -f shard_total=8
done
gh run watch   # tail the first shard or two before trusting the projection above

After all 8 shards land (watch gh run list --workflow=lidar-acquire.yml for 8 green runs), reconcile the DB against the ACTUAL R2 listing with the stamp loader (below) — never a hand-assembled SQL file from the 8 scattered per-shard ledger artifacts.

The STAMP LOADER — stamp-tiles.ts (production path, supersedes stage-db-stamp.ts)

The durable fix for the one-off-SQL-file pattern: stage-db-stamp.ts reads ONE local .acquire-ledger.json, which a sharded cloud dispatch scatters across 8 separate runners (each uploading its own ledger as a run artifact) — reassembling those by hand before every stamp is the exact ceremony this loader kills. stamp-tiles.ts instead reconciles canonical.lidar_tile against the ACTUAL R2 bucket listing (ListObjectsV2, ground truth) — no ledger needed at all, from any shard, any manual run, any future re-run.

# read-only: DB read + R2 listing only, prints exact would-stamp / already-correct / not-in-r2 counts
tsx stamp-tiles.ts --dry-run

# apply (Felix-gated — UPDATE canonical.lidar_tile, Rule 6):
tsx stamp-tiles.ts --apply --i-am-felix

# scope the DB read to one tier (default is unscoped — cheap even so, one SELECT):
tsx stamp-tiles.ts --tier cmm-metro --dry-run

Batch-committed (500 rows/transaction), natural-key UPDATE (idempotent — a re-run over already-correct rows is a no-op). Proof against prod (2026-07-21, before any bulk shard has run): already-correct=5 would-stamp=0 not-in-r2=34994 — the exact 3 proof tiles from the table above plus 2 more from an earlier local proof run, all already correctly stamped by stage-db-stamp.ts; zero new stamps until the bulk shards actually land tiles in R2.

apps/worker/minutes-harvester/README.md

minutes-harvester — Stage 2 of the QC meeting-minutes lane

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: harvests QC municipal meeting-minutes PDFs into raw landing (the live Watch front).
• Input boundary (Layer-1): WorkflowArgsSchemas['minutes-ingest'] = { municipality } and WorkflowArgsSchemas['minutes-discovery'] = { codes } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema on the queue.
• Output: raw.qc_minutes_* rows. DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/minutes-harvester plan / ingest / local.
• Deploy: gh workflow run minutes-ingest.yml -f municipality=<code> (runs-on ortova-mac).

Pulls every council-minutes / avis-public / permit-bulletin PDF a QC municipality publishes on its own open web archive, and stores the bytes verbatim in durable object storage, registering each one in raw.qc_minutes_documents (migration 0244_qc_minutes_documents.sql, DRAFT, Felix-gated apply).

This worker does NOT extract. It does not itemize dérogation-mineure / PIIA / démolition records, does not resolve an address to property_id, and does not write any canonical satellite. That is Stage 3 — future SPINE work (see "Path to Stage 3" below). Felix's directive for Stage 2 is explicit: no segmentation, pull every municipality, full history, store verbatim, stop there.

• Decision it informs: is the council-minutes lane's supply real at full-history scale (not just the 1-2-document Stage-1 census sample)?
• Action it unblocks: a Stage 3 extraction pass (dérogation-mineure / PIIA / démolition itemization → a canonical satellite) has a durable, verbatim corpus to run against instead of re-fetching the web on every experiment.

Census this worker operationalizes: docs/research/qc-minutes-lane/2026-07-13-qc-meeting-minutes-census.md (35 no-open-permit-feed municipalities + the Gatineau permit-bulletin bonus).

The four permit-feed cities (2026-07-27, tile 503)

The registry is no longer the 35-no-feed-town list this README was written against. Montréal, Laval, Longueuil, and Québec City are all registered, and tile 503 closed the coverage holes inside them. Measured --plan, 2026-07-27:

Longueuil was previously recorded as permanently robots-blocked. It is not: www3.longueuil.quebec serves Disallow: /, but longueuil.quebec and cms.longueuil.quebec (where the 2021-to-present PDFs actually live) both serve a narrow Drupal robots.txt with no blanket disallow. Pre-2021 archives and both comité exécutif streams do stay on the blocked host.

Montréal must be SHARDED — read this before dispatching 66023 (2026-07-27)

Montréal died on every harvest attempt (three failed runs 2026-07-21/22, a 900 s fleet kill, a 3,600 s fleet kill that registered zero). None of it was montreal.ca blocking us. The hour is discovery, and discovery is all-or-nothing — nothing is acquired until every seed is walked, so a run killed mid-discovery writes exactly nothing.

Measured 2026-07-27:

So: shard by archive year and give the lane a clock. A single year is ~45 s of discovery plus ~700 docs × 1.4 s ≈ 18 min — comfortable inside a 3,600 s lane. Both knobs are workflow_dispatch inputs on minutes-ingest.yml (index_filter, deadline_min) and env vars locally (MINUTES_INDEX_FILTER, MINUTES_DEADLINE_MIN):

# plan one shard (read-only, always safe)
MUNICIPALITY=66023 MINUTES_INDEX_FILTER='dateDebut=2024' npx tsx ingest.ts --plan

# the real lane, one year, Felix-gated
gh workflow run minutes-ingest.yml \
  -f municipality=66023 -f mode=i-am-felix \
  -f index_filter='dateDebut=2024' -f deadline_min=50

Re-firing the same shard is always safe (INV-4 resume), so a year that does not finish just needs another dispatch.

The 117 blocked_at rows for 66023 are FALSE blocks. A polite live re-probe returned HTTP 206 / application/pdf / %PDF on 10 of 10 sampled — every one of those documents is servable right now. They were poisoned by a run-scoped host block cascading into per-document blocks (fixed: see isHostScopedFailure in ingest.ts), and every row has a NULL last_error because recordDocFailure never wrote the column (also fixed). Re-audit any city's ledger with MUNICIPALITY_CODE=<code> npx tsx probe-blocked-docs.ts. Clearing the rows is a DB write and stays Felix-gated — until they are cleared, those 117 documents stay skipped on every run.

Laval's 593 "shortfall" is phantom — do not harvest it (2026-07-27)

Laval's child exits 0 in 27 s with discovered=3,659, acquired=0 and zero blocked rows, and the scoreboard calls that a shortfall of 593. It is a measurement artifact. The scoreboard computes discovered - count(rows), but the harvester is content-addressed: a document republished at a second URL appends to also_seen_at_urls instead of inserting a row. Measured:

discovered 3,659 | held rows 3,066 | alias URLs 699 | known URL set 3,765
naive shortfall 593  ->  GENUINELY MISSING 0

Laval's archive is complete. Reproduce with npx tsx verify-alias-shortfall.ts. The fix belongs in the scoreboard: a municipality's held URL set is source_url UNION unnest(also_seen_at_urls), never count(*).

These cities' documents are harvested but NOT parsed -- raw.qc_governance_resolutions holds 8 rows for Laval and 0 for the other three against 81,134 held documents. The decision lane for the feed cities is waiting on the governance-parser, not on this worker. Full return: docs/product/qc/verification/2026-07-27-tile-503-feed-city-harvest.md.

THE ADAPTER CONFORMANCE HARNESS (conformance/) — read before touching any adapter

Every harvest adapter is run against ONE contract, offline and deterministic: npx vitest run conformance. Readiness verdict + the hardening/stress backlog: docs/product/qc/2026-07-30-adapter-conformance-readiness.md — read its §7 first, it is what is true now; §1–§5 are the gate report as written before the hardening pass and three of its claims are superseded there.

Two files, two jobs:

• conformance.test.ts — the five clauses (selection / three numbers / idempotence / constraints / degradation) over 7 enumerator façades, including all four index-recipe recipes.
• stress.test.ts — how the adapters behave when the far end MISBEHAVES: oversized and endless bodies, injected latency against the per-target budget, soft-404s, redirect loops, contradictory pagination, adversarial payloads and filenames, the five spellings of «procès-verbal», and concurrency (per-host serialization must hold under our own parallelism). Also offline; also impossible to point at a real host.

Two bounds every new adapter inherits — use them, do not re-invent them:

• body-cap.ts (readCapped) for every response body. Never res.text(), and never Buffer.from(await res.arrayBuffer()).subarray(0, MAX) — that reads the whole body first and caps nothing. readCapped streams, stops at the ceiling and cancels, so the rest never leaves the origin. Truncation is reported; a half-read list must never read as a short one.
• budget.ts (TargetBudget) for the per-target wall clock. Cooperative — checked at every loop head and before every network call — because a Promise.race unblocks the caller while the work keeps running, which on a fleet box is a leak rather than a bound. A spent budget is a partial result flagged partial, never a throw: throwing discards everything found before the deadline and turns a slow target into a zero.

The recurring defect of this corpus, stated once: a label is EVIDENCE, never a GATE. Wherever a page title, a section label or a list name decides the fate of a whole COLLECTION, the failure is total and silent — that is what lost Saint-Adrien's 632 files (a page called «Document») and what still hid neural sections a clerk named «Documents» until 2026-07-30. The fix shape is always the same: demote, do not skip — judge each file on its own name (looksStrictlyLikeMinutesFile), and bound how many demoted collections you read, because reading them costs a town requests. All three known occurrences (vplus page title, neural section label, weblex sitemap list name) are closed. The shape to watch for in any NEW adapter: a continue or early return whose condition tests a CONTAINER's name.

Adding an adapter = adding one façade object to ENUMERATORS (or a router case in routers.test.ts). The clauses come free. That is the whole point — "all adapters are correct" is only a checkable claim if all adapters answer the same questions; N bespoke test files make it an aspiration, because the clause nobody wrote for adapter #7 is exactly the one that will bite.

Four things to inherit rather than re-derive:

1. The network is mocked at globalThis.fetch, NOT at fetch-polite.js. tolerantFetch is the only place this worker touches the network, so mocking there runs the REAL polite stack under test: robots really fetched/parsed/applied, aiBotRestrictions really ANDed, RFC 9309 §2.3.1.2/§2.3.1.3 really enforced, Crawl-delay and per-host serialization really pacing. Mocking fetch-polite — the easier choice — proves selection while proving nothing about whether we fetch what we must not. Do not "simplify" it back.
2. An unrouted URL throws. Not a 404, not a passthrough. It also makes a real request to a real municipal server structurally impossible from the suite.
3. A selection trap is about the doc_type, not the fetch. Reg_873_Tenue_seances_conseil_web.pdf (a by-law about meetings) is legitimately banked — the corpus deliberately holds 19 document kinds — and must never be TYPED proces-verbal. Assert at the doc_type the row would carry; "must not select" is both weaker and wrong.
4. An adapter that cannot satisfy a clause DECLARES it (EnumeratorAdapter.cannot) and the suite asserts the declaration exists. Never weaken an assertion to make it pass — that is the one failure mode that makes the whole exercise worthless. cannot is a debt marker, not a resting state: all three declarations the harness shipped with were retired within a day by fixing the adapters (weblex and neural now RETURN their three numbers; WalkResult now counts examined). If you add one, it belongs in the backlog the same commit.
5. Pin ratchets to the INVARIANT, not to a constant. Two clauses here asserted "vplus has no adapter" and went red the day somebody built one. A ratchet whose only failure mode is "the work got done" teaches the next agent to delete ratchets. Both were re-pinned to "a vendor may claim a route iff an adapter is dispatched for it", read off ingest.ts so there is no second copy to drift.

WE NEVER STRESS-TEST REAL MUNICIPAL SERVERS. Small-town web hosts, often one shared box per region. Stress means local fixtures, synthetic payloads, injected latency/truncation and adversarial parser inputs — offline, in conformance/. Same standing order that makes robots binding.

Run modes

# read-only dry-run: discover candidate PDFs + resume-check against the DB,
# print counts. NO fetch of document bytes, NO writes anywhere. Always safe.
MUNICIPALITY=Beloeil npx tsx ingest.ts --plan

# proof mode: fetch + hash + OCR + store to LOCAL DISK (OUT_DIR), write a
# local manifest.json. NO Blob upload, NO DB write. This is how the harvester
# is verified end-to-end without a Felix-gated write.
MUNICIPALITY=Beloeil OUT_DIR=./out npx tsx ingest.ts --local-only

# the real run: R2 upload + raw.qc_minutes_documents INSERT. Felix-gated —
# the Release Agent runs this (via .github/workflows/minutes-ingest.yml),
# never a SCALE agent against prod.
# (BLOB_READ_WRITE_TOKEN is no longer needed to WRITE — since 2026-07-31 the
#  store of record is R2 — but keep it set while the backfill runs: it is what
#  lets the dual-read seam still resolve a not-yet-copied object.)
MUNICIPALITY=ALL SUPABASE_POSTGRES_POOLER_URL=... \
  R2_ENDPOINT=... R2_ACCESS_KEY_ID=... R2_SECRET_ACCESS_KEY=... R2_BUCKET=... \
  BLOB_READ_WRITE_TOKEN=... \
  npx tsx ingest.ts --i-am-felix

MUNICIPALITY accepts a municipality name (Beloeil, Vaudreuil-Dorion) or its MAMH code (57040), or ALL to run every configured entry in the registry in one process (fine for --plan; for --i-am-felix the sanctioned path is one municipality per GitHub Actions dispatch — see below — so a bad regex on one town fails small, not the whole province).

WATCHING A PULL — what the database can and cannot see MID-RUN (2026-07-31)

Read this before building anything that claims to show a harvest live, and before believing a surface that does. The harvester writes to the spine at three moments:

1. START time (new — migration 0495, tile 544) — one raw.qc_minutes_harvest_inflight row per (target, run), INSERTed by recordInflightStart() when a child begins real work on a target, plus one phase stamp at the discovery→acquisition transition and one closing UPDATE at the end. i-am-felix only. Carries started_at, phase, phase_at, finished_at, run_id, and the entity_id identity anchor. It deliberately carries no counts.
2. BANK time — raw.qc_minutes_documents rows, flushed per batch (not per document), carrying municipality_code + fetched_at. This is the only continuous progress signal, and it lags the actual fetch by up to one batch.
3. SETTLE time — one raw.qc_minutes_harvest_state row per municipality, written by recordHarvestState() when that municipality's child process finishes, and only in i-am-felix mode (ingest.ts, if (m === 'felix')). It carries last_discovered_count (what discovery found) and consecutive_empty_discoveries.

What (1) bought. A target with an OPEN in-flight row has a reported phase, a true per-target wall time measured from its own start stamp, and a run it can be attributed to. That is the whole of what changed — every claim below still holds for every target without such a row.

What is still NOT observable — state these rather than papering over them:

• Phase is reported only where an in-flight row exists. Everywhere else (a --plan sweep, a target the harvester never reached, anything before 0495 applied) it is still derived from observed writes, and a target enumerating a deep archive with nothing banked yet is still indistinguishable from one that has not started.
• An open row cannot tell a working child from a DEAD one. Nothing back-fills finished_at, on purpose — inventing a finish for work that never reported back is the confident-label-for-an-absence error. Read a long-open row with an old phase_at as "started and has not reported back", never as proof of life.
• elapsed and age are different quantities and are never merged. elapsed is the true wall time from the start stamp; age is time since the last observed write. A surface showing one under the other's heading is lying.
• "Documents found" is unknown until settle, and never in --plan. Mid-run it is unknown, never zero. Unchanged by 0495: the in-flight table stores no count, because a copy of harvest_state's answer would be a second answer (same-question-columns).
• discovering does not mean "nothing downloaded". Acquisition streams inside discovery (2026-07-28), so the phase means the enumeration is unfinished.
• Run attribution holds only for a gha- run id. runId is gha-<GITHUB_RUN_ID> under GitHub Actions and local-<epoch_ms> on a laptop/fleet box (ingest.ts main()); the latter is per-process and joins to nothing. (Correction, 2026-07-31: this section previously stated flatly that last_run_id is "a per-process id (local-<epoch_ms>), not the GitHub run id". That is only true for a child running OFF Actions — the gha- branch has always existed. The underlying caution was still right and still applies: measured 2026-07-31, a REGISTRY-ALL mode=plan run was in flight while harvest_state settled seven targets from a different, felix-mode process, so joining runs to settles by time would have credited the plan run with work it did not do. Join on run_id, or not at all.)

REGISTRY-ALL matters here too: a province-wide dispatch is one workflow run covering every eligible target, so the GH runs list gives one row for the whole pull, not one per municipality. Per-municipality dispatches do render their code in display_title (minutes-ingest 57020 mode=i-am-felix) — see scripts/ops/fleet.ts on why display_title is the only real-time input channel the GH API exposes.

The surface that consumes all of this is the QC pull panel on /monitor (scripts/ops/monitor/pull.ts → /monitor/pull → qc-pull-panel.client.tsx) and its terminal twin pnpm monitor:pull. Its NOT_OBSERVABLE list is these gaps, rendered on the page — and it is feature-detected: NOT_OBSERVABLE_PRE_INFLIGHT is served while raw.qc_minutes_harvest_inflight is absent, so the panel never claims liveness the schema cannot back. If you extend the harvester's reporting again, shrink that list in the same change — a stale honesty note is worse than none. Reader test (both schema states, offline): pnpm test:pull-reader. Writer tests: inflight-sql.test.ts.

Do not add a chattier stamp. Three psql spawns per TARGET is the whole budget; a per-document or per-batch write would cost more than the panel (refreshing every 3-10 s) can use, on a fleet that runs a process per municipality.

Denominators, derived live from serving.qc_target_state (measured 2026-07-31, quoted so a reader can spot drift, not so anyone hardcodes them): 900 eligible municipalities + 80 eligible bodies; 124 municipality + 15 body locations terminal by rule (robots-disallowed incl. AI-bot-disallowed, or auth-walled) and therefore excluded by design, not remaining work. Corpus 272,324 documents. advertised_count sums to 358,350 over 342 of 980 eligible locations and is a ceiling — what sites claim, not what is available and not what is minutes.

TWO TARGETS, ONE URL — the collision that banks bytes and no rows (2026-07-31)

Symptom, and it will fool you. A town harvests, ~47 objects appear in R2 under qc-minutes/<code>/, raw.qc_minutes_harvest_state settles with a real last_discovered_count, the run prints ✓ and exits 0 — and raw.qc_minutes_documents has ZERO rows for that code, indefinitely. It reads exactly like a broken write path. It is not. Measured on Senneterre 89045, 2026-07-31.

The mechanism. content_hash is UNIQUE corpus-wide, so a document belongs to exactly ONE municipality_code: the first target that banked it. The registry held TWO eligible targets named Senneterre — 89040 and 89045 — carrying the same minutes_url. 89040 banked 26 documents on 2026-07-19/20. 89045 then discovers the same 25 PDFs at the same URL, downloads them, writes them to R2 under its own content-addressed key, and every INSERT lands on 89040's row as an ON CONFLICT no-op. Bytes paid for, objects orphaned, corpus unmoved.

Why it survived so long. registrationLostDocs deliberately did not fire on upsertTouched > 0 — "a content-hash re-assert is real, correct work". True of a re-assert against our own row, false against another target's, and the RETURNING clause could not tell them apart. Also note alreadyKnown is 0 every run (the resume check keys on municipality_code, and the rows live under the other code), so the town looks permanently behind and re-downloads its whole archive every beat.

Now mechanical. RETURNING carries the owning municipality_code; tallyRegistrationRows has THREE outcomes (registered / upsertTouched / crossTargetCollisions); the run logs both codes, calls the objects ORPHANED, and exits 1 instead of settling clean. Pinned in cross-target-collision.test.ts.

Blast radius is small — do NOT let it hold the pull. 6 contested locations, 15 eligible targets, 9 that can never bank. Find them with (group on the GENERATED location_key, not the raw URL — two URLs differing only by scheme or trailing slash are ONE location to the PK and to the harvester; measured 2026-07-31 the two groupings agree today, so this is fragility avoided, not a bug fixed):

SELECT s.location_key, string_agg(DISTINCT s.target_code, ',' ORDER BY s.target_code)
FROM serving.qc_target_state s
WHERE s.harvest_eligible AND s.location_key <> '(none)'
GROUP BY 1 HAVING count(DISTINCT s.target_code) > 1;

Ratcheted 2026-07-31 as pnpm audit:data's qc-minutes-location-owned-twice (red at 6, green at 0 once the repair lands), and the ownership repair is migration 0497 with its measurement set in docs/architecture/2026-07-31-collision-repair-queries/. Four things that pass measured this session and will otherwise be re-derived:

1. There are EIGHT groups, not six. Two more are LATENT — zero eligible targets, so they burn nothing and the query above cannot see them: 4 Sherbrooke arrondissement bodies on contenu.maruche.ca (a bare vendor-CDN ORIGIN, not a publication location for any of them) and 3 Lévis arrondissement bodies on a shared /arrondissements/ index. They collide the instant anything makes one eligible. Drop the WHERE s.harvest_eligible to see them.
2. The registry's own generated harvest_eligible column DISAGREES with serving.qc_target_state — it keys on live_proof_outcome, one of the 13 columns 0474 marked DEPRECATED. Measured: it reports 0 eligible for the Mont-Tremblant and Saguenay groups that the view (and therefore the fleet) reports as eligible. The view is the live invariant, because the Watch front enumerates on it. Never write a guard or a partial index against the registry column — it would certify the bug.
3. Clearing a loser's minutes_url does NOT, by itself, stop it fetching that URL. deriveRegistryConfig() returns undefined when a target has no harvestable location, and resolveConfig() then FALLS THROUGH to the legacy tiers — which still hold the same URL: serving.qc_minutes_coverage_census.minutes_url carries the colliding URL for all five municipal losers, body-targets.json for agg-cookshire-eaton, discovered-archives.json for 84082 and 89045. What saves the nightly drive is only that the front enumerates eligible-only, so a cleared target is never dispatched; a direct tsx ingest.ts <code> run still resolves the shared URL. This is the 2026-07-29 "the seven stores are INPUTS, not answers" debt, unfinished: a registry saying this target has no location must be a TERMINAL answer, not silence that the lower tiers fill in. Fixing it is a resolver change, not a migration.
4. A registry repair changes nothing on the fleet until the projection is regenerated. target-registry.json is FILE-FIRST for resolution and ships to every box via git archive HEAD, so after any registry migration run tsx discover-targets.ts --project-from-db and commit the result.

Ownership rule used, in precedence order (0497's header carries the per-group evidence): (a) if the URL PATH names one of the contenders, it owns the location — a path is first-party evidence, and it outranks first-banker because first-banker would otherwise enshrine a misattribution forever (this is why Matapédia 06045 owns /citoyens/matapedia/ even though 06060 banked 8 documents there first); (b) otherwise the CURRENT DOCUMENT OWNER wins, since the corpus already encodes that answer; (c) a tie on both is an open question, never a silent choice. A shared regional portal (pontiacouest.ca, matapedialesplateaux.com) gives each municipality its own path, so a neighbour pointed at another town's path is a MAP DEFECT, not a co-owner — and the documents it banked are that other town's, misfiled under its code.

The durable repair is in the target map — ONE target owns a publication location — not in the schema. Re-banking is idempotent by construction: the object key is a total function of (municipality_code, content_hash), so once ownership is settled the next harvest re-derives the identical key, R2 HEADs it, sees the bytes, and skips the upload.

THE FRONT AND THE HARVESTER MUST READ ONE UNIVERSE (2026-07-31)

scripts/ops/watch/fronts/minutes-harvest.ts enumerated from serving.qc_minutes_coverage_census (1,098 rows) and never read serving.qc_target_state, while ingest.ts resolves findRegistryConfig() FIRST. Two universes. Consequence: the eligibility jump of 2026-07-31 (76 → 547 → 847 → 900 municipalities + 80 bodies) moved the front's unit set by exactly zero units, and all 80 eligible bodies have no census row at all, so they were structurally un-enumerable — no eligibility change could ever surface them. The census classified 792 done / 299 blocked, leaving 7 pending, and 7 is what the beat ground on, forever.

Second, smaller precedence bug in the same file: EMPTY_STREAK_BLOCK_THRESHOLD = 3 existed but sat ~35 lines below the disc?.status === 'resolved' short-circuit, so a one-shot resolved verdict in discovered-archives.json outranked 576 consecutive live observations of zero (17030 Sainte-Perpétue). 120 of 434 discovery entries are resolved — a class, not a one-off.

The rule both fixes encode: newer, repeated, MEASURED evidence outranks an older one-shot survey. Live before → after (pnpm watch status minutes-harvest): 835/1098 PENDING 7 BLOCKED 299 → 835/1178 PENDING 174 BLOCKED 169.

If the fleet looks busy but the corpus is flat, check PENDING first. A front can be DRIVING and productive-looking while re-firing a handful of permanently-stuck units: every one of the 7 was stuck for a different reason (cross-target collision; a host-scoped fetch failure halting acquisition at doc 1; a 576-deep empty streak; two capped at 100 docs). registered summed to 0 across all of them and the corpus gained zero rows in 24h while the log showed constant activity.

Deep discovery (MINUTES_DEEP=1 / seed.deep) — the has_gaps frontier

The seed model (index_urls + include, one follow hop) only reaches what a seed's index page directly lists. The has_gaps class (443 munis) holds most of its missing procès-verbaux one/two hops deeper on the LIVE site — behind pagination, per-year archive child pages, "+ Archives" expander links, and JS accordions. deep-crawl.ts recovers them: a recursive, bounded, archive-aware crawl that reuses the SAME polite fetch (fetch-polite.ts) and the SAME include/exclude selection — a strict superset of the shallow path, so it can only ADD reach. Turn it on per run (MINUTES_DEEP=1) or per seed (deep: true); off by default the shallow path is byte-for-byte unchanged.

# read-only proof of the discovery delta (shallow vs deep) on has_gaps munis:
MINUTES_CODES="34007 66112 63060" npx tsx prototypes/deep-proof.ts
# a real deep harvest of one muni (writes gated exactly as today):
MUNICIPALITY=<code> MINUTES_DEEP=1 npx tsx ingest.ts --i-am-felix

Design + Firecrawl verdict + measured deltas (Neuville 20→170, Baie-D'Urfé 118→277, Sainte-Julienne 206→543 = 648 live docs recovered): docs/ops/2026-07-28-deep-discovery-design.md. Bounds: deepMaxDepth (3), deepMaxPages (300, floored at the seed's follow ceiling), same-host, minutes-section-scoped, cycle-detected. The reconciliation driver (scripts/ops/hetzner/reconcile-drive.py) should route has_gaps to a MINUTES_DEEP=1 harvest and shrink wayback to the CDX-confirmed-absent subset (design doc §6).

Prior-art architecture research + the three measured defects (2026-07-30)

Full findings, with citations: docs/research/2026-07-30-municipal-crawl-architecture-research.md (juriscraper / city-scrapers / Open States / Councilmatic read from source; QC vendor fingerprint scan of 80 live sites; ranked generic techniques). Read it before reworking discovery. The four things a future agent must NOT re-derive:

1. The gap is SHALLOWNESS, not darkness. Measured on raw.qc_minutes_documents 2026-07-29: 272,322 docs / 942 munis, but 356 munis hold <100 docs covering an average of 3.4 distinct years, against 200–500 docs / 10–20 years for a genuinely reconstructed archive. 156 munis are at zero. So the not-reconstructed count is **510, not ~220**. Deep discovery (above) is the highest-value lever in the harvester and is still default OFF.
2. wordpress-pdf averages 26 docs/muni across 146 munis — it is scraping a landing page, not an archive. /wp-json/wp/v2/media?mime_type=application/pdf&per_page=100 is a bulk document API with the site's own total in the X-WP-Total header, needs no auth, and agents have already verified by hand that it works on our municipalities (see the manual wp-json comments in municipalities.ts around lines 796, 2038, 2047, 2065, 2066, 2074, 2455). It has never been automated — discover-archive.ts:820 uses wp-json only as a labelling regex. Biggest unexploited unlock in this worker. Trap: 32% of sampled QC municipal hosts return HTTP 200 for any path, including /wp-json/wp/v2/types. A 200 there is NOT evidence of WordPress. Require wp-content or a <meta name="generator" content="WordPress"> tag and a body that parses as the expected JSON.
3. serving.qc_minutes_coverage_census.platform conflates two orthogonal axes and is hiding WordPress sites. Measured: all 7 sampled munis labelled accescite-voila were WordPress underneath. AccèsCité/Voilà is a PG Solutions citizen-services module bolted onto an ordinary CMS, not a CMS. The column needs splitting into (a) base CMS × (b) bolt-on portal; until then 173 munis are being denied the WordPress path. Plausible WordPress population is ~568 of 1,098, not 395. Also note 517 munis (self-hosted-cms-pdf 238 + other 216 + unknown 63) carry no vendor identification at all — and 163 of the 231 zero-doc munis sit in other.
4. Wayback is secondary, as the deep-discovery design already argued. Measured: it contributed 33,758 docs across 377 munis but reaches further back than the live adapters for only 106 of those 377 (28%). Keep it narrowed to the CDX-confirmed-absent subset; do not widen the sweep.

Two operational mechanisms every comparable project has and we do not (both cheap, both in the doc): a run yielding zero docs for a body that should have docs is a hard FAILING state with a dated stamp (city-scrapers status.py: item_count == 0 ⇒ FAILING), and every run diffs its parsed count against the source's own declared total (X-WP-Total, "1 à 20 de 347 résultats", a paginator's last page) — a completeness oracle needing no ground truth of ours.

The DARK municipalities — 231, and how to reach them (2026-07-30)

Full write-up + machine-readable artifact: docs/product/qc/2026-07-30-dark-municipality-access.{md,json}. Tool: probe-dark-access.ts (proves a route WITHOUT banking a document; --robots-audit re-adjudicates permission from robots.txt itself). Read these five before touching the dark set:

1. It is 231, and you must ANTI-JOIN — never subtract two counts. 1098 - count(distinct municipality_code) gives ~156 and is WRONG: 75 of the 942 distinct municipality_code values in raw.qc_minutes_documents are MRC/CMQ body codes parked in the municipality column. Documents cover 867 municipalities + 75 bodies. Always: kind='municipality' in the registry AND NOT EXISTS (…).

2. robots-terminal in the registry is largely FICTION — re-adjudicate before believing it. fetch-polite.ts fails closed on an unreadable robots.txt (5xx/timeout → DENY_ALL → the same RobotsDisallowedError a real Disallow: / raises). Correct to crawl on, wrong to record. Two passes over the same 52 municipalities minutes apart disagreed on 13. The registry claimed 38 dark robots-blocked; the files say 12. And robots-terminal is the one state nothing retries, so the error never self-corrects. Same class as the 2026-07-29 phantom frontier, opposite direction. Use probeHostRobots (robots-verdict.ts) — it returns disallowed | allowed | unreachable, and ONLY disallowed is terminal.

3. AI-crawler intent binds us, and now it is ENFORCED. Our token is ortovabot, so a site whose only statement about AI crawlers is User-agent: ClaudeBot / Disallow: / fell through to * and we would have crawled it. robots.ts's AI_BOT_TOKENS/aiBotRestrictions is now ANDed into isAllowedByRobots — disallow-only, named-tokens-only, so it can only make us crawl LESS and never re-attributes the * group to a crawler the site never mentioned. Measured: 10 dark municipalities are terminal on this basis alone, 9 of them MRC Abitibi-Ouest.

4. A vendor API the fingerprinter CANNOT see is the recurring unlock shape. findEmbeddedApiCandidates is same-origin-only and requires an /api/ path segment, so every vendor whose API is on a shared vendor host is invisible to it — that is why c-vendor-json = 0 across all 1,205 targets. Two found so far, both this shape:

VPlus pages fetch as a ~2.6 KB Angular shell with zero document anchors, which is why they fingerprint base CMS: unidentified and every plain-fetch pass found nothing. Documents live in vplus-documents.s3.ca-central-1.amazonaws.com. Remaining step: the per-municipality tenant handle, minified as apiUrl:uo.U in main.*.js. When a cohort fingerprints "unidentified" en masse, look for a shared vendor host, not a per-town crawl.

5. capability_class and vendor are two stores that disagree. All 54 Weblex municipalities carry capability_class='e-crawl-only' while vendor-clusters.json calls them weblex. The harvester routes on the capability class, so the built adapter was never reached and 52 towns sat at zero. This is the 2026-07-29 ONE-registry defect recurring on a new column — reconcile it, do not add a third answer.

The capability map — RE-DERIVED over all 1,205 targets (2026-07-30) — READ THIS FIRST

The 2026-07-29 histogram immediately below is SUPERSEDED. It was produced by a classifier that could not have been right: fingerprintOne fetched only the homepage and the two API paths, and never the stored minutes_url. Cross-tabbed over all 1,205 rows, d-html-recipe was EXACTLY the previously-verified set and e-crawl-only EXACTLY the unverified one, zero exceptions — so the D/E boundary was 100% our own crawl history and 0% anything the sites do. Keep reading the old section for its traps and mechanisms, which are still true; do not trust its class counts.

The corrected map (migrations 0462 shape + 0463 data, both DRAFT; loader load-capability-map.ts, 24 tests):

1,072 of 1,205 now carry a class derived from evidence about the SITE (site-measured 932 + vendor-cluster 140). The other 133 say so on the row: 115 inherited-history (still decided by our own crawl history — that is the re-probe queue) + 18 not-probed.

101 locations carry a map_defect — 43 url-is-site-root, 46 no-url, 12 url-404. A map defect is not a capability finding: it is a broken map entry to repair, and it was previously wearing a capability class, which sent adapter work at typos.

./node_modules/.bin/tsx load-capability-map.ts --report     # merged map, no DB, no network
./node_modules/.bin/tsx load-capability-map.ts --emit-sql --out ../../../migrations/0463_*.sql

Three things a future agent must not re-derive:

1. A corrected flag whose candidate list starts with a DIFFERENT column is a tautology. prove-cohort-ad.ts flags 265 of 285 class-A rows corrected, which reads as "93% of stored WordPress URLs are stale". It is not: corrected = storedUrl !== provenUrl and candidate #1 is always api_endpoint, so it is TRUE for every row that proved (265 === reachable 264 + empty 1; stored == api_endpoint in 0 of 265). Writing those into minutes_url would overwrite 265 valid archive pages with API URLs and rewrite 265 primary keys (location_key is GENERATED from minutes_url). Full working: docs/product/qc/2026-07-30-cohort-ad-live-proof.md §3.
2. confidence is not a proof signal, and the fix is a NEW column, not a redefinition. Measured on the live table: confidence='verified' is exactly the 76 rows whose tier is target-registry-verified. Use live_proven_at (0462) for "has this location been proven by a real fetch, and when" — confidence keeps its meaning and its readers.
3. site_family is the leverage axis, not the town. vplus-modellium 71 · blanko 57 · weblex 54 · numerique-sitepascher 49 · adn-communication 33 · wix 22 · accescite-voila 22 · municipalites-du-quebec 18 (the year is literally in the path) · neural 15. One adapter serves a family.

The capability fingerprint — MEASURED over all 1,205 targets (2026-07-29, SUPERSEDED above)

MAP COMPLETENESS — how many municipalities have an image, and what is missing (2026-07-30)

pnpm qc:map-gauge (scripts/ops/qc/map-completeness-gauge.ts) is the ONE answer to "is this municipality's publication image complete, and if not what exactly is missing", rolled up to one province number. Read it before sizing any acquisition work — it gates the ~74k-document pull (Felix's standing order: nothing is pulled before the image is built for every municipality). Definition + today's number: docs/product/qc/2026-07-30-map-completeness-gauge.md. Tier logic lives in migration 0464 (DRAFT) between the GAUGE-BODY markers and NOWHERE else — the script lifts that same block pre-apply, so the two paths cannot drift.

Today: 1,038 / 1,098 resolved (94.5%) — PROVEN 1,020, TERMINAL 18, CLAIMED 22, UNMAPPED 38. (First pass the same day read 956 / 87.1%; the three traps that moved it are below.) pnpm qc:map-gauge --rows lists the remaining 60 individually — you cannot probe an aggregate. The load of the probe evidence into the DB is migration 0466 (DRAFT), generated by pnpm qc:map-gauge --emit-loader <path> from the same collectProbes() the gauge reads.

Three traps that kept 82 municipalities dark, all measured 2026-07-30 — do not re-derive them:

A. A merged, proven adapter does not move the gauge until the gauge READS its output. prove-vplus.ts had live-proved 72 of 74 VPlus tenants, but collectProbes() overlaid four cohort JSONs and not that one, so 70 targets holding a real fetch verdict were tiered off the OLDER e-crawl fingerprint. When a work-queue item names a cluster whose adapter is already built, suspect the overlay before re-running the prover.

B. unreachable is a claim about OUR transport until proven otherwise. All 25 retry-unreachable-host targets failed with the identical string robots.txt unreachable (network error). Twenty-two unrelated hosts do not break the same way on the same day — and none was down: 13 had a cert not covering the www. name (while :80 served fine), 5 an incomplete TLS chain (UNABLE_TO_VERIFY_LEAF_SIGNATURE, fails in Node and curl; browsers hide it via AIA fetching), 4 no 443 listener at all. tlsLadder() (prove-cohort-ad.ts) walks http-same-host → https/http on the www-toggled host, same path and query. Not a downgrade attack — no auth, no robots directive, no working https redirect; the robots.txt fetched over http is the same binding file, every candidate still rides politeFetch, and a 401/403 returns auth-walled before the ladder is reached. Still open: the 8 incomplete-chain hosts need AIA intermediate fetching.

C. empty usually means "wrong page", not "publishes nothing". 15 municipalities run one CMS (#contn anchors, /documents/?c=N, /docs/fichiers_documents/N.pdf); all 15 were unresolved because the registry stored a content page that answers 200 with no PDFs on it. contn-documents.ts reads the minutes category off the site's own nav labels every run — the id is a per-install db key (c=7 Howick, c=3 La Morandière, c=12 Saint-Jean-de-Brébeuf), so a hardcoded map would rot on the next clerk edit. 12 of 15 proved, 2,437 documents named, none banked. Run: ./node_modules/.bin/tsx prove-contn.ts --out /tmp/contn.json.

Two tools that could not be RUN, now fixed:

• --codes on prove-cohort-ad.ts used to and-in capability_class (the memoir column), so a target misclassed f-unreachable by one bad fetch could never be re-proved by the prover that answers it. When the caller names codes, the caller has chosen the prover.
• This package had no env loader. pgUrl() trusted the caller to have run source .env.local, which exists only in the main checkout — so from any git worktree every DB pass (discover-archive.ts, ingest.ts) died with Set SUPABASE_POSTGRES_POOLER_URL while the credential sat two directories up. repo-env.ts mirrors scripts/lib/repo-env.ts (not importable here — non-member npm package). Environment still wins over file.

Four things this cost to learn — do not re-derive them:

1. Nothing in the DB can say "this location was fetched, and when." raw.qc_minutes_fetch_cache has exactly the right shape and exactly one row. probe_status is blank on 948 of 1,098 and cannot say WHICH location was probed. last_verified_at covers 76 rows. The 403 live fetches of 2026-07-30 exist only as JSON on disk. Migration 0464 adds the missing store, raw.qc_minutes_location_probe — append-only, one row per fetch. A fetch is an EVENT; it belongs in a ledger with its own provenance, never in a mutable column on the target. Update 2026-07-30: 0464 is APPLIED and migration 0469 (DRAFT) is its first write — the 1,028 on-disk probe rows, generated by pnpm qc:probe-backfill. Rehearsed: after the load the pure-DB view returns PROVEN 939 / TERMINAL 17 / CLAIMED 111 / UNMAPPED 31, identical to the on-disk overlay, so loading retires the overlay and moves the gate number by zero. Until 0469 applies, pnpm qc:map-gauge --db-only under-reports by 72 PROVEN and all 17 TERMINAL. Known gap: 2026-07-30-dark-robots-audit.json is NOT loaded — its verdicts reached the registry via 0463, which is why 35 locations read robots-terminal in the registry and unreachable in the ledger (both true; the registry took the safety-dominant reading). Loading it is owed and will move the TERMINAL tier.

2. capability_probed_at is the trap, because it is 100% filled (1,205/1,205). It timestamps the ROOT-ORIGIN CMS fingerprint, not the minutes_url. Every downstream reader that treated it as "this target was verified" inherited the capability_class memoir defect described above. Same for confidence='verified': four tiers set it (target-registry-verified, hand-registry, body-targets, discovered-archive) and only the first is a real fetch — of 172 d-html-recipe municipalities, 54 read verified and 118 do not, from the same probe. The eligibility repair is its own section below (migration 0468) — read it before quoting any "how many can we harvest" number.

3. A FINGERPRINT IS NOT A VERDICT, and recency must not let it act like one. The gauge's probe join first used plain latest-wins. A capability re-fingerprint that ran 53 minutes after the fetch which had already retrieved a municipality's documents was overturning that fetch: 43 municipalities demoted PROVEN → CLAIMED, the gate reading 83.2% instead of 87.1%. Ordering is now (outcome = 'route-identified'), probed_at DESC — verdicts outrank fingerprints, recency breaks ties only within a class. The outcome CHECK keeps served-documents and route-identified as separate values precisely so this cannot be re-lost.

4. discovery_url CANNOT answer "do different kinds come from different locations." A naive grouping says 359 of 659 municipalities split kinds across locations. That number is an artifact. discovery_url is DOCUMENT-grained for crawl routes — Wayback snapshot URLs and per-document detail pages, not indexes (municipality 35027: 21 "locations", nearly all individual PDFs). The sound measurement is the registry's own grain: it has held exactly one location per target for the whole harvest (1,205 rows / 1,205 targets), and 717 of the 867 municipalities with any corpus (82.7%) got ≥2 document kinds through that single location (up to 12). Publication is statute-governed, so one index or media API is kind-agnostic in practice. serves_kinds is empty on all 1,205 rows and per-kind locations are a FOOTNOTE — do not spend a front on them.

THE FILE IS A PROJECTION OF THE DB — never a second answer (Felix's ruling, 2026-07-30)

target-registry.json derives NOTHING. It is a deploy artifact: a regenerated projection of the database, committed so git archive HEAD can ship it to a box. The 2026-07-29 ONE-registry decision retired seven stores as independent answers; this applies the same move to the eighth — our own cache.

# THE generator. Reads raw.qc_minutes_target_registry (the map) + serving.qc_target_state
# (the ledger-derived state), writes the file with a `projection` header, and prints the
# diff against the previous file CLASS BY CLASS. Read-only against the DB (Rule 6).
./node_modules/.bin/tsx discover-targets.ts --project-from-db

# is the shipped file current? (audits the header AND compares against the live view)
./node_modules/.bin/tsx discover-targets.ts --report

Three mechanisms keep the role structural rather than disciplinary:

• The marker. The file carries projection: { source, from[], row_count, target_count, harvest_eligible, regenerate_with }. --reconcile / --refresh-proof still exist for the no-DB and surgical cases and stamp source: 'reconcile' — legal, but never silent.
• The load-time guard. loadRegistryForResolution() audits that header on EVERY load and shouts on stderr when it is missing, says reconcile, disagrees with the rows actually in the file, or is older than MINUTES_REGISTRY_MAX_AGE_DAYS (default 7). It never throws: a stale resolution must still run — the operator has to be told, not stopped.
• The CI ratchet. target-registry.test.ts asserts the COMMITTED file carries source: 'db', a self-consistent row_count / harvest_eligible, and a reachability + harvest_eligible + harvest_eligibility_basis on every row. A hand-edited or --reconciled file fails there, before it ships.

Why the guard does NOT phone the database on every load. It easily could — one aggregate over 1,205 rows. It does not, because the harvest path is PROCESS-PER-MUNI: a DB touch here costs a fresh pooler connection per child, measured at 1,667 ms, ×50 lanes per box. A guard slower than the drift it detects is a bad trade. So the free local audit runs always; the DB comparison runs where someone is already paying for a connection — the generator (which REFUSES to write a projection that already disagrees with the view), --report, and any run setting MINUTES_REGISTRY_VERIFY=1.

HARVEST ELIGIBILITY — what a pull would ACTUALLY seed (2026-07-30, migrations 0468 → 0473)

Ask harvest_eligible, never confidence. They are different questions and reading one as the other was the pull blocker.

The answer now comes from serving.qc_target_state (migration 0473), not from 0468's GENERATED columns on the registry. 0468's derivation was right and its INPUT was not: it read the live_proof_outcome snapshot, which no pass rewrote, so it reached 428 locations where the probe ledger proves 554. 0474 marks it and twelve siblings DEPRECATED with named successors.

-- what a pull would seed, and why. THE VIEW, not the registry columns.
SELECT harvest_eligibility_basis, count(*) FILTER (WHERE harvest_eligible) AS eligible, count(*)
  FROM serving.qc_target_state GROUP BY 1 ORDER BY 3 DESC;

# the same numbers off the FILE store the fleet actually reads — now a projection of that view
./node_modules/.bin/tsx discover-targets.ts --report

LIVE, 2026-07-30: 554 of 1,205 location rows are eligible — and every one of them is a MUNICIPALITY. 481 live-proof-reachable + 73 tier-verified-unproven. See the body-tier finding below.

THE DEFECT, MEASURED. isHarvestable() read confidence === 'verified' && !!minutes_url — 76 of 1,205 rows — while 357 locations carried live_proof_outcome='reachable' from a real fetch (migration 0462) and the gauge read 939 PROVEN. A pull started that morning would have seeded 76 of 1,098 municipalities, and the 96% shortfall would have read as adapter failure or dead towns rather than as a column nobody wired. Three of those 76 were robots-terminal — the old gate would have crawled three hosts that forbade us.

THE ORDERING, and it is total (eligibilityBasis() in target-registry.ts, pinned branch-for-branch to 0468's GENERATED CASE by test):

1. TERMINAL BEATS EVERYTHING — robots-terminal / auth-walled → ineligible, checked FIRST and unconditionally, so no tier can override a forbid. These are complete BY EXCLUSION, never a hole.
2. no minutes_url → nothing to seed.
3. live_proof_outcome='reachable' → eligible. The strongest positive evidence we hold.
4. any OTHER live proof (empty / stale-or-404 / unreachable) → ineligible. A negative FETCH outranks a positive TIER.
5. only in the ABSENCE of any live proof does confidence='verified' apply — which is what keeps the 73 never-probed rows working instead of regressing them in the name of rigour.

Three things this cost to learn — do not re-derive them:

1. The FILE store is what the fleet reads, so proof written only to the table changes nothing. loadRegistryForResolution() is FILE-FIRST (the measured 1,667 ms/call reason is in target-registry.ts). SETTLED 2026-07-30 by the projection ruling above: the file now carries the view's harvest_eligible verdict verbatim, so a box does no recomputation at all. Regenerate with --project-from-db; --refresh-proof (two fields, surgical) survives only for the case where the state view is unavailable.

2. The file/table confidence disagreement — 338 vs 76 — is SETTLED: the DB wins. The file's 338 came from a --reconcile walk of TIER_PRECEDENCE, in which body-targets / discovered-archive / hand-registry all yield verified; the table's 76 came from the load. Two derivations of one fact was the defect. The projection corrected 262 rows verified → unverified and the file no longer derives confidence at all.

2b. THE CONSEQUENCE, and it is the finding of that regeneration: ZERO governance bodies are harvest-eligible. All 107 bodies (87 MRCs + CMQ + agglomerations + the Lévis/Saguenay/Sherbrooke boroughs) carry confidence='unverified' and minutes_url_source_tier='body-targets' in the table; 100 of them have never been probed at all (reachability='unprobed') and 7 are robots-terminal. The old file made those 100 eligible purely on the reconciler's body-targets → verified label — a tier name, not a fetch. Net movement on regeneration: +126 municipalities gained (the ledger proves them reachable and the stale snapshot did not) and −100 bodies lost, 528 → 554. The fix is a proving pass over the 107 body targets, never a re-labelling of confidence. Until one runs, a QC-wide pull covers municipalities only — say so rather than discovering it in a shortfall.

2c. Two ledger defects found while accounting for the projection diff (2026-07-30). Neither is fixed here — both are DB writes.

• 6 municipalities are eligible on a self-contradictory probe. raw.qc_minutes_location_probe holds outcome='served-documents' with documents_seen=0 and evidence=NULL for 05015 Saint-Godefroi, 05077 Cascapédia–Saint-Jules, 15013 La Malbaie, 30080 Lac-Drolet, 50100 Baie-du-Febvre, 68005 Saint-Bernard-de-Lacolle — all route='wp-rest-media'. The view reads "served documents" and hands them harvest_eligible=true. A verdict that served zero documents is reached-no-documents (empty), not a proof. The defect is in the wp-rest-media probe WRITER: a positive outcome must not be stamped without a count and a trail.
• On a terminal row, documents_seen describes a DIFFERENT probe than the row's state. The view takes reachability / state_as_of / evidence from the terminal probe but documents_seen unconditionally from the latest non-terminal verdict, so 15 robots-terminal rows report counts up to 364 (e.g. 46040, 80087, 71095). Harmless for eligibility (terminal is checked first) and misleading for anyone reading the row. Fix belongs in 0473's SELECT: gate documents_seen on the same branch that produced reachability.

3. Banked documents are NOT eligibility evidence. The gauge's 939 PROVEN leans on raw.qc_minutes_documents, which proves a fetch of the TARGET succeeded — often through a legacy municipalities.ts seed, not the registry's stored minutes_url. 43 targets store a bare site root and 12 store a 404. Only a fetch OF THIS LOCATION answers "is this location proven".

The capability fingerprint — MEASURED over all 1,205 targets (2026-07-29)

Step 1 of the research doc above is done, live, over the whole universe: fingerprint-capability.ts (34 offline tests). It asks each site "what bulk enumeration do you support?" rather than "who built you", and writes the answer into the ONE target registry (capability_class, api_endpoint, advertised_count, fingerprint_evidence, capability_probed_at — migration 0454, DRAFT). Run it:

./node_modules/.bin/tsx fingerprint-capability.ts --report                      # histogram, free
./node_modules/.bin/tsx fingerprint-capability.ts --run --all --concurrency 12  # ~25 min, 1,205 hosts
./node_modules/.bin/tsx fingerprint-capability.ts --run --codes-file /tmp/cohort.txt

The histogram, all 1,205 probed, zero unclassified:

Four findings a future agent must not re-derive:

1. The research's WordPress prediction is CONFIRMED and the census column is wrong. Base-CMS WordPress detected on 528 of 1,205 targets (predicted ~568; census platform says 395), with the bolt-on portal recorded on its own axis: 160 accescite-voila, 8 bciti, 6 sitesearch360. AccèsCité/Voilà sits on top of WordPress exactly as predicted — the two-axis split is real, and platform must never again be used as an adapter selector.
2. 345 of those 528 WordPress sites have a working REST media endpoint; 182 do not, and the reasons are specific and actionable: 90 → /wp-json/… 404s and ?rest_route= returns the homepage (route rewritten away), 50 → HTTP 401, a Disable-REST-API / Wordfence hardening (a plugin choice, not an absence), 16 → soft-404 on both entrypoints, 11 → robots disallows the /wp-json/ path specifically, 8 → 404 on both, 2 → HTTP 500.
3. advertised_count immediately proved the shallowness thesis, per municipality. Across the 307 class-A municipalities: the sites themselves declare 294,325 PDFs; we hold 59,893. 260 of 307 hold under half of what their own site declares, 20 hold zero, and only 12 are at or above. Worst absolute gaps: Drummondville 8,338 declared / 629 held · Sainte-Agathe-des-Monts 6,179 / 482 · Terrebonne 4,042 / 0 · Magog 3,037 / 25 · Sutton 3,306 / 426. Caveat, do not overclaim: X-WP-Total counts every PDF in the media library (budgets, by-laws, newsletters), so it is a ceiling to measure against, never a target to match. It is still the only completeness oracle we have that needs no ground truth of ours.
4. Class C is 0 because static HTML cannot reveal an XHR endpoint, not because none exists. The detector only follows a document-shaped API URL the page itself printed; the ~220-municipality ASP.NET cohort (125 pure ASP.NET + 88 WordPress+ASP.NET) renders its document lists client-side. Finding those endpoints needs one browser reverse-engineering session per vendor — the highest-leverage repeatable activity left, and it is NOT something this pass can do. Also newly named on the base-CMS axis, so "unidentified" shrinks from a shrug to a fact: 28 Wix, 13 Joomla, 13 Drupal, 9 Webflow, 2 Squarespace; 334 genuinely unidentified.

Two traps, both encoded in code + tests so they cannot come back:

• A 200 is not evidence. Every positive class requires BOTH a corroborating HTML/header signal AND a body that parses into the expected shape. Confirmed live: /wp-json/… returned 200-with-HTML on 16 real WordPress hosts and on soft-404 hosts alike.
• A CMS marker must be a path/asset token, never a bare word. The token divi matched www.st-clement.ca — a Wix site — because divi is a substring of ordinary French words (division, individu). Weak markers are now anchored (et_pb_, wp-content/themes/divi) and are discarded entirely when a hosted builder (Wix/Squarespace/Webflow/Duda) claims the site. Regression-tested. The soft-404 gate caught the harvest consequence, but the row's evidence still said "WordPress", and an evidence field naming the wrong fact is exactly what this pass exists to end.

Also learned about the evidence field itself: reporting only the LAST probe's reason mislabelled 107 municipalities as "SOFT-404" when the truth was a clean 404 on /wp-json/ and a homepage response from the ?rest_route= fallback. Same class, wrong fact. Every attempt is now recorded with the path it was made against (/wp-json/wp/v2/media -> HTTP 404 ; ?rest_route= -> …).

Next lever, ranked by measured value: build the class-A adapter. 345 targets, one adapter, and a declared-vs-held gap of ~234k documents to close on the municipalities alone.

The "everything else" cohorts — REST-blocked WordPress + class D (2026-07-29)

Three findings below cost real measurement to establish and are not inferable from the code. Read them before touching either cohort.

1. /wp-content/ appearing ONCE is not WordPress — it mislabelled 114 of 527 rows

detectCmsSignals treated the literal /wp-content/ as a STRONG WordPress signal on a substring test. It is not. One occurrence is one link pointing at somebody's wp-content upload; a real WordPress theme emits the path on every stylesheet, script, and image. Measured, 30 suspect hosts vs 30 class-A control hosts (class A is proven WordPress — its REST API returned a well-formed attachment collection, which no other host can fake):

114 of the 527 rows labelled "base CMS: WordPress" rest on that single hit (111 of them inside the 182-target REST-unusable cohort). The harm is worse than a wrong label: it manufactures a "WordPress site blocking its REST API" story for sites that never had one, and sends the next agent hunting an entrypoint that cannot exist. Fixed via WP_CONTENT_STRONG_MIN = 3 — a lone hit is demoted to weak, not deleted, so recall is unchanged and the 3 lone-hit rows whose API really did answer keep class A. Reproduce: tsx prototypes/wp-evidence-strength.ts --n 30.

2. Split the REST-unusable cohort by POSTURE, never by HTTP status (wp-alternate-routes.ts)

The 182 are three different problems and only one of them is ours to solve:

The precedence rule is a safety rule: a 401 appearing on the SECOND attempt still makes the target off-limits. 7 targets have exactly that shape (/wp-json/ → 404, ?rest_route= → 401) and a "first reason wins" reader files them as re-probeable.

Measured recovery, all 182 (2026-07-29):

139 of 182 recovered. The four REST hits declare 4,383 PDFs between them (Sainte-Claire 1,581 · Grosse-Île 1,779 · L'Anse-Saint-Jean 640 · Schefferville 383).

53 of the 57 owner-blocked sites publish a sitemap that reaches their minutes archive. A site that closes its REST API and still serves a sitemap has told us both things on purpose — the public route is an advertisement, not a back door.

A sitemap gives PAGES, never documents. WP core's /wp-sitemap.xml excludes attachment by design, and Yoast does too. Anything calling those 7,994 URLs "documents recovered" is lying.

3. Class D clusters into 4 shapes — and the parity oracle mostly DOES NOT EXIST there

index-shape.ts measures each index page on six structural axes (linkage · naming · history · declared count · rendering · vendor). Measured live over all 241 class-D targets — 17 shapes, 6 cover 80%:

Four recipes in index-recipe.ts cover 209 of 241: R1 href-token (150) · R2 text-token (33) · R3 opaque-href (15) · R4 detail-pages (11). 130 targets expose their archive BY YEAR — the full-history axis walkArchive drives.

THE PARITY ORACLE WAS MEASURED WRONG, AND ONLY OPENING THE EVIDENCE CAUGHT IT. The first pass reported a declared count on 49 of 241 targets summing to 93,153 documents. Reading the evidence strings: 46 of the 49 were YEARS. QC archives are laid out as year sections headed literally «2026 Procès-verbaux», which a bare <number> <noun> pattern reads as a total — "2026 procès-verbaux" appeared on 26 separate targets. After the year guard (YEAR_LIKE_UNSAFE): 4 targets, sum 2,047.

So: X-WP-Total's class-A parity oracle essentially does not exist on class-D pages — 4 of 241 (1.7%). Do not plan around it. That is a finding, not a failure, and it is worth far more than a confident 93,153 that was mostly the current year repeated 26 times.

Proven live — 25 municipalities across 13 shapes (prove-index-recipe.ts, real bytes on disk)

TOTAL  found 988   already-held 584   NET NEW 404
       downloaded 11, of which 9 are real PDFs (%PDF- magic bytes)

Saint-Narcisse-de-Rimouski is the thesis in one row: 15 years of archive against 97 documents held.

Two honest caveats, both measured:

• NET NEW is a DISCOVERY number, not a bankable one. 2 of the 11 sampled downloads came back as 2,090-byte HTML pages served with HTTP 200 for a .pdf URL — stale hrefs on the site, soft-404'd at the DOCUMENT level. The existing %PDF- magic-byte guard and the error-page interstitial gate reject these at acquisition, so the corpus is protected, but expect the banked count to land below NET NEW.
• R4 (detail-pages, 11 targets) found 0 documents and correctly so: it emits its items with detail: true, and walkArchive excludes detail pages from the document set by design. The one-hop fetch is ingest.ts's existing follow seam and is not yet wired to these recipes. That is the next piece of work on this cohort, not a bug in the walker.

Honest remainder

Of the 182: 43 unreached (42 none + 1 feed-only) — 35 route-missing, 3 owner-blocked, 3 robots-disallowed, 1 server-error. Of the 241: 16 unreachable + 11 none-visible (JS-rendered shells needing the headless path) + 5 already owned by vendor-recipes.ts.

Steady-state re-crawl caching (harvest-engine Phase 1) — all flag-gated, default OFF

The corpus is ~75% built; the engine's ongoing job is detecting the handful of NEW docs each cycle. Four levers make an unchanged re-crawl near-free. Every one is default-OFF — nothing changes on the fleet until its flag is set — and each obeys the hard invariant that caching can only skip re-WORK, never a re-CHECK (a new/changed doc is never missed).

MINUTES_CACHE=1 is also the silent-death detector's missing half (2026-07-29). Beyond saving re-crawl work, link_fingerprint (the index page's content hash) and discovered_count (how many documents the source's own listing page advertises) are the two index-side observations pnpm minutes:freshness needs to tell "a CMS redesign broke our selectors" apart from "the town stopped publishing" — today indistinguishable, both landing in SOURCE_QUIET. The table holds one row (Sherbrooke) because the flag is off across the fleet, so that half of the 2×2 is dark for 1,204 of 1,205 bodies. Turning the flag on fleet-wide is the cheapest unlock; migration 0452 (DRAFT) then adds the previous-cycle columns that make "the index changed" a computable fact. See scripts/ops/minutes/freshness-sql.ts.

Backing store — why the DB, not a file. raw.qc_minutes_fetch_cache lives in the data plane, not a gitignored file, for the exact reason 0368/0380 do: the fleet's watch-drive.yml runs actions/checkout every tick and WIPES local state, so a file cache would be cold every tick and save nothing. A file-backed variant of the same PageCache interface exists for --local-only proof runs (where there is deliberately no DB write) — that is how the warm-vs-cold measurement is taken without a Felix-gated write.

Origin-dependent. The 304 win only materializes on origins that emit ETag / Last-Modified. Many WordPress pages send Cache-Control: no-cache and no validator (e.g. biencourt.ca) — those re-fetch every cycle (correct, just no saving). Static/nginx/CDN-fronted munis (Hemmingford, Laforce, cloridorme, l'Anse-Saint-Jean, Magog, Boucherville, Drummondville archive …) DO validate and 304 cleanly.

NOT cached (on purpose — see the handback): the deep-crawl path (deep-crawl.ts) and headless seeds. A deep crawl's per-page 304-reuse is coupled to its headless-escalation decision (a JS-gated page whose HTTP shell 304s can still serve fresh JS-fetched data), so caching it safely needs more than a validator on the shell — deferred rather than risk missing a child-only new doc. The deep frontier still gets keep-alive.

# measure a warm-vs-cold re-crawl with no DB write (file-backed cache):
MINUTES_CACHE=1 MINUTES_DEFER_OCR=1 OUT_DIR=/tmp/mh MUNICIPALITY=68015 npx tsx ingest.ts --local-only --max 1   # cold: "1 fetched"
MINUTES_CACHE=1 MINUTES_DEFER_OCR=1 OUT_DIR=/tmp/mh MUNICIPALITY=68015 npx tsx ingest.ts --local-only --max 1   # warm: "1 served from cache (304), 0 fetched"

Ingest-boundary quality gates (harvest-engine Phase 4) — flag-gated, default ON

Three defensive checks on the fetched buffer before it is hashed + stored, each in acquireOne (ingest.ts), extending the existing %PDF- magic-byte guard. Both new checks are default ON; set the override to 0 to disable — every rejection is counted (summary.failed) and logged with its reason, never silently dropped. Full rationale + live proof: pdf-quality-gates.ts.

# count the current truncation cohort (read-only, Rule 6):
psql "$SUPABASE_POSTGRES_POOLER_URL" -c \
  "SELECT count(*) FROM raw.qc_minutes_documents WHERE bytes = 1048576;"
# 1,853 as of 2026-07-29 — this run's re-harvest worklist (their content_hash
# is real, but the bytes are not; they should be treated as never-acquired
# and re-fetched once a fixed run passes this gate).

Tests: pdf-quality-gates.test.ts (network-free, fixtures modeled on the live truncated-cohort shape + the corpus's own smallest real document).

The PHANTOM FRONTIER — robots-forbidden work counted as owed (2026-07-29)

Read this before sizing any minutes shortfall or diagnosing a municipality that "never converges". It cost a full investigation to establish and is not inferable from the code.

The mechanism. serving.qc_minutes_completeness grades a municipality on one question — did the archive advertise more PDFs than we hold? It never asks whether we are permitted to fetch them. Where robots.txt says no, fetch-polite.ts's isAllowedByRobots gate refuses every candidate before any request goes out (correct, non-negotiable, unchanged). So the shortfall cannot fall, the muni stays on the frontier (fan-driver.ts selects short|has_gaps|unverified), it is re-crawled every cycle, and it banks nothing.

The blocking shape is NOT Disallow: / — this is the part everyone gets wrong, including the brief that opened the investigation. Of 51 frontier hosts probed, only 3 disallow the whole site. The dominant shape is the stock Joomla/WordPress hardening line, which lands exactly on the PDF directory while leaving the archive index crawlable:

Discovery succeeds, acquisition is forbidden. That asymmetry is what manufactures a permanent shortfall, and it is systemic across the QC long tail rather than a few hostile operators. A fix keyed to Disallow: / misses 8 of the 11 disallowed hosts.

Measured cost (2026-07-29): 2,246 refused attempts across 22 municipalities = 27% of every acquisition attempt ever logged (2,246 of 8,245), still climbing daily.

DO NOT size this from completeness_state. The short membership churns within minutes while the fleet runs — measured the same afternoon it went 17 munis/1,211 owed → 16/952 → 15/724, and the two worst offenders (476 owed docs, both 100% robots-refused) left short between two reads twenty minutes apart. Size it from the refusal ledger, which does not churn:

# the durable cut — every muni we have ever refused a URL for, on robots grounds
psql "$SUPABASE_POSTGRES_POOLER_URL" -c \
  "SELECT municipality_code, split_part(split_part(doc_url,'://',2),'/',1) AS host,
          sum(attempts) AS refused_attempts, max(last_attempt_at)::date AS last
     FROM raw.qc_minutes_doc_attempts
    WHERE last_error ILIKE '%robots.txt disallows%'
    GROUP BY 1,2 ORDER BY 3 DESC;"

# then probe each distinct host's robots.txt ONCE and classify (read-only; the
# ONLY URL fetched is /robots.txt, always-allowed under RFC 9309 §2.2.2 —
# candidate paths are evaluated in memory and never requested):
tsx probe-frontier-robots.ts --source refusal-ledger          # the durable cut
tsx probe-frontier-robots.ts --states short                   # the owed-docs cut
tsx probe-frontier-robots.ts --source refusal-ledger --emit-sql   # Felix-gated UPSERTs

The fix is migration 0450 (DRAFT): the robots_blocked terminal state, backed by raw.qc_minutes_robots_verdict (the durable, 90-day-leased per-host verdict — robots-verdict.ts) rolled up per municipality by serving.qc_minutes_robots_block. Same off-frontier mechanism 0440 used for gaps_exhausted: the frontier is an inclusion list, so a new terminal state drops off it with no fan-driver.ts change.

Three things about that design worth not re-deriving:

1. The coverage gate cannot be >= shortfall. isHostScopedFailure correctly treats a robots disallow as a HOST fact, so after the first refusal the host is skipped for the rest of the run and the ledger records 1–2 refused URLs against a shortfall of hundreds (58037: 2 rows / 205 attempts vs shortfall 491). The sealed-class test that works for blocked_at would never fire here. The gate is forbidden_candidate_urls > 0 AND allowed_candidate_urls = 0 — every outstanding candidate forbidden, not one allowed/unreachable/unverdicted.
2. unreachable is never a block. A 5xx/DNS failure on /robots.txt makes us refuse the host meanwhile (RFC 9309 §2.3.1.3, fail-closed) but the site never told us to stay out. Treating it as terminal is how a municipality gets sealed shut by our own infrastructure. It also correctly excludes the malformed-URL ledger rows (below).
3. It under-fires by construction. The ledger holds only attempted URLs, so a muni whose forbidden docs were never attempted has no candidate rows. Absence from serving.qc_minutes_robots_block is not evidence a municipality is unblocked. Closing that needs the discovered URL set persisted; raw.qc_minutes_harvest_state stores only last_discovered_count.

Two open defects found while doing this. The first is now FIXED — and its stated cause above was WRONG; the second is still open:

• Malformed candidate URLs — FIXED 2026-07-29 (candidate-url.ts). Discovery is resolving relative hrefs against the wrong base. That diagnosis was refuted. Re-measured: 6 URLs, 319 attempts (not 309), five hosts — include (92050, 226), http (13050, 73), localhost (mrcgsl, 15), a (61040, 3), https (50113, 2) — all still firing on 2026-07-29.

  Fetching the producing page under the honest UA (https://stedmond.ca/proces-verbaux, 200, 44 052 bytes) shows no <base> tag at all and the raw markup literally carrying href="http://include/uploads/images/media-bloc-text-27-…pdf". Our relative-href resolution and our <base href> honoring are correct and were never the producer. The real cause is upstream authoring: a CMS link field that auto-prefixes http:// onto whatever the editor typed (a site-relative path include/…, a colon-less http//example.ca/…, a bare a), or a link written while the site lived on a dev host. The ledger pairs each bad URL with the same path under the right authority, which is the tell: http://http//municipalitelejeune.com/… vs http://municipalitelejeune.com/….

  Why they burned forever rather than dying at DOC_RETRY_CAP: those hosts do not resolve, so the robots.txt probe fails, the parser fails CLOSED (correct), and the attempt logs as robots.txt disallows …. isHostScopedFailure() then reads that as a property of the HOST and deliberately does not advance blocked_at. Right rule, wrong input — so an impossible URL has to be killed before it becomes an attempt.

  Fix: candidate-url.ts's candidateUrlDefect() drops a candidate at discovery, with the reason logged (deduped per host+defect), when the authority is a doubled scheme (http/https/ftp), a loopback/dev host, a bare single-label name that is not an IP literal, or the scheme is not http(s). Conservative by construction — a false drop silently loses a real document, so ports, deep subdomains, punycode IDNs, IPv4 literals, FQDN trailing dots, and junk-looking paths/queries under a valid authority are all kept. Wired into extractAnchors (ingest.ts), the headless $$eval link read (ingest.ts), extractRichAnchors (deep-crawl.ts), and parseSitemapLocs (sitemap.ts). Tests: candidate-url.test.ts. The 6 existing ledger rows are not purged — that is a Felix-gated DB write.

  Lesson for the next agent: "malformed URL in the ledger" reads like our resolver's bug and is not. Fetch the producing page and look at the raw href before touching the <base href> code — that handling is load-bearing for Vaudreuil-Dorion and Lévis and must not be weakened.

• Governance bodies are not covered. serving.qc_minutes_frontier is a UNION of qc_minutes_completeness and qc_governance_body_completeness (0446). 0450 patches only the first. mrc-la-jacques-cartier, mrc-maskinonge and mrc-temiscouata are being refused right now (54 attempts, all first seen 2026-07-29 — this front has only just started hitting it) and need the same two arms applied to the governance-body view.

Telemetry. raw.qc_minutes_doc_attempts.error_kind (0450) records the typed failure kind from the thrown error's class identity (classifyThrownError, cloud/error-taxonomy.ts), so this never again has to be sized by grepping last_error. The write is column-guarded (errorKindColPresent()), so the harvester is safe to run against either schema shape.

The WordPress REST media adapter (wp-rest-media.ts) — the class-A lever

Read this before harvesting, sizing, or diagnosing any a-wordpress-rest target. Built + proven live 2026-07-29/30; the traps below cost the build and are not inferable from the code.

fingerprint-capability.ts classifies every target by what bulk enumeration it supports rather than by CMS brand. 345 of 1,205 targets are a-wordpress-rest — a live /wp-json/wp/v2/media endpoint. For those, wp-rest-media.ts reads the CMS's own database instead of scraping whatever the current theme renders.

# one target, read-only, REST path alone
MINUTES_WP_REST_ONLY=1 MUNICIPALITY=64008 npx tsx ingest.ts --plan
bash prototypes/wp-rest-proof.sh plan 64008 45072 49058     # the sample runner

Levers: MINUTES_WP_REST=0 kills the tier; MINUTES_WP_REST_ONLY=1 makes REST replace the crawl. Default is ADDITIVE — a class-A target resolves to [wp-rest-config, <its existing tier>] and both run, because the media library is not a strict superset of what the archive page links (externally-hosted and pre-migration PDFs live outside it) and the corpus is content-addressed, so the overlap is a skip. A purpose-built adapter (montreal-docid / quebec-gpd / laval-findstr) is never displaced.

Three traps, all measured

1. X-WP-Total is a CEILING TWICE OVER — never subtract from it. It counts every PDF in the media library (budgets, newsletters, by-laws, logos), so it is never the minutes count. Worse: WordPress counts every matching attachment in found_posts → X-WP-Total but OMITS from the response BODY any attachment whose PARENT post is not publicly readable (WP_REST_Attachments_Controller::check_read_permission). Measured on Drummondville (49058): X-WP-Total 8,338, rows the API will actually hand back across the whole offset space ~1,678 — a per_page=100 page returns 55, then 17, then 11. So advertised is not even the enumerable ceiling. Report advertised / examined / selected as three separate numbers.

2. An empty page is NOT the end of the collection. Direct consequence of (1): a run of offsets whose attachments all hang off private parents returns [] with thousands of rows still beyond it. Breaking on an empty body truncated the walk mid-archive and exited green. Only an explicit rest_post_invalid_page_number (400) or exhausting X-WP-TotalPages ends a window.

3. Never offset-page; slice by DATE. The plan is derived from the archive's OWN oldest media row (orderby=date&order=asc&per_page=1) — one window per year to today, subdivided year→month→day when a window's own X-WP-Total exceeds the 10k wall. Windows overlap 2 s at each edge because WP's after/before are EXCLUSIVE; abutting windows silently drop boundary-second rows. A day that still overflows is recorded in stats.truncatedWindows, never dropped silently.

Also: the media date is an UPLOAD date, not a session date. Towns bulk-import decades in an afternoon. It slices windows and is never emitted as a document date — classify-doc.ts still reads the real date off the filename.

Resilience: one bad window retries once, then subdivides, then lands in stats.failedWindows and the walk continues. Pointe-aux-Outardes (96030) aborted at zero discovered on a single transient non-JSON response before that was in; after, 811 minutes-like found.

Selection reuses pdf-heuristics.ts verbatim — isPdfCandidateUrl + MINUTES_KEYWORD (URL) or MINUTES_TEXT_KEYWORD (media title), minus GENERIC_EXCLUDE. GENERIC_INCLUDE is deliberately not used: its MEDIA_PATH arm matches every wp-content/uploads file and would harvest the town's entire document dump.

WHOLE-COLLECTION FIRST — do not re-derive this bound

Date windows exist for exactly one reason: the offset wall. A library that fits comfortably under it is walked in ONE unwindowed pass (WHOLE_COLLECTION_MAX, 1,000 rows = 10 pages). The bound is deliberately far BELOW PAGE_WALL — degradation starts long before 10k, and the point of this adapter is shallow offsets. Measured on the live cohort: 218 of 345 class-A targets fit and cost 2–4 requests; the other 127 keep date windows. Laforce (85070, 23 PDFs, no usable media date) went from 60 requests to 2 when this landed.

Measured sample, 2026-07-29/30 (--plan, MINUTES_WP_REST_ONLY=1)

alreadyKnown was 0 on all ten. Every URL the REST enumeration returns is new to the corpus — including for municipalities where we already hold hundreds of documents (Brownsburg-Chatham holds 1,044 and shares none of the 336). The crawl path and the media library reach genuinely different URL sets. That is the empirical case for keeping the tier ADDITIVE, and it is confirmed the other way too: on Laforce the crawl finds 9 and REST finds 1.

Selection yield varies enormously — 0.7 % of the library on Hampstead, 77 % on Havre-Saint-Pierre — so never project one municipality's ratio onto another.

73 documents downloaded end to end (--local-only --max 25, the three worst-gap municipalities): 72 files, 101.4 MB, 72/72 valid %PDF magic, 72 distinct content hashes, classified proces-verbal / reglement / avis-public / ordre-du-jour / resolution.

Target resolution — ONE table, then the legacy tiers (2026-07-29)

Felix, verbatim: "We need one fucking column with minutes URLs."

"Where does this body publish its procès-verbaux?" used to be answered in SEVEN places with no rule about which won — serving.qc_minutes_coverage_census .minutes_url / .website, serving.qc_minutes_coverage_census_seed.minutes_url, serving.qc_municipality_directory.website, canonical.entity_governance.official_url, municipalities.ts, and discovered-archives.json. Two measured consequences on 2026-07-29: the fleet idled on no resolvable config (registry/census/discovered all miss) for 240 municipalities whose minutes_url was already in our own database, and an agent was dispatched to scrape the MAMH open-data directory to reproduce a column we already store.

raw.qc_minutes_target_registry (migrations 0451 + 0454 + 0455 applied) is now the single answer. It covers the whole universe — 1,098 municipalities + 107 MRC/CM/agglomeration/borough bodies = every row of serving.qc_minutes_frontier (1,205) — with ONE authoritative location URL per row plus the provenance beside it (minutes_url_source_tier, evidence, confidence, last_verified_at) and the canon-P0 identity link (entity_id → canonical.entity). A target we know nothing about still gets a row, with confidence='unknown': a queryable gap is a work item, a missing row is invisible.

THE GRAIN (migration 0458, DRAFT) — one row per (body, publication location)

Québec municipal documents are statute-governed, so all 1,098 municipalities produce the same KINDS (procès-verbaux, règlements, avis publics, budget, PTI, états financiers, rôle d'évaluation) — the corpus already holds 19 distinct doc_type values. What varies is only how each kind is ACCESSED. So:

• A publication location is usually kind-agnostic — a WordPress media collection (345 targets are class a-wordpress-rest) hands over the whole library and you classify afterwards.
• Some bodies publish different kinds at genuinely different locations, which one minutes_url per body could not express.

The key is therefore (target_code, location_key), where location_key is a GENERATED column (the URL with scheme + trailing slashes stripped, or the sentinel (none) so a dark target keeps exactly one queryable row). serves_kinds[] declares which kinds a location is known to serve; EMPTY means kind-agnostic, not "serves nothing". doc_type stays a property of the document (raw.qc_minutes_documents.doc_type) — it never becomes a column here.

In code, every store is target_code -> TargetRecord[] (TargetRows), and a target's N verified locations become the N seeds of ONE MunicipalityConfig, so ingest.ts's pipeline consumed the grain change unchanged.

The drawer is not renamed: qc_minutes_* is a legacy string exactly like overwatch in this repo's package names (ARCHITECTURE.md invariant 3 — drawers evolve additively). This is the municipal document publication registry, and the harvester it feeds is a municipal document harvester that was built for minutes.

website / website_source_tier were retired by 0458 (the one-fact paydown): canonical.entity_governance.official_url owns that fact. Derive it — coalesce(official_url joined on entity_id where valid_to IS NULL, substring(minutes_url from '^https?://[^/]+')) — proven lossless over all 1,182 stored values before the drop (measured 2026-07-30: 1,075 recoverable through the FK, 107 body rows through the location origin, 23 held none, 0 remainder).

Reconciliation precedence (one place: TIER_PRECEDENCE, pinned against 0450 by target-registry.test.ts): target-registry-verified → hand-registry → body-targets → discovered-archive → hand-registry-unverified → census → census-seed. Websites are no longer reconciled from a precedence list of stored columns — since 0458 they are DERIVED: entity-governance (the owner) → location-origin (the body tier, which has no entity_governance row).

The registry can only ADD resolution, never replace a working tier with a guess: findRegistryConfig answers exclusively for confidence='verified' rows — a page WE fetched that carried ≥ 2 minutes-shaped PDFs, or a human-validated hand seed / live-mapped body source. Everything else falls through to the legacy tiers, which keep harvesting under their own honest labels.

The platform-label hole (census-generic.ts)

Every census adapter keyed on a platform LABEL — wordpress-pdf, self-hosted-cms-pdf, accescite-voila — and the 2026-07-16 survey filed 216 rows under other. Measured universe-wide: 1,027 of 1,098 municipalities have a minutes_url, 787 match a census tier, 240 have a usable URL no predicate can match (168 other/online-scrapable, 48 other/online-nonstandard, and a tail of bciti / sitesearch360 / maruche).

census-generic.ts keys on the ACCESS shape instead — online-scrapable + a usable minutes_url, minus the three served labels and the api bucket (0378) — so the four census tiers stay mutually exclusive. It reuses deriveWordpressConfig verbatim: the discovery heuristic never consulted the label except as a provenance stamp (census-accescite.ts's header measured exactly this for its own 173-row cohort). The 70 online-nonstandard rows are deliberately excluded — that is the census's own "client-rendered, needs headless" note, and a plain-fetch seed pointed at one discovers 0 and looks like a shortfall.

Proven live on 25 unverified municipalities: 15 went from NO RESOLVABLE CONFIG → resolves (04015 Mont-Saint-Pierre, 06005 Maria, 06025 Escuminac, 08005 Les Méchins, …), each with a real archive URL.

./node_modules/.bin/tsx discover-targets.ts --reconcile          # seven stores -> one row per target
./node_modules/.bin/tsx discover-targets.ts --report             # the headline
./node_modules/.bin/tsx discover-targets.ts --verify --limit 150 --concurrency 8
./node_modules/.bin/tsx prove-resolution.ts --state unverified --limit 25

--verify is robots-binding end to end (it drives discover-archive.ts's prover through fetch-polite.ts); --concurrency is across hosts only — pacing within any one host is still fetch-polite.ts's job and is unchanged.

VENDOR APIs — cracking the client-rendered cohort (2026-07-30)

Read this before writing any new discovery code for a municipality that "discovers 0". The answer is usually not a better regex or a headless render; it is that the town's CMS has a document API nobody looked for.

Why the capability fingerprint said c-vendor-json = 0

fingerprint-capability.ts classified all 1,205 targets and returned zero vendor-JSON sites. That was a property of the DETECTOR, not the universe. findEmbeddedApiCandidates only follows a URL the page printed AND only when it matches API_PATH_HINT — /(api|rest|graphql|services|webapi|umbraco/api|ajax)/… — AND only same-origin. Three rules, each defensible alone, that compose into a blind spot covering the entire ASP.NET cohort. The two APIs cracked below fail them in three different ways:

"Vendor API" is shape-agnostic. What makes something an API is that it enumerates the archive from parameters we control — not that it speaks JSON.

The clustering method (vendor-cluster-scan.ts)

One cheap GET per target, robots-bound, --concurrency across hosts only. Two signals, both un-fakeable:

1. Response headers a vendor stamps on everything — x-created-by: DotMedias, x-web-platform: Commerscale. Strongest possible evidence.
2. Vendor-ORIGIN asset hostnames in the markup — a CMS vendor cannot serve its widget bundles from the town's own domain, so a hostname shared by dozens of municipalities IS a vendor. The report prints a histogram of these, so the cluster list is self-extending: a vendor nobody has named still shows up as a row with a count.

./node_modules/.bin/tsx vendor-cluster-scan.ts --scan --all --concurrency 8
./node_modules/.bin/tsx vendor-cluster-scan.ts --report
bash prototypes/vendor-coverage.sh     # the join that matters — see below

CLUSTER SIZE IS NOT UNLOCK SIZE — measure held, never membership

The single most important number from this pass. Joining the cluster census to what the spine actually holds (prototypes/vendor-coverage.sh, 2026-07-30):

The three LARGEST clusters are already fully served by the generic crawl path — their archives are plain <a href="…pdf"> (Stoneham on blanko.ca: 186 static PDF anchors; Saint-Laurent-de-l'Île-d'Orléans on bixocontact.com: 234). Writing adapters for them would add nothing. Weblex is the whole prize: 54 towns, 15 documents, 52 at literal zero. Never size a vendor unlock from membership.

The vendors NOT worth an adapter, and why (so nobody re-derives it)

• numerique.ca (57), goazimut.com (55), blanko.ca (33 + 24 on the byscuit-wrapper subdomain), bixocontact.com (20), infotechdev.com (19), platform-api.sharethis.com (21). Plain static .pdf anchors, well-stocked in the corpus (125–320 docs/muni, ~0 at zero). The generic crawl already owns them. Note bixocontact.com / sharethis / addtoany are contact-form and share widgets, not CMSes — they appear in the histogram because they are common, which is exactly why the histogram is a CANDIDATE list to check, never a vendor list to trust.
• adncomm.com / hebergementadn.ca (33 munis, 26.2 docs/muni, 19 of 33 under 25 docs). The only real secondary gap, and it is not an API case: Saint-Aubert and Donnacona serve 18 and 26 static PDF anchors respectively — the CURRENT year only, with the back-archive behind a year selector. That is a DEPTH problem, and it belongs to the existing MINUTES_DEEP=1 deep-crawl lane (deep-crawl.ts), not to a new adapter. Filed here so the next agent does not spend a night reverse-engineering an API that is not there.

Weblex / GestionWebLex — CRACKED (closes tile 479)

Full spec in weblex-doclist.ts's header. The short version:

ENUMERATE  GET https://apps.gestionweblex.ca/doc-list/assets/list.ashx?listid=<GUID>&culture=fr-CA
           -> JS assignments (NOT JSON); one documents.push({id,name,type,size,group,publishedOn,url})
              per document; the WHOLE archive, every year, in ONE request. No token/session/pagination.
DOWNLOAD   GET https://apps.gestionweblex.ca/doc-list/handlers/document.ashx?documentid=<GUID>
           -> 200, application/pdf, %PDF, content-disposition carries the real filename.
HANDLES    the listid is printed by the SERVER into static HTML as a <script src>. No browser needed.
ORACLE     the documents.push count = the site's own declared total.

The one file everyone stopped short of. A 2026-07-22 pass reached list.ashx by hand and recorded in municipalities.ts that «every entry's url field is empty; download URLs are built client-side and never appear as a static .pdf-token href» — and concluded the family was un-harvestable. The builder is a single line in apps.gestionweblex.ca/doc-list/assets/scripts.js. An empty url field is not a dead end; it is the signal that the entry is a hosted file addressed by its own id. A non-empty url means type == 'web' — an outbound link, which we do NOT synthesize a handler URL for. Lesson: when a widget's payload looks like a dead end, read the widget's own script bundle before concluding anything.

The month-literal trap. The payload emits dates as the JS EXPRESSION new Date(2026, 07-1, 13) — the 1-based month templated into source, with the subtraction left to the engine. The printed first number IS the human month. Reading it as already-0-based shifts every date in the cohort back one month. Regression-tested in weblex-doclist.test.ts.

publishedOn is NOT the session date. Caught by opening the first --local-only proof rather than reading its counts: Saint-Omer's entire 2026 archive shares publishedOn = 2026-07-13 (a site migration), while the entry NAMES say «2 juin 2026», «4 mai 2026». Taking publishedOn gives a corpus where dozens of meetings happen on one day — populated-looking and useless. The document's own French title wins (weblexSessionDate reuses parseFrenchDate).

Neural (Oznogco Multimédia) — CRACKED

Full spec in neural-collection.ts's header.

HANDLES   <div class="cAjaxCollection" id=… data-type=<c> data-nbr=<n> data-date=<year> data-titre=…>
          — server-printed into static HTML, one per document section.
ENUMERATE GET {origin}/neural/ajax_collection.asp?id=…&m=…&c=…&a=<year>&n=…&t=…
          -> an HTML FRAGMENT: a year-picker (the year enumeration — never guess a range)
             plus plain same-origin .pdf hrefs.
ORACLE    {origin}/documents/xml/collections/collection-<c>.rdf

Three traps, all regression-tested:

• http:// hrefs on an https:// page. Neural stores fully-qualified URLs from whenever the site was built. A same-ORIGIN (scheme-inclusive) check drops every document and the adapter reports a confident zero. Judge membership on the hostname. And do not "upgrade" the scheme: knownUrlsFromDb is an exact-string set and the corpus already holds these under http URLs, so a rewrite makes every held document look net-new forever. (Verified there is nothing to gain: these origins serve the PDF over plain http with no redirect.)
• A per-request ?v=<timestamp> cache-buster on every href. Left on, the same document presents a new source_url every cycle, the resume check misses, and every pass re-downloads the whole archive while reading as a permanent shortfall. stripCacheBuster removes v and preserves every other parameter.
• Two template generations with DIFFERENT ajax contracts. v7.8 (Saint-Simon) takes a=<year>&n=0; v8.0 (Saint-Cyprien, Saint-Hubert) keys off the container's data-date and answers n=0 with an HTTP 500. Chasing that as a per-variant matrix is a losing game. The adapter instead unions the RDF feed in, always — additive, so it can only ADD reach — while keeping the live ajax path primary because the feeds run months stale (Saint-Simon 2026-01-27, Saint-Cyprien 2026-04-22, both read 2026-07-30). A variant we do not parse degrades to "the archive minus its recent months" instead of to zero. This is what took 12005 / 12010 / 12020 from advertised 398, emitted 0 to four-for-four acquired.

Also: a Neural page declares EVERY section it renders — «Listes des déboursés», «Bulletins». isMinutesSection keeps only the council ones. Saint-Simon's 123 already-held documents are almost all déboursés and comptes-à-payer, which is why its 114 discovered PVs are genuinely net-new.

VPlus (Modellium) — CRACKED, and the widest cohort yet (74 municipalities)

Full spec in vplus-collection.ts's header.

DETECT    <vplus-app-root> in the static shell (Angular bootstrap element), corroborated by
          cdn.icomoon.io/202015/VPlus/style.css. Both are in markup we already stored.
HANDLE    the site's OWN HOSTNAME, verbatim — derived, never tabulated. From the vendor's bundle:
            getSubdomain()  = hostname, or its first label on vplusportal/portailcitoyen/vplus{test,dev,sim}
            getApiBaseUrl() = "https://vplus.modellium.com/api" + "/" + that handle
SITE MAP  GET {apiBase}/config/pc?localisation=fr
          -> routesTree: EVERY page the site has, nested, in ONE ~50 KB response. No crawl.
PAGE      GET {apiBase}/structure/detail/<path-segment>?inStructure=false&localisation=fr
          -> { titre, contenu } — contenu is the page's stored HTML, document links are plain <a href>.
DOCUMENTS vplus-documents.s3.ca-central-1.amazonaws.com/<appNom>/_publication/fichiers/*.pdf

Why this cohort was invisible. The homepage is a ~2.6 KB shell with zero anchors, so detectCmsSignals reports unidentified (83 of the 103 unproven dark municipalities) and every path-based API detector finds nothing. An SPA writes its API BASE in its bundle and nothing in its markup — a third blind spot beyond the two Weblex exposed (different origin, no /api/ segment). fingerprint-capability.ts now carries API_ROOT_HINT, cross-origin candidates from first-party bundles, looksLikeSpaShell, an opt-in scanSpaBundles pass, and VENDOR_SHELLS — the zero-request markup table that makes the NEXT SPA vendor a table row instead of a night of bundle archaeology.

Four traps, all regression-tested:

• slug is not the API key; the PATH segment is. Maria's «Séances du conseil municipal» carries slug: 'seances-du-conseil-municipal' and path: '/publication/ordre-du-jour-et-calendrier-des-reunions' — only the path segment answers, the slug returns HTTP 403. They diverge when a clerk renames a page, i.e. on the pages most actively maintained.
• The stored minutes_url is often a legacy path the SPA renders as not-found (Maria's /proces-verbaux; the live page is /publication/proces-verbaux). Only its ORIGIN is used; routesTree is the authority. That inversion is why the adapter proves towns whose stored URL is wrong — no seed-based path could.
• The vendor DELETES accents from stored object names. Saint-Adrien's archive is filed as Procs-verbal de la sance ordinaire du 7 avril 2025.pdf — no è, and no e. fold() only strips accents that are PRESENT, so every pattern accepts the vowel-dropped spelling. Found only because the anchor LABEL still carries them and the two disagreed.
• A page-title gate that SKIPS loses whole archives. Saint-Adrien files 632 files on a page called «Document». The title gate now DEMOTES instead: the page's links are judged individually on a stricter vocabulary (looksStrictlyLikeMinutes) that requires an explicit minutes/agenda token AND excludes other document kinds. The loose vocabulary, applied to bare filenames, selected Reg_873_Tenue_seances_conseil_web.pdf and Reglement-2024-324.pdf — by-laws whose SUBJECT is meetings. Caught by opening the output, not by reading the counts.

MEASURED, whole cohort, prove-vplus.ts 2026-07-30 — discovery only, zero documents fetched:

Machine-readable: docs/product/qc/2026-07-30-vplus-cohort-access.json.

The vendor-API resolution tier (census-vendor-api.ts)

Membership is a MEASURED property of a site, not a hand-written row, so it is read from vendor-clusters.json rather than municipalities.ts — hand-listing 69 municipalities would recreate exactly the per-city sprawl raw.qc_minutes_target_registry exists to end. The tier sits FIRST in ingest.ts's chain (above the target registry): its evidence is a live-proven enumerable endpoint, a stronger claim than "a page we fetched had ≥ 2 PDFs", and every municipality it claims is one where the crawl path discovers zero by construction — so it can only ADD resolution.

bash prototypes/vendor-api-sweep.sh                                 # whole cohort, read-only --plan
bash prototypes/vendor-api-proof-all.sh 4 17005 19010 11055 12005   # real downloads, no DB

MEASURED, whole cohort, --plan 2026-07-30 (VENDOR-API-ALL, 69 targets):

alreadyKnown: 0 — all 7,614 are net-new. For context the whole corpus is 272,322 documents from 942 municipalities, and the Weblex cohort contributed 15 of them before this. 37 of the 69 targets now yield documents where the crawl path yielded zero.

The 32 that still yield nothing are honest, not hidden: ~16 declare a doc-list whose advertised count is genuinely 0 (the town has published nothing to it), and ~16 declare no handle on any page the sitemap tier reached. Both print a named reason per run. That residual is the next piece of work, not a silence.

ao.ca / MRC Abitibi-Ouest "Inforoute" — NOT CRACKED, and never will be

robots.txt (Cloudflare-managed) carries User-agent: * / Allow: / and separate Disallow: / groups for ClaudeBot, GPTBot, CCBot, Amazonbot, Applebot-Extended, Bytespider, Google-Extended, meta-externalagent, and CloudflareBrowserRenderingCrawler. AI-bot intent binds us (standing order; same verdict as docs/ops/2026-07-28-has-gaps-investigation.md). Recorded as terminal, the page was never fetched, and the browser was never pointed at it.

robots.ts cannot express this — it evaluates OUR token and reads these files as "allowed". So the rule is encoded as a pure, tested function, aiBotDisallowed() in vendor-cluster-scan.ts, and applied BEFORE the page fetch. Measured universe-wide: 46 targets are AI-bot-disallowed — a cohort that was previously invisible and would otherwise be re-crawled forever.

TOMBSTONE — generic-crawl.ts + tinker-sample.ts, deleted 2026-07-31

Both are gone from this directory. Recorded here rather than left to be rediscovered, and recoverable with git log --follow / git show <commit>^:apps/worker/minutes-harvester/generic-crawl.ts.

What they were. A tile-439 follow-on tinker: a heuristic French-QC link-scoring crawler (generic-crawl.ts, crawlMunicipality()) plus a 27-municipality dry-run driver (tinker-sample.ts). Their stated output workflow, in tinker-sample.ts's own header, was "a human copies this skeleton into municipalities.ts".

Why they were deleted, and it is not "unused code". Every premise they rested on has been retired by something better:

• The workflow is gone. municipalities.ts stopped being where a publication location is decided on 2026-07-29 — raw.qc_minutes_target_registry is the ONE table (ARCHITECTURE.md, 2026-07-29), and a discovery result now lands in raw.qc_minutes_location_probe as evidence, not in a hand-edited file.
• The function is gone. discover-archive.ts's discoverOne() is the live discovery path and is 4x the size; it is what the 2026-07-31 gap-closure run used to repair 42 of the 77 live-proof-negative municipalities. Two crawlers answering "where does this municipality publish?" is the same two-answers defect the ONE-registry decision exists to end.
• It was the last unbounded read left in the worker, and the 2026-07-31 pull-readiness pass named it as such: raw fetch rather than politeFetch, and a bare res.text() with no cap. Hardening it would have meant porting the polite/bounded stack into a module nothing calls.

Nothing imported them but each other. crawlMunicipality had exactly one importer, tinker-sample.ts; tinker-sample.ts had none — no package.json script, no workflow, no doc pointing at it. The one remaining occurrence of the string, adapter: 'generic-crawl' in registration-count.test.ts, is a test fixture's arbitrary label, not a reference to the module.

The registry (municipalities.ts)

One entry per censused municipality. Two confidence tiers, both real inventory (Felix's directive is "pull EVERY municipality" — the registry covers all 35 + Gatineau):

• verified: true (8 entries) — seeds were derived from an ACTUAL live fetch of the archive page during this build (2026-07-13) and the PDF hrefs were inspected by hand: Drummondville, Rimouski, Beloeil, Vaudreuil-Dorion, Mascouche, Saint-Eustache, Sainte-Julie, Saguenay (the last two copied verbatim from apps/worker/permit-doc-parser/city-configs.ts, already proven there). Safe to dispatch as-is.
• verified: false (27 entries) — a real portal URL (root domain, or the actual conseil/séances page where hand-navigation found it) but the exact PDF-link include regex was NOT live-derived in this pass (seeds: []). Dispatching one of these with --plan is safe (read-only) and is exactly how a future session completes the entry: a --plan that discovers 0 PDFs is the honest signal to go add a real seed (mirror a verified: true entry above), never a silent wrong write. notes on each stub carries the Stage-1 census's own hint.

Hard cases (5, flagged not silently dropped): Repentigny + Shawinigan (scanned-ocr — image-only PDFs, this worker's OCR path still ACQUIRES them fine, text_method reads ocr/mixed), Granby (waf-blocked — every automated fetch 403s despite permissive robots.txt, needs a headless-browser fetch strategy this worker does not implement), Trois-Rivières + Victoriaville (js-portal — minutes sit behind a JS document-search widget, no link-guessable index page; each has a documented possible side-channel in its notes). All five are follow-up SCALE tiles, not silent gaps.

Measured (2026-07-13, MUNICIPALITY=ALL --plan against the live pooler, real counts)

Full MUNICIPALITY=ALL --plan run also correctly reported the 27 registry stubs by name (each printing its census hint / "needs a real seed" note) and the 5 hard cases by name + reason — one process pass over the ENTIRE 35-town

• Gatineau registry, zero crashes, zero writes.

These are discovery counts (candidate PDF urls found), not yet acquisition counts — see "Proof-pull" below for the 3 municipalities actually downloaded + hashed + stored + would-be-registered end to end. The scale here (hundreds of documents per city, not the 1-2 the Stage-1 census sampled) is the concrete answer to "is full history real" — yes, and it is far bigger than a spot-check suggested.

Proof-pull (2026-07-13, --local-only, real files on disk — not just written, opened)

Three municipalities acquired end to end (discover → download → sha256 hash → OCR/text-extract → store → would-be-registration record in a local manifest — everything --i-am-felix does except the Blob upload + the real DB write):

INV-4 resumability demonstrated live, not just described: re-running the IDENTICAL Vaudreuil-Dorion --max 15 command a second time reported already known: 15 (skipped), net-new: 781 (down from 796), and acquired the NEXT 15 documents — proving both (a) a re-dispatch skips everything already held and (b) it makes forward progress rather than reprocessing the same content. The local manifest after both runs holds 30 entries, 30 distinct content_hash values, 60 files on disk (30 PDFs + 30 cached .txt siblings).

Content verified, not just byte-counted: the cached text of a Beloeil document contains real itemized council content — grep -i "dérogation" <cached .txt> returns:

12. AUDITION DES PERSONNES INTÉRESSÉES ET AUTORISATION D'UNE DEMANDE DE DÉROGATION
        CONSIDÉRANT la réception d'une demande de dérogation mineure (DM-2026-9037) pour la
dérogations mineures;
        D'autoriser la dérogation mineure numéro DM-2026-9037 telle que demandée pour le

— confirming the Stage-1 census's finding (itemized, addressed dérogation- mineure records) survives the actual harvest, not just the sample read.

A real bug the proof-pull caught (documented, not hidden): Vaudreuil- Dorion's archive page declares <base href="/" />; the first pass ignored it and resolved every relative href against the wrong path segment, so all 15 downloads 404'd. extractLinks() now honors a page's own <base> tag — exactly the kind of defect a real end-to-end proof-pull surfaces that a regex-only "looks right" read would not.

Storage choice: Cloudflare R2 (moved 2026-07-31)

storage.ts banks every document to Cloudflare R2 via the S3-compatible API (@aws-sdk/client-s3), the house object store (ARCHITECTURE.md invariant 8). It previously used Vercel Blob; the ratified reasoning for the move is the ARCHITECTURE.md ## Decisions entry "The municipal document corpus lives in R2, and its address is derived, never parsed".

Why the old justification failed, kept because it is the lesson. The Blob section that stood here argued the choice on a stated estimate — that the corpus "even at full 35-municipality/full-history scale, is a few GB … nowhere near the S3-vs-Blob pivot point (province-scale, hundreds of GB)". Measured live against raw.qc_minutes_documents on 2026-07-31:

Three orders of magnitude past the estimate, and past the very pivot point the old text named as the reason to move. The estimate was not wrong when written — it was never re-measured. A capacity assumption written into a design comment is a claim with an expiry date. Blob is also egress-billed on a corpus the OCR / redate / metadata repair passes re-read whole, and R2 is not.

Config

The corpus lands in ortova-geo/qc-minutes/, a sibling of the lidar/, imagery/, overture/, and minutes-harvest/ prefixes already in that bucket (so it is already not geo-only — "geo" is a legacy string in the overwatch sense). It is not a dedicated bucket because the live R2 API token is scoped to this one: ListBuckets returns AccessDenied. A corpus bucket needs Felix to create it and mint a token — a gate, not a code change — and MINUTES_R2_BUCKET makes that later move a one-value change, because the key does not contain the bucket.

The key did not change, and that is load-bearing

documentKey() still yields qc-minutes/<municipality_code>/<sha256>.<ext> — byte-for-byte the path Blob served under. Verified against the live table: 191,262 / 191,262 object_uri paths and 225,776 / 225,776 text_object_uri paths equal documentKey(municipality_code, content_hash, ext) exactly. Because of that:

• the Blob → R2 backfill is a pure copy with nothing to remap;
• the DB rewrite (migration 0486) is invertible, so it can prove itself lossless at apply time instead of asking to be trusted;
• the dual read is exact rather than a guess.

Dual read (the transition seam)

Reads go through readObjectBytes(uri) / readObjectRange(uri, start, end) / readObjectSize(uri). Each accepts either URI shape and, on a miss, tries the other store at the same key. Both directions matter: copying objects and rewriting stored URIs are two operations that cannot be atomic with each other, so during the transition a row's address and its bytes' actual home can disagree either way round.

Never fetch() a stored object_uri. R2 is private — a bare fetch of an r2:// URI resolves nothing, and a public HTTPS corpus is precisely the exposure class the 2026-07-30 exposure-registry decision exists to prevent. The three repair scripts (recover-text.ts, recover-pdf-metadata.ts, redate-bodytext.ts) were rewired onto the seam for this reason.

index-only://robots-blocked-pdf-host/<hash> (81,062 rows, bytes = 0) marks a document whose PDF was never fetched. readObjectBytes() returns null for it: an absence, not a fault, and not an address.

Moving the corpus

• Smoke proof (writes + deletes a handful of throwaway objects): bash scripts/with-env.sh npx tsx apps/worker/minutes-harvester/scripts/smoke-r2-store.ts
• Backfill: blob-to-r2-migrate.ts — resumable, idempotent, hash-verified per object, no DB write. Staged on the fleet as .github/workflows/minutes-blob-to-r2.yml. Its work list is URI-BLIND and must stay that way. It selects on durable facts (object_uri NOT LIKE 'index-only://%', text_object_uri IS NOT NULL) and derives every key from (municipality_code, content_hash), so it picks the same set before, during, and after 0486. The first version keyed on URI shape; 0486 was applied first, and run 30637592942 exited success on settled 0 / 0 with all 179 GB still in Blob. blob-to-r2-migrate.test.ts pins it (red-before proven), and a false-green guard fails any run that reports "nothing to do" without confirming a sample of keys in R2.
• DB rewrite: migration 0486 (DRAFT). Apply only once the copier reports mismatch = 0, source-missing = 0, errors = 0, settled = corpus total.

Blob is not retired until that report is clean. The dual-read seam carries reads until then, so there is no window in which anything 404s.

--local-only writes to local disk instead (explicitly durable: false in storage.ts) — this is proof-mode only, never how the real harvest runs.

Idempotent + resumable (INV-4), by construction

• Resume-check (knownUrlsFromDb): before fetching ANY document byte, the worker reads every source_url (+ also_seen_at_urls alias) already registered for this municipality_code — a free SELECT (Rule 6) — and skips re-downloading/re-uploading/re-writing anything already held. A re-dispatch after a mid-run crash resumes exactly where it left off. --local-only uses a local manifest.json for the same purpose (so the proof-pull is resumable too, without a DB).
• content_hash is the DB's own idempotency key (UNIQUE constraint, migration 0244) even if the URL-skip were somehow bypassed: ON CONFLICT (content_hash) DO UPDATE never inserts a duplicate document row — it only ever appends an alias URL to also_seen_at_urls when the SAME bytes are found at a NEW url (a doc re-posted under a new path).
• Chunked writes: DOCS_PER_INSERT = 50 — the register step never issues one whole-municipality INSERT, even for Vaudreuil-Dorion's 796 documents (mirrors the geo/permit ingesters' per-batch-transaction discipline, never a province-size single statement).
• 429/5xx backoff: fetchTextBackoff / fetchBytesBackoff retry up to 5 times with exponential backoff (Retry-After honored on 429, capped at 30s), on both the index-page crawl and the document download. A 4xx other than 429 (404, 403) is treated as permanent and NOT retried — that document is recorded as failed, the run continues.
• Silent-0 guard: a --i-am-felix run against every attempted municipality (one with seeds.length > 0) that discovers 0 candidates on ALL of them fails the job (main()'s zero-yield check) — a broken regex or a site change must never exit green as "nothing to harvest."

Deploy config (drafted for Felix — GitHub Actions, no Railway)

Per the 2026-07-13 build-model decision (docs/architecture/cloud-execution-paths.md, overwatch.2026-07-13.001) — GitHub Actions is the default sanctioned batch runner; do not provision new Railway services. This worker therefore has no railway.json / Dockerfile — .github/workflows/minutes-ingest.yml IS its deploy config:

# one municipality at a time, plan first (always safe, read-only):
gh workflow run minutes-ingest.yml -f municipality=Beloeil -f mode=plan
gh run watch

# then the real harvest (Felix-gated — Release Agent only):
gh workflow run minutes-ingest.yml -f municipality=Beloeil -f mode=i-am-felix

The workflow installs poppler-utils + tesseract-ocr (+ fra/eng models)

• postgresql-client directly on the ubuntu-latest runner (no Docker image build), npm installs this worker's own package.json (it is NOT a pnpm-workspace member — see the repo-root pnpm-workspace.yaml comment — so a plain npm install here mirrors geo-ingester/tile-builder), then runs --plan (gate) then --i-am-felix (only when mode=i-am-felix).

Ordered plan for the Release Agent (after Felix approves the migration)

1. Apply migration 0244: scripts/apply-migration.ts --i-am-felix migrations/0244_qc_minutes_documents.sql (raw table + the one catalog row; no canonical satellite — see the migration's header for why).
2. Set secrets (all already present — the R2 four from lidar-acquire, the pooler URL and the Blob token from the earlier Blob era): SUPABASE_POSTGRES_POOLER_URL, R2_ENDPOINT, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET on the repo's GitHub Actions secrets. BLOB_READ_WRITE_TOKEN stays set until the Blob→R2 backfill reports clean — it is the dual-read seam's fallback leg, not a write credential any more.
3. Dispatch the 8 verified: true municipalities first, one at a time, mode=plan then mode=i-am-felix: Drummondville, Rimouski, Beloeil, Vaudreuil-Dorion, Mascouche, Saint-Eustache, Sainte-Julie, Saguenay.
4. For each of the 27 verified: false stubs: dispatch mode=plan first. A nonzero discovery count means the stub's portal URL was enough to at least reach a page (still confirm the regex is right before i-am-felix); a zero count means the seed needs a real regex — mirror one of the 8 validated entries in municipalities.ts (fetch the stub's feed_url, find the real archive page + PDF link pattern, same 15-30 line pattern as the entries above) before dispatching i-am-felix for that town.
5. The 5 hard cases (Repentigny, Shawinigan, Granby, Trois-Rivières, Victoriaville) need a follow-up SCALE tile (a headless-browser fetch strategy) before they can be dispatched at all — flagged, not silently scheduled.
6. Verify: SELECT municipality_code, count(*), min(fetched_at), max(fetched_at) FROM raw.qc_minutes_documents GROUP BY 1 ORDER BY 1; (read-only, Rule 6) after each dispatch — the same silent-0 discipline permit-ingest.yml uses.

OCR-recovery backfill (recover-text.ts)

A held PDF's bytes are durable in Blob (object_uri), but a scan-only PDF whose text extraction never ran (no tesseract at harvest time, or a tool degrade) sits with text_method='unavailable' — its dates and governance facts stay invisible even though the document is already in hand. recover-text.ts re-runs extract-text.ts (poppler + tesseract -l fra+eng) against the ALREADY-STORED bytes — no re-fetch from the municipality — and UPDATEs the row + writes a Blob text sibling. Same shape as recover-pdf-metadata.ts (resumable, batched, the predicate itself is the progress marker).

Also catches the mislabeled text_method='text-layer' AND text_object_uri IS NULL fingerprint (a bug in extract-text.ts's page-split, fixed 2026-07-23 — see that file's extractText() header comment) — a fully-scanned PDF whose pdftotext output was ALL formfeeds had every page silently dropped by a looping trailing-empty-pop, so OCR never ran and the row was mislabeled "recovered" with zero actual text.

tesseract --list-langs   # must show `fra` and `eng` — brew install tesseract-lang
                          # (slow bottle) or curl the single traineddata file:
                          # curl -fsSL https://github.com/tesseract-ocr/tessdata_fast/raw/main/fra.traineddata \
                          #   -o "$(brew --prefix tesseract)/share/tessdata/fra.traineddata"

pnpm --filter @ortova/minutes-harvester recover-text -- --sample 8 --munis 66023     # dry-run, one muni
pnpm --filter @ortova/minutes-harvester recover-text -- --i-am-felix --munis 66023   # real backfill, one muni
pnpm --filter @ortova/minutes-harvester recover-text -- --i-am-felix                 # full lane, all munis (long — CPU-bound OCR, ~50-60 rows/min at CONCURRENCY=6 on median-size docs; large multi-MB scans are much slower)

Target lane (municipalities holding at least one affected doc, ordered by population): docs/ops/minutes-lanes/ocr-recovery-target.tsv (measured directly off raw.qc_minutes_documents + serving.qc_municipality_directory, 2026-07-23 — 542 municipalities, ~30k affected docs total across both predicates). RECOVER_TEXT_CONCURRENCY / RECOVER_TEXT_BATCH env vars tune throughput; OCR is CPU-bound (tesseract + pdftoppm), not I/O-bound like the metadata backfill, so concurrency should track real cores, not be pushed as high as recover-pdf-metadata.ts's I/O-wait CONCURRENCY=24.

Recovering text does NOT itself backfill session_date for documents whose date was never resolvable from the filename/URL (e.g. Montréal's sel/adi-public portal — ALL 999 of its docs carry session_date IS NULL, independent of OCR status). That is a separate reclassification pass reading the newly-recovered text content, out of this script's scope.

Path to Stage 3 (future, NOT this worker's job)

apps/worker/permit-doc-parser already prototypes the itemization step (HARVEST → EXTRACT → JOIN → MEASURE) for a DIFFERENT 10-town list from an earlier research pass — its extract-records.ts (French-language DM/PIIA/ démolition record extraction) and join.ts (address/lot resolve to property_id, read-only) are the shape a Stage 3 pass over THIS worker's 35-municipality corpus would reuse. Wiring that up — reading raw.qc_minutes_documents, running the extractor, designing the canonical satellite this ONE table currently deliberately omits — is a SPINE (Zone 4) tile, not a SCALE one: SCALE runs the canonical-upsert at scale, it does not design what it writes.

apps/worker/permit-doc-parser/README.md

permit-doc-parser — lossless permit extraction from QC municipal PDFs

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: LLM extraction of structured permit records + feed classification from municipal PDFs.
• Input/output boundaries (Layer-1): PermitRecordSchema, DocExtractionSchema, CompletenessReportSchema, FeedClassificationSchema — rich Zod already in types.ts (boundary table §C).
• Run locally: pnpm --filter @ortova/permit-doc-parser parse (also classify, pipeline).
• Deploy: container / CLI on the ortova-mac fleet.

Most Québec municipalities do not publish permits as a data feed. Of ~40 cities surveyed, only 4 have a machine-readable feed (Montréal, Québec City, Laval, Longueuil — see docs/research/qc-permits/2026-05-28-municipality-coverage.md). The rest publish permit decisions inside French-language PDFs: council procès-verbaux (minutes), avis publics (public notices), and some permit-statistics reports. Those PDFs list address-level dérogation mineure, PIIA, démolition, construction, lotissement, and changement d'usage cases — the exact permit events the spine wants.

This worker turns those PDFs into structured permit records, and classifies each municipality's publication as a candidate feed.

• Decision it informs: do the ~14 QC towns previously judged "closed" become tappable permit feeds via document parsing?
• Action it unblocks: expanding QC permit coverage beyond Montréal / Laval / Québec by parsing council minutes + avis-publics PDFs.

It lives in apps/worker/ because the heavy work is PDF text extraction + OCR (the same reason geo-ingester / tile-builder are here). This is parser logic — it is NOT yet wired into the engine dispatchers. The output is a clean canonical.property_permit-shaped superset that a future source adapter feeds through the canonical-upsert; see "Path to engine wiring" below.

The full pipeline (BUILD 3, below) is HARVEST → EXTRACT → JOIN → MEASURE, per city, driven by city-configs.ts. Scaling to the other ~10 closed towns is adding a config, not writing code. Jump to "BUILD 3" for the runbook + the two-city end-to-end numbers.

BUILD 1 — the lossless extraction parser

Input: one municipal PDF (French). Output: a complete structured set of permit records + a completeness proof.

Pipeline (parse.ts → the modules):

1. Acquire, raw-native-first (fetch-doc.ts). Fetch the URL (or read a local path), hash it, and store the verbatim PDF + the verbatim extracted text under out/ before any structured extraction. Extraction derives over the stored raw, never the other way round (data-shape canon P2 / feedback_bought_imagery_persists_to_db).
2. Full-text extraction, every page, never capped (extract-text.ts). pdftotext -layout for the native text layer, split per page on the form-feed poppler emits. Pages whose text layer is empty are detected as scanned and OCR'd individually (pdftoppm -r 300 → tesseract -l fra+eng). No token or page limit anywhere; long docs are handled page-by-page so nothing is dropped.
3. Doc classification (classify-doc.ts): doc_type (proces-verbal | avis-public | ordre-du-jour | permit-stats) + doc_date from the filename/header.
4. Record extraction (extract-records.ts, behind the Extractor seam in llm-seam.ts). An anchor model reads both document shapes:
   • procès-verbal / ordre-du-jour: each item anchored by a demand id (2025-DM-289, 2026-PIIA-006, ranges …225 à …228 expanded), a numbered urbanism line with a locus, or a resolution number (CM26 02 064, 2025-11-17-29). Agenda + resolution mentions of the same demand merge.
   • avis public: one subject per notice — objet + IMMEUBLE VISÉ carry the locus; the meeting-venue address in the body is explicitly ignored.
5. Completeness proof (completeness.ts) — see below.

The superset schema (types.ts)

Every field is optional; anything unexpected lands in a raw_fields bag, so nothing valuable is truncated:

municipality, source_url, doc_type, doc_date,
item_type (derogation-mineure | piia | demolition | construction | renovation
           | lotissement | usage | other),
address, lot_number, matricule, cadastre, resolution_number, applicant,
declared_value, units, description, demand_id,
raw_excerpt (verbatim), page_no, extraction_confidence, raw_fields{}

The losslessness proof (the gap check)

completeness.ts scans the raw lines independently of the extractor for every permit signal (civic address, permis, dérogation, démolition, PIIA, lotissement, construction, usage, demand-id) and classifies each:

• matched — the line is inside an extracted record's source span.
• reviewed-not-permit — a by-law / procedural / venue line, or a signal word with no civic locus (a real permit is always locatable; discussion prose is not). E.g. "Règlement de démolition numéro 334" is a by-law, not a permit.
• UNMATCHED — a locatable permit signal the extractor missed. unmatched > 0 is a gap = a bug and is reported verbatim.

lossless = (unmatched === 0). Every record is also re-validated against the superset Zod schema before it is emitted.

BUILD 2 — the feed classifier (classify-feed.ts)

Per municipality, from its sampled docs:

refresh_rate (monthly | per-council-session | irregular | unknown)   ← from doc dates
format_consistency (structured-table | semi-structured | prose | scanned-image)
doc_types_published[]
permit_types_present[]     ← THE most important field: taxonomy ACTUALLY found
joinability (address-level | aggregate-only | mixed | none)
est_volume_per_period, parse_confidence, docs_sampled, notes

parse_confidence is discounted when any sampled doc failed the completeness check, so a lossy parse can never masquerade as a trustworthy feed.

BUILD 3 — the end-to-end pipeline (HARVEST → EXTRACT → JOIN → MEASURE)

pipeline.ts runs the whole chain per municipality, driven by city-configs.ts:

1. HARVEST (harvest.ts) — a generalized municipal-PDF crawler. From a city's seed archive URL(s) it discovers every permit-relevant PDF (an include/ exclude regex on the link), downloads the verbatim bytes (raw-native- first), and writes a manifest.json of {url, doc_type, doc_date, sha256, bytes, local_path}. Idempotent + resumable (keyed by URL; a re-run skips held docs; kill it mid-run and re-run), content-deduped (identical bytes under a new URL collapse to one file, both URLs recorded), and polite (rate_limit_ms between fetches). Both proven cities list their whole multi- year archive on one index page, so no pagination is needed; follow_index handles deeper archives when they paginate.
2. EXTRACT — the existing lossless parser (parse-one.ts) over the whole harvested corpus; the independent completeness scan gates each doc.
3. JOIN (join.ts) — resolve each addressed record to canonical.property_id, READ-ONLY, reusing the spine's OWN keys: Tier 1 deterministic address_key = canonical.address_key(addr) AND mamh_code = canonical.mamh_of(muni) (the exact QC join the live permit sources use, DR-190), Tier 2 pg_trgm similarity ≥ 0.82 scoped to the city. Batched: a whole city is 2 queries, not one-per-record.
4. MEASURE — per city: PDFs harvested, records extracted, completeness gaps, join-rate to property_id, permit-type mix. Writes report.json + property-permit-load.jsonl (the load-ready canonical.property_permit shape).

# full pipeline, all configured cities (harvest full archive → extract → join → measure)
pnpm --filter @ortova/permit-doc-parser pipeline
#   one city:            pnpm --filter @ortova/permit-doc-parser pipeline -- "Mont-Tremblant"
#   bounded proof run:   … pipeline -- "Saint-Georges" --max 8
#   re-measure offline:  … pipeline -- --offline           (reuse the on-disk manifest)

# harvest only (populate/refresh the manifest, download nothing twice)
pnpm --filter @ortova/permit-doc-parser harvest -- "Mont-Tremblant"

Per-city output lands under out/<city>/: raw/ (verbatim PDFs), manifest.json, extractions/<slug>.json, join.json (per-record verdicts + summary), property-permit-load.jsonl (load-ready, one canonical.property_permit row per record), report.json. A top-level out/pipeline-summary.json rolls up all cities.

What the pipeline proves end-to-end (2026-07-02, FULL archives)

570 extracted records attach deterministically to a real canonical.property_id (Tier-1 address_key, confidence 1.0) — the proof that PDF-only towns land on the spine. pnpm --filter @ortova/permit-doc-parser pipeline reproduces it.

Honest rough edges the scale run surfaced (the fixture was clean; a 5-year archive is not):

• Completeness is NOT uniform at scale. 126/221 MT docs are lossless; the other 95 produce 423 gap lines (402 addresses, the rest construction/permis/ lotissement). The check is doing its job — flagging that the demand-id anchor model misses addresses that live in resolution bodies / prose of older & agglo PVs (e.g. "…au 1303-1305, rue Labelle…"). Closing these is the next parser pass; the gap report names every missed line.
• Address extraction truncates on line-wrapped streets — "1001 rue de" instead of "1001 rue de Saint-Jovite" when pdftotext wraps the street across a line. ~168 of the 404 distinct MT misses are this shape.
• Genuinely unjoinable-by-address: 159 street-only refs (a chemin with no civic number) + 76 civic ranges ("2971-3067 chemin de la Chapelle" — the spine stores single civics). These need the cadastre/lot→parcel join (pilot-only), not an address fix.
• Lot "derrière le" case: an avis whose subject is a lot behind a civic address joins to the reference address, not the true parcel (semantic, not a bug).

Two province-general normalizations were added at the join boundary (join.ts normalizeForKey, exactly what a source adapter's normalize.ts does): strip trailing procedural clauses ("…en vertu du Règlement") and fold a trailing cardinal to the roll's single letter ("Avenue Nord" → "N", stored "AVENUE N"). Those lifted Saint-Georges from 60% → 100% of addressed on the bounded run.

Adding a city (the config format — this is how it scales)

A new town is ~15 lines in city-configs.ts, no code:

{
  municipality: 'Sainte-Adèle',
  municipality_code: '78030',   // MAMH code — the JOIN key (canonical.mamh_of arg). VERIFY it:
                                //   SELECT count(*) FROM canonical.property WHERE mamh_code='78030';
  feed_url: 'https://…/proces-verbaux',   // provenance
  rate_limit_ms: 800,
  seeds: [{
    doc_type_hint: 'proces-verbal',
    index_urls: ['https://…/proces-verbaux'],      // the archive listing page(s)
    include: /\/pv[^"]*\.pdf$/i,                    // which links are permit PDFs
    exclude: /reglement|budget/i,                   // optional — parser is the real gate
  }],
}

If the new town uses a French phrasing the taxonomy misses, that is a taxonomy.ts extension (province-general) — never a per-city hack. The join key is always the MAMH code; verify it returns the town's properties before trusting its join rate.

Run it (parser-only, offline)

# whole committed corpus, offline (uses the fixtures/*.pdf)
pnpm --filter @ortova/permit-doc-parser corpus
#   add --online to also fetch the remote-only siblings (Saguenay, La Pocatière)

# one document (URL or local path)
pnpm --filter @ortova/permit-doc-parser parse -- <url|path> --municipality "Mont-Tremblant"

# fixture-driven tests (offline, via poppler)
pnpm --filter @ortova/permit-doc-parser test
pnpm --filter @ortova/permit-doc-parser typecheck

Outputs land under out/: raw/ (verbatim PDFs), text/ (verbatim text), extractions/<slug>.json (per-doc DocExtraction), corpus-summary.json.

System deps: poppler (pdftotext, pdftoppm) + tesseract with the fra model. The Dockerfile installs them. Locally on macOS: brew install poppler tesseract tesseract-lang.

What the corpus run proves today (2026-07-02)

Four committed fixtures, real docs pulled from the live sites:

4/4 docs lossless, 18 records. Feed classification:

• Mont-Tremblant → address-level, structured-table, permit types [derogation-mineure, piia], ~15 items/session, parse_confidence 1.0. This is the strongest "closed → tappable" result: a rich, joinable permit feed hiding in the minutes.
• Saint-Georges → address-level, avis publics carry one dérogation each, address in the notice (often in the filename too). A clean, low-volume feed.
• Richmond → address-level but this session was almost all by-law work; the parser correctly extracts nothing spurious and proves it lost nothing.
• (--online) Saguenay / La Pocatière parse cleanly to 0 records on the sampled docs — the sampled sessions had no address-level urbanism items; worth sampling more dates before a verdict (feedback_never_claim_exhausted).

What works vs what's still rough

Works (proven by the run + tests):

• Full-document, every-page text extraction with per-page OCR fallback; no cap.
• Raw-native-first persistence (verbatim PDF + text) before extraction.
• The demand-id-anchored PV extractor (Mont-Tremblant is essentially perfect).
• The avis-public single-subject reader (venue address correctly rejected).
• The independent completeness proof; 4/4 fixtures lossless.
• The superset schema + raw_fields catch-all; Zod-validated on emit.

Still rough / not yet done:

• OCR is untested on a truly scanned doc — all four fixtures have text layers. The code path exists and is guarded, and the local tesseract has no fra model (would run eng with a logged confidence penalty; the Dockerfile adds fra). A scanned fixture is the next test to add.
• permit-stats tier is stubbed — classified but no aggregate-number extractor yet (Trois-Rivières / Gatineau bulletins). The seam is there.
• applicant and declared_value rarely appear in these docs and are lightly covered; extend taxonomy.ts as more phrasings surface.
• Coverage is a line-span heuristic, not a semantic guarantee — a permit whose address failed to parse would surface as an UNMATCHED gap (correct), but a malformed record could over-cover. The completeness check is the backstop, not a proof of per-field correctness.
• Not wired into the engine (deliberate — see below). No DB writes, no migration. Felix-gated.

The LLM seam (llm-seam.ts)

Extraction is behind one Extractor contract. ruleBasedExtractor is the default and is what the corpus + pipeline runs prove. buildExtractionPrompt() is the exact prompt an LLM extractor would receive — the same superset schema, so the same completeness check + Zod validation adjudicate an LLM's output identically.

Checked 2026-07-02: no LLM key is present (the only key in .env.local is SUPABASE_POSTGRES_POOLER_URL; no ANTHROPIC_API_KEY / OPENAI_API_KEY). So the rule engine stays the default and llmExtractor throws until a provider is wired.

The exact wiring (drop-in, when a key lands — the seam, not a rewrite): add @anthropic-ai/sdk to package.json, put ANTHROPIC_API_KEY in the worker env, and replace llmExtractor.extract in llm-seam.ts with:

import Anthropic from '@anthropic-ai/sdk';
const anthropic = new Anthropic(); // reads ANTHROPIC_API_KEY

export const llmExtractor: Extractor = {
  name: 'llm-claude-v1',
  async extract(pages, meta) {
    const msg = await anthropic.messages.create({
      model: 'claude-opus-4-8',       // or claude-haiku for the cheap first pass
      max_tokens: 8192,
      messages: [{ role: 'user', content: buildExtractionPrompt(pages, meta) }],
    });
    const json = (msg.content[0] as { text: string }).text;
    const raw = JSON.parse(json.slice(json.indexOf('['), json.lastIndexOf(']') + 1));
    // adjudicate identically: parse each through PermitRecordSchema; derive
    // covered_lines from each record's raw_excerpt against the page text, then
    // checkCompleteness() gates it exactly as it gates the rule engine.
    return coverFromExcerpts(raw, pages, meta);
  },
};

pickExtractor('llm') then selects it (pipeline run … --extractor llm). The one new helper, coverFromExcerpts, maps each returned raw_excerpt back to its source line indices so the independent completeness scan can still prove losslessness — that is the whole point of the seam: the adjudicator does not change.

Path to engine wiring (the next tile, Felix-gated)

Follow docs/architecture/source-onboarding-kit.md: one pdf-kind source per municipality (or one parameterised source keyed by municipality) whose adapter yields these records as native raw rows into raw.<muni>_permit_docs; the canonical-upsert then resolves each address / lot_number to property_id and lands canonical.property_permit — the identical shape every existing permit city already writes. The parser output here is deliberately that shape so the adapter is thin. The bulk PDF fetch runs cloud-side in this worker; the migration apply + canonical join stay Felix-gated.

Provenance

The four fixtures were fetched 2026-07-02 from the live municipal sites; each carries its live URL in corpus.ts (feedback_source_attribution_always). Documents are public municipal records. When a feed is onboarded, its meta.dataset_catalog.source records authority + license + the per-doc URL.

apps/worker/permit-ingester/README.md

permit-ingester — cloud bulk-ingest runtime for the permit canonical join

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: cloud-side bulk canonical-upsert of a permit feed (config-driven, resumable).
• Input boundary (Layer-1): WorkflowArgsSchemas['permit-ingest'] = { source_key, phase? } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema when Watch-driven.
• Output: canonical permit upsert (platform config-driven). DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/permit-ingester plan / ingest.
• Deploy: gh workflow run permit-ingest.yml -f source=<key> (runs-on ortova-mac).

A run-to-completion worker that drives the permit canonical-upsert (and an optional resumable raw sweep) from a box sitting next to the database, instead of from a laptop over home wifi.

Why it exists (worksite tile 213, Zone 5 SCALE)

The canonical-upsert dispatcher is sound — keyset paging + a single LEFT JOIN identity resolve, flat ~9 MB heap at scale (tile 205). The cost of finishing a big-city join is network: Montréal's 554k permit rows are ~hundreds of thousands of round-trips to the Supabase pooler, which over home wifi is hours of pure latency tax — and the pooler's forced 2-minute statement_timeout turns any straggler query into a hard failure. The durable home is a worker near the DB. Same posture as the geo-ingester / tile-builder siblings.

It is also the runtime the national permit feeds (Calgary, Toronto, Edmonton…) ingest through: every permit city lands the same canonical shape on the same property subject, so one source-parameterised worker serves them all.

What it runs

--plan is read-only: prints the raw/canonical counts and what would run.

It does not re-stamp rows into synthetic chunk run_ids. Tile 205's keyset paging made each batch an O(limit) seek regardless of a run_id's size, so the old "split one big run_id into 5k-row chunks" dance is obsolete — and that dance also tripped raw.<source>.run_id's FK to adapter_run_log and an accidental string-vs-bigint chunk loop in the laptop script. Running over the existing run_ids sidesteps both.

Run it

# read-only preview
SOURCE_KEY=montreal-ckan-permits tsx ingest.ts --plan

# finish the Montréal canonical join (Felix-gated write)
SOURCE_KEY=montreal-ckan-permits tsx ingest.ts --i-am-felix

# a fresh national city: raw sweep then canonical
SOURCE_KEY=calgary-socrata-permits tsx ingest.ts --i-am-felix --phase all

Env: SOURCE_KEY (one of the registered permit sources in ingest.ts) and SUPABASE_POSTGRES_POOLER_URL. Writes require --i-am-felix (Rule 6).

Deploy (Felix-gated)

This worker runs the TypeScript engine, so unlike the self-contained siblings the Docker build context is the repo root, not this dir.

Build model (decided Felix 2026-07-08): git-connected, not laptop-uploaded. Data-ingestion workers under apps/worker/** build their Railway image directly from GitHub main — the deployed image is always merged main, never an arbitrary laptop checkout. The prior model (manual railway up uploads) is exactly what let the typed-watermark fix (commit cabdcca) sit merged on main while the running service kept executing pre-fix code (source: null on the service — no gate between "merged" and "running"). See scripts/ops/release-worker.sh for the run-trigger vehicle.

Root Directory is . (repo root) for THIS service — not the subdir convention the self-contained siblings use. This worker's Dockerfile COPYs packages/, apps/, data/, and scripts/ from repo-root-relative paths (it needs the whole pnpm workspace to run the TS engine), so Root Directory MUST stay the repo root or those COPYs resolve to nothing and the build fails. dockerfilePath stays repo-root-relative (apps/worker/permit-ingester/Dockerfile) to match. Path-filtering the rebuild trigger to only apps/worker/** (like the self-contained siblings) would be WRONG here and would silently reintroduce a narrower version of the exact staleness bug this build model exists to kill — permit-canonical-upsert.config.ts lives under apps/core/, not apps/worker/permit-ingester/, and the Dockerfile copies packages/ and scripts/ too. railway.json's build.watchPatterns is therefore scoped to everything this Dockerfile actually depends on (apps/worker/permit-ingester/**, apps/core/**, packages/**, scripts/**, data/**, the root workspace manifests) — narrower would be unsafe, wider (no filter at all) would also be a legitimate, safer default if this list ever drifts from the Dockerfile's real COPY surface.

1. One-time setup (Release Agent, per service): railway service source connect --repo <owner>/<repo> --branch main --service permit-ingester. Set Root Directory = . (repo root) and Dockerfile path = apps/worker/permit-ingester/Dockerfile (mirrors railway.json, which also carries the watchPatterns rebuild scope — no separate dashboard step needed for that part).
2. Pick a region close to the Supabase database — that proximity is the entire point.
3. Service variables: SOURCE_KEY, SUPABASE_POSTGRES_POOLER_URL. Optionally a start-command override for --phase/--plan, and CANONICAL_UPSERT_BATCH_SIZE to tune the canonical-phase chunk size. Set/update these via scripts/ops/release-worker.sh --i-am-felix permit-ingester SOURCE_KEY=... (never hand-set through the dashboard — the script logs evidence).
4. restartPolicyType: NEVER (it's a one-shot job; the canonical phase is idempotent so a manual re-run is always safe).
5. Triggering a run (every subsequent ingest, after step 1's one-time connect): bash scripts/ops/release-worker.sh --i-am-felix permit-ingester — pulls the latest commit from main and reruns. Raw railway up / railway redeploy are not an allowlisted agent path; release-worker.sh is the only sanctioned trigger (mirrors release-push.sh / release-deploy.sh).

The connect step, the variable sets, and every run trigger are Felix-gated (outward + spends money) — the Release Agent runs them on Felix's approved set.

Fast cohort load (tile 314) — the canonical-upsert throughput fix

The permit canonical join floored at ~140 rows/s (SF's 1.29M ≈ 2.5 h; NYC's 5M ≈ 10 h). Measured on prod (EXPLAIN (ANALYZE, BUFFERS), live SF load): the identity resolve — LEFT JOIN canonical.property ON (address_key, mamh_code) to fetch property_id — is ~85% of per-batch wall time (2,238 ms of ~2,600 ms / 500 rows), and 100% of that is the random heap fetch for property_id (not in the key index): 1,759 ms / 500 probes with 327 heap disk reads vs 2.9 ms / 0 when the identical probes stay index-only. The 253 MB key index fits t4g.small cache; the 1,334 MB heap does not, so every property_id fetch is a cold random read.

Two complementary levers make the cohort load fast (both preserve byte-identical join results — proven: 30,000 / 30,000 SF permits recompute to the exact persisted property_id):

1. Covering index migrations/0186_property_addrkey_mamh_covering_index.sql — (address_key, mamh_code) INCLUDE (property_id) → the resolve plans as an Index Only Scan; the 290 MB covering index stays cache-resident and the heap fetch disappears. Projected resolve 2,238 ms → ~155 ms / 500 (14×), lifting the server-side resolve ceiling from ~223 to ~3,000+ rows/s for the whole permit cohort (QC + US). Felix-gated apply (CREATE INDEX CONCURRENTLY, direct/ session connection — see the migration header for mechanics).
2. CANONICAL_UPSERT_BATCH_SIZE (default 500) — set to e.g. 2500 for the cloud-side cohort load. Larger WAL-safe chunks fold the same work into ~5× fewer round-trips + set-based statements. INV-4-safe: still chunked per-batch transactions (clamped ≤ 20 000 so it can never become a WAL-PANIC province-size INSERT), keyset-paged, idempotent, resumable. Result-invariant (batching is pure pagination over a deterministic join).

Order of operations for the redeploy: apply 0186 first (so the resolve is index-only), then bash scripts/ops/release-worker.sh --i-am-felix permit-ingester CANONICAL_UPSERT_BATCH_SIZE=2500 and re-run the canonical phase per source (idempotent — already-loaded rows hash-match and skip).

apps/worker/qc-mcp-tap/README.md

qc-mcp-tap — the Québec data tap

A Cloudflare Worker that serves the Québec municipal council-decision corpus to an outside collaborator's own AI client, over the Model Context Protocol, read-only.

The point is that one named person can experiment with the corpus — ask questions, page through decisions, check coverage — without a copy of the data being handed over and without any credential existing that could reach past the serving wall.

Status: DEPLOYED and LIVE at https://ortova-qc-tap.felixbosse.workers.dev. Migrations 0434, 0438, and 0439 are all applied (manifests in migrations/applied/2026-07-28-*.json); all eight tools and all three resources are live against production api.*.

What it is

The wall

This is the part worth reading before the code.

ARCHITECTURE.md invariant 5 says the api.* read-contract is PII-free by construction and api_reader holds zero grant on canonical.*. This Worker is built so that invariant holds even if every line of its logic were wrong.

The credential. The Worker's only database secret is the Supabase publishable (anon) key, bound as SUPABASE_API_READER_KEY. PostgREST authenticates as authenticator and immediately SET ROLEs to the key's role, anon. Verified live 2026-07-28:

pg_auth_members   api_reader ← anon           -- anon INHERITS api_reader
pg_namespace      api        nspacl = {postgres=UC/postgres, api_reader=U/postgres, ...}
                  canonical  nspacl has NO api_reader and NO anon entry

So anon's only path into the api schema is its membership in api_reader, and it has no path at all into canonical. The effective privilege envelope is exactly api_reader's.

Why not connect as api_reader? Because it cannot log in — pg_roles.rolcanlogin = f, verified 2026-07-28. Giving it a password is ALTER ROLE, a Felix-gated DB write, and it would buy nothing: the envelope is already api_reader's. Listed under Open calls in case Felix wants the role to be its own rather than inherited.

The reach. src/api-client.ts can construct exactly one kind of URL — /rest/v1/rpc/<name> under Accept-Profile: api, where <name> is parsed against ApiRpcNameSchema, a closed union of six function names. There is no table-read code path in this Worker. Adding one means adding a function that does not exist, in a reviewed diff. That is why this uses ~90 lines of fetch instead of @supabase/supabase-js, which ships .from() in the same object.

What is deliberately absent. No pooler URL, no direct connection string, no service-role key. They are not in wrangler.toml, not in the env schema, not in the deploy commands. A Worker cannot misuse a credential it was never given.

D68 / D28. Verbatim source text is excluded at emit time (migration 0425) and again at delivery time (0426, hardened by 0434). This Worker does not re-derive that protection and does not open a hole around it. Verified on live data — see Evidence.

Inputs and outputs, by schema name

Every hand-shaped boundary is a named Zod schema in src/schemas.ts.

Tool inputs — QcFeedPageInputSchema, QcFeedWatermarkInputSchema, QcActeLookupBaseSchema (the published shape) + QcActeLookupInputSchema (the enforced refinement), QcCoverageSummaryInputSchema, QcMuniProfileInputSchema. Shared values: CodeGeoSchema, EventKindSchema, IngestLaneSchema, FeedCursorSchema.

The api.* call boundary — ApiRpcNameSchema. This is the wall as a type.

Response envelopes — FeedEventSchema, FeedBatchResponseSchema, FeedWatermarkResponseSchema, FeedHeadResponseSchema, ActeLookupResponseSchema, CorpusLadderResponseSchema, MuniProfileResponseSchema. Payload interiors stay z.record(z.unknown()) on purpose: the DB owns that shape (invariant 4's passthrough clause), and mirroring the 45-key acte projection here would create a second copy that drifts.

Bindings — WorkerEnvSchema.

The eight tools

Five plumbing tools (read / sync / look up / coverage) plus three signal tools that turn "read the feed" into "explore a reveal" — filter by KIND × TYPE × GEOGRAPHY × TIME × STAGE, roll those up over time, and read where an instrument sits in its approval path. The reveal → query mapping and honesty constraints are in docs/product/qc/mcp-tap-toolset-design.md.

Each signal tool is coverage-aware (every answer names its denominator), carries a not_a_forecast line (D71 — no permit/parcel lead-time join is populated), and returns facts not documents (row payloads pass through api.feed_payload_public, D68). Every tool declares a coarse envelope outputSchema and emits structuredContent (typed for the consuming model), leaving the DB-owned interior as to_jsonb passthrough (invariant 4).

The three resources — the discoverable face

Tools are the query half of the read-only-DB MCP pattern; resources are the discover half — the static, always-present description of the dataset the model can load without spending a tool call. Read live over the same api_reader / D75 wall, PII-free, no verbatim text. Source: src/resources.ts.

The initialize instructions blurb frames the dataset as reveals and states the honesty rules (denominators always, no forecast, no verbatim, the ~94%-unknown caveat).

Hardening (MCP spec MUSTs — reference docs/reference/2026-07-28-mcp-tap-design-references.md)

• Origin validation (src/auth.ts validateOrigin, enforced in worker.fetch) — Streamable HTTP spec 2025-11-25 MUST. A disallowed Origin gets 403 before auth or any DB call; a missing Origin (non-browser client — claude.ai, Claude Code, curl) is allowed because DNS-rebinding is by definition a browser cross-origin request. Allowlist = self + claude.ai / claude.com + ALLOWED_ORIGINS.
• Rate limiting (wrangler.toml RATE_LIMITER binding, enforced in worker.fetch) — MCP tools §Security MUST. 300 req/60s keyed on client IP; 429 before any DB call. Guarded, so a deploy without the binding degrades to no-limit rather than crashing.
• RFC 8707 audience binding — satisfied by construction: this tap is single-tenant AS = RS, the provider accepts only tokens it issued for this one resource, so audience-confusion has no surface. Documented, not assumed (server.ts, near the OAuthProvider config).

An undeployed function answers with a clean gap report — "The api.X() function is not deployed to the database yet ... The other tools on this server still work." — not a 500. A tool that reports the gap lets the client model route around it and say something true; a tool that crashes teaches it the whole connector is broken.

The tool descriptions are the product surface here, not documentation. They are the only thing the model on the other end reads before choosing a call, so each one states what the numbers actually mean (an acte is a decision, not a document), what the tool will not do (never verbatim text), and when to reach for a different tool. Counting rules travel with the numbers: anything reporting an acte count also reports the distinct résolution count, and the coverage stages carry an explicit never-sum note.

Routes

GET  /                          plain landing text
GET  /healthz                   liveness + endpoint + auth modes (no secrets)
POST /mcp                       the MCP endpoint (JSON-RPC)
GET  /mcp                       405 — this server sends no unsolicited messages
GET  /.well-known/mcp           server metadata + data policy
GET  /oauth/authorize           the "paste your access token" page
POST /oauth/authorize           verifies the token, completes the grant
/oauth/token, /oauth/register   @cloudflare/workers-oauth-provider
/.well-known/oauth-*            @cloudflare/workers-oauth-provider

Why there are two ways in

The brief called for a static bearer token, and that is what the server actually checks — one secret, constant-time. But claude.ai's Connectors UI has no field for a bearer token. Verified against Anthropic's own custom-connector help page, fetched 2026-07-28: you enter a URL, and "Advanced settings" takes an OAuth Client ID and Client Secret. A bearer-only server is a server Marc cannot add to Claude, which fails the mission.

So the same secret is reachable two ways:

1. Authorization: Bearer <MARC_TOKEN> — Claude Code, the MCP Inspector, curl, any script. Fully testable offline.
2. OAuth 2.1 with dynamic client registration — what claude.ai speaks. The authorize page is one field asking for the access token; on a constant-time match it completes the grant. The token is the password; OAuth is only the envelope claude.ai insists on.

This is deliberately not the auto-approving authorize endpoint the BOI tap uses (~/work/boi/apps/tap/src/index.ts, boi.2026-04-30.008/.009), where anyone who learns the URL is granted a token. That is defensible for a single-tenant surface behind Cloudflare Access. It is not defensible for a URL handed to an outside collaborator.

Run

cd apps/worker/qc-mcp-tap
pnpm install --ignore-workspace --prefer-offline   # not a workspace member, like tile-builder
pnpm typecheck
pnpm test                                          # 122 tests
bash make-dev-vars.sh                              # writes gitignored .dev.vars from ../../../.env.local
npx wrangler dev --port 8799 --local
bash smoke.sh                                      # drives the running Worker against LIVE api.*

pnpm test runs in plain Node because every module is written against web-standard globals (fetch, Request, crypto.subtle) — the same objects workerd exposes. The one exception is cloudflare:workers, aliased to test/stubs/cloudflare-workers.ts, which explains exactly what it does and does not stand in for. The suite proves the application layer, not the platform. KV semantics and the OAuth storage path need wrangler dev.

The entry module has exactly one export. workerd reads every named export of the entry file as a service entrypoint and refuses to boot otherwise — a failure invisible to tsc and to the unit suite. So src/index.ts is a one-line re-export and everything else lives in src/server.ts.

Deploy — DONE; this is the redeploy path

The Worker is live at https://ortova-qc-tap.felixbosse.workers.dev (the default workers.dev subdomain — the qc-tap.nexod.ca custom-domain route below is not yet wired, see Open calls). OAUTH_KV, MARC_TOKEN, and SUPABASE_API_READER_KEY are already provisioned; the steps below are what a future code change (like a schema hardening pass) re-runs, not a first-time bring-up.

cd apps/worker/qc-mcp-tap

# Re-run only if a secret needs rotating (see Rotation above).
# npx wrangler kv namespace create OAUTH_KV
# npx wrangler secret put MARC_TOKEN
# npx wrangler secret put SUPABASE_API_READER_KEY

# Dry run, then deploy.
npx wrangler deploy --dry-run
npx wrangler deploy

# Verify from outside.
curl -s https://ortova-qc-tap.felixbosse.workers.dev/healthz

Rotation. wrangler secret put MARC_TOKEN changes the bearer path immediately, but OAuth grants already issued keep working until their TTL (7 days). To revoke fully, rotate the secret and purge OAUTH_KV.

Migrations 0434 + 0438 + 0439 — all APPLIED (2026-07-28)

Applied in dependency order, each with a manifest in migrations/applied/: 2026-07-28-0434_feed_delivery_contract_honesty.json → 2026-07-28-0438_api_qc_tap_read_surface.json → 2026-07-28-0439_api_qc_tap_signal_surface.json.

migrations/0438_api_qc_tap_read_surface.sql — the plumbing read surface: api.acte_lookup, api.qc_corpus_ladder, api.qc_muni_profile. It also closed a defect it found: six api.* functions carried EXECUTE to PUBLIC. 0426 and 0434 issued their GRANT without the matching REVOKE ALL ... FROM PUBLIC that every other api.* migration in this repo pairs with it, and Postgres grants PUBLIC by default. On a SECURITY DEFINER function that is the real exposure shape, because the definer bypasses the very wall these functions enforce. Section E revoked it; its verification block made it impossible to apply the file with the hole open.

migrations/0439_api_qc_tap_signal_surface.sql — the signal query surface: api.acte_search, api.acte_trend, api.instrument_status. Mirrors 0438's pattern exactly (SECURITY DEFINER + search_path='' + REVOKE ALL FROM PUBLIC + GRANT EXECUTE TO api_reader + a verification DO block that fails the apply on any hole). No grant widens past api_reader. Its function bodies were dry-verified read-only against live production (acte_search 35 ms, instrument_status 39 ms) before drafting, then applied.

Both depended on 0434 (api.feed_payload_public), applied first. Each file's verification block fails loudly rather than creating functions that skip the D68 exclusion or leave a PUBLIC-execute hole — that guarantee held on the real apply, not just in the dry-run.

Evidence (2026-07-28, live production, via wrangler dev + smoke.sh)

• initialize → ortova-qc-tap 0.1.0, protocol 2025-06-18, instructions delivered.
• tools/list → all five tools.
• qc_feed_watermark → real api.feed_watermark + api.feed_head: lag_seconds 22.171, head_cursor djE6MTM4Nzg0MDoxOTA5Mzcx, reading "Healthy".
• qc_feed_page limit 2 → real api.feed_batch: 2 events, has_more true, real cursor. Payload verbatim-key check: none present — of nature, observed_type_string, lot_raw, matricule_raw, zone_code_raw, party_name_business, raw_row_hash, source_content_hash, zero appeared. Attribution present on every event.
• Bad cursor → invalid_cursor with instructions, not a silent restart from genesis (which would re-deliver the whole corpus and read to the consumer as success).
• qc_muni_profile, qc_acte_lookup, qc_coverage_summary, qc_signal_search, qc_signal_trend, qc_instrument_status → all live against production api.* (0438/0439 applied); the not_deployed report only fires if a function ever goes missing again (e.g. a rollback) — see the ApiRpcError path in api-client.ts.
• code_geo: "MTL" → refused at the published schema; no database call made.
• Two addressing modes on qc_acte_lookup → refused with the instruction, not a silent winner.
• No token → 401 + WWW-Authenticate pointing at the resource metadata. Wrong token →
  401. Missing MARC_TOKEN → 503, never open.
• 2026-07-28 hardening pass: every tool input schema is now .strict() (Zod), and registerTool's inputSchema is given the schema INSTANCE rather than its bare .shape — passing .shape lets the MCP SDK re-wrap it in its own non-strict z.object(), silently stripping an unrecognized argument name instead of rejecting it. Before this fix, qc_signal_search called with kind instead of resolution_kind (or decided_since instead of decided_from) ran UNFILTERED — 35,847 rows instead of the intended 307, no error at all. Now the same call returns a clean JSON-RPC -32602 Input validation error naming the unknown key, matching the path a wrong-TYPE argument already used. See test/worker.test.ts's "unknown-argument rejection (2026-07-28 defect)" suite and test/schemas.test.ts's matching schema-level suite for the red-before/green-after evidence.

The qc_coverage_summary ladder body was dry-verified read-only against production before 0438 was applied: 2.4 s (232,383 held / 117,553 parsed / 815,681 extracted / 813,549 classified / 820,353 actes / 9,190 lot-resolved; 1,098 universe), Montréal profile 0.6 s, acte lookup 45 ms. The no-match path returning zero rows rather than zeros is why api.acte_lookup is plpgsql with an explicit IF v_n IS NULL guard.

Open calls

• api_reader is NOLOGIN, so the Worker inherits its envelope through anon rather than holding the role directly. Making it its own login role is an ALTER ROLE — a gated write, and no privilege change.
• @cloudflare/workers-oauth-provider is pinned at 0.4.0 (what BOI proved). 0.8.2 exists; upgrading is worth a look but was not taken blind.
• The qc-tap.nexod.ca custom-domain route does not exist yet — the live URL is the default ortova-qc-tap.felixbosse.workers.dev subdomain.

Prior art

~/work/boi/apps/tap — the stateless WebStandardStreamableHTTPServerTransport + enableJsonResponse shape, and the OAuth-provider wiring, are lifted from there (boi.2026-04-30.013 measured the ~10 s stream-close latency the SSE path cost). What is deliberately different: no Durable Object, no auto-approve authorize endpoint, no supabase-js, and no service-role key.

Attribution

Every payload this server delivers carries the required line, and the api.* functions enforce it (invariant 11):

Municipal council proceedings (procès-verbaux, ordres du jour, avis publics), municipality of origin

apps/worker/rf-stagehand/README.md

rf-stagehand — Registre foncier lookup worker

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: a Fastify + Stagehand HTTP worker that looks up Registre foncier ownership.
• Input/output boundary (Layer-1): HTTP POST /registre-foncier/lookup — LookupRequest, LookupResult, ExtractedOwnershipSchema (Zod already, src/lookup.ts / src/server.ts; boundary table §C).
• Run locally: pnpm --filter @ortova/worker-rf-stagehand dev (Fastify server).
• Deploy: Docker container.

Railway-hosted HTTP sidecar that scrapes registrefoncier.gouv.qc.ca via Stagehand + Browserbase. Fills the canonical.property_ownership table that drives the ownership_change signal.

API

POST /registre-foncier/lookup

{
  "cadastralLot": "1 234 567",
  "lookupId": "abc123"
}

Headers:

• X-API-Key: <RF_WORKER_API_KEY> (required if the worker has the env var set)

Response 200:

{
  "lookupId": "abc123",
  "cadastralLot": "1 234 567",
  "found": true,
  "ownerName": "Jean Tremblay",
  "acquisitionDate": "2023-09-14",
  "registrationNumber": "25 379 544",
  "subsequentCreditors": ["Desjardins Caisse Centrale"],
  "declarationResidenceFamiliale": false,
  "costCad": 0.20,
  "sessionId": "sess_abc",
  "durationMs": 12350
}

GET /health → { ok: true, service: "rf-stagehand" }

Environment

Deployment

# First time:
railway login
railway init    # create a new Railway project (Hobby tier, $5/mo)
railway link    # if re-linking

# Set secrets:
railway variables set \
  BROWSERBASE_API_KEY="bb_..." \
  BROWSERBASE_PROJECT_ID="prj_..." \
  ANTHROPIC_API_KEY="sk-ant-..." \
  RF_WORKER_API_KEY="$(openssl rand -base64 32)" \
  LOG_LEVEL="info"

# Deploy:
railway up

# Get the public URL:
railway domain

After deploy, set LAMBDA_BROWSER_URL + LAMBDA_BROWSER_API_KEY on the overwatch-dev Vercel project. The qc-registre-foncier-ownership source adapter reads both to fire lookups.

Cost model

Each session burns $0.05 Claude (Stagehand planner, ~3 turns) + ~$0.15 Browserbase (30s session) = **$0.20 / session**. Platform-side preFlightCostCap() enforces a $200 CAD/month cap against raw.rf_cost_log before each session fires. Sessions that succeed report their actual cost back and the ledger updates.

Stagehand + Browserbase caching

The Stagehand selector cache means Day 1 of a new lookup burns Claude tokens for page interpretation; Days 2-30 execute cached selectors at $0 LLM cost. Browserbase compute still accrues per session. Expect total cost to drop ~80% after the first week of production runs.

apps/worker/tile-builder/README.md

tile-builder

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: bakes the map substrate — PMTiles from canonical.* layers.
• Input boundary (Layer-1): CLI --layer / --sql-filter; reads canonical.*. No Zod schema (FLAGGED, boundary table §C): batch CLI, env/flag input.
• Output: PMTiles → R2. R2-owned artifact (§C).
• Run locally: pnpm --filter @ortova/tile-builder plan / build.
• Deploy: gh workflow run tile-builder.yml (runs-on ortova-mac).

Reusable vector-tile build job: turns canonical.* layers into PMTiles and uploads them to Vercel Blob. Built to run on a cloud box with a fat pipe to the DB — not a laptop. Province-scale geometry (cadastre ~3.7 GB, property centroids ~2.78M points) is stream-bound; from a home uplink the parcels layer alone would take hours, while cloud-to-cloud in ca-central-1 (where Supabase lives) it runs to completion in tens of minutes.

Sibling to apps/worker/geo-ingester/ — same shape (run-to-completion, declarative manifest, Felix-gated writes, --plan dry-run, optional --local-only).

What it does (per enabled layer in layers.ts)

1. Streams features from canonical.* as GeoJSONSeq via psql -At COPY (...) TO STDOUT (one JSON Feature per line).
2. Pipes the stream into tippecanoe with the per-layer flags (zoom range, simplification, clustering, drop/coalesce policy), writing <layer>.mbtiles.
3. Converts to PMTiles with pmtiles convert.
4. Uploads to Vercel Blob as tiles/<layer>.pmtiles (public, no random suffix, allowOverwrite), capturing the public URL.
5. Rewrites the TILE_URLS block in apps/dev/src/lib/tile-urls.ts so the /home MapLibre canvas reads stable, committed URLs.

Read-only on canonical.* (Rule 6). The only state-changing side effects are the Blob upload and the manifest rewrite, both gated by --i-am-felix.

Per-layer config (verbatim from the brief)

footprints is declared but disabled: true until the Overture province ingest (docs/briefs/2026-05-26-qc-object-graph-scaleout.md Wave 2) lands.

Run it

Deploy to Railway (matches the other apps/worker/* sidecars) in a region close to Supabase, or run as an AWS Fargate/EC2 task in ca-central-1 (optimal — same region as the DB; this is what the geo-ingester uses for cadastre/footprint ingests of similar size). Set:

SUPABASE_POSTGRES_POOLER_URL = <session-pooler url>
BLOB_READ_WRITE_TOKEN        = <Vercel Blob token from the linked project>
WORK_DIR                     = /tmp/tile-builder      # optional, default

Then it runs once on start (CMD = tsx build.ts --i-am-felix).

Local sanity checks (no cloud needed)

# Dry run — counts + the exact tippecanoe command per layer, no streaming.
SUPABASE_POSTGRES_POOLER_URL=... tsx build.ts --plan

# Build one layer locally (territories is fastest — ~1.4k features).
# Leaves WORK_DIR/<layer>.pmtiles on disk for inspection. Skips Blob.
SUPABASE_POSTGRES_POOLER_URL=... tsx build.ts --layer territories --local-only

--local-only and --plan do NOT touch the manifest or Blob.

Generated artifacts

• WORK_DIR/<layer>.mbtiles — tippecanoe output (intermediate).
• WORK_DIR/<layer>.pmtiles — final tile bundle.
• tiles/<layer>.pmtiles — Blob object key (overwritten on each run).
• apps/dev/src/lib/tile-urls.ts — manifest, rewritten in place per run.

Felix-gating posture

Same as geo-ingester:

• --plan — read-only.
• --local-only — read-only on the DB; only side effect is files in WORK_DIR.
• --i-am-felix — required for Blob upload + manifest rewrite.

The DB connection has statement_timeout=0 idle_in_transaction_session_timeout=0 applied via PGOPTIONS for the long-running COPY streams (parcels and property take minutes each).

Adding a layer

Add an entry to LAYERS in layers.ts — no code change. To pin tile parameters to the brief, paste the tippecanoe flags verbatim from the per-layer table above. The runner is layer-agnostic.

packages/brand/README.md

@ortova/brand

The frozen, versioned home for Ortova's design assets — the repo side of the contract in docs/design/design-asset-management.md. Figma (Ortova — Brand & Design System, fileKey oNDdUL9rGunX7Yuh9VRSi7) is the design source of truth; this package holds the exported, approved files apps actually consume. The app never reads Figma live, and never keeps a per-app inline copy — this is the one place.

Naming

kind/variant/context/size, lower-kebab, mirrored 1:1 as the folder path (contract Layer 2). Today:

logo/lockup/horizontal/svg/color-on-indigo.svg   — full color lockup ON the indigo ground rect
logo/lockup/horizontal/svg/color-transparent.svg — the SAME lockup, indigo ground stripped (derived)
logo/lockup/horizontal/svg/mono-white.svg        — dark chrome
logo/lockup/horizontal/svg/mono-indigo.svg       — light marketing chrome
logo/lockup/horizontal/svg/mono-ink.svg          — light operator chrome
logo/lockup/horizontal/png/<variant>@1x.png      — raster proofs (@1x/@2x), only where SVG can't go
logo/lockup/horizontal/png/<variant>@2x.png

color-transparent is a derived asset, not a separate Figma node: brand:sync computes it by stripping the single flat-#000F6A ground <path> from color-on-indigo (every glyph + wordmark path byte-identical), so it always tracks the pinned grey (#D9D9D9) wordmark and never carries the ~2ppm control-point jitter a Figma clone() would. Grounds belong to the surface, not the asset (Felix, 2026-07-20) — a page supplies its own background and drops in color-transparent, no rectangular seam. The mono trio shares one aspect ratio (280 × 115.5); the two color lockups share 500 × 206.25. Set height, width follows.

Keeping this fresh

Figma (oNDdUL9rGunX7Yuh9VRSi7) is the source; this directory is a frozen mirror, never a place to hand edit. Run pnpm brand:sync to pull the current state of every asset in sync-manifest.json (the node id -> path wiring), strip Figma's export artifacts, and rewrite the SVG + PNG proofs here in place. Needs FIGMA_TOKEN (a personal access token, figma.com -> Settings -> Personal access tokens) in .env.local locally or as a GitHub Actions secret in CI; pnpm brand:sync --dry-run validates the manifest without one.

Provenance (2026-07-14, tile 411)

Exported from Figma nodes 3:2 / 73:4 / 73:13 / 73:22 (frame names match the paths above verbatim). The Figma MCP/desktop-app export path bled the Foundations page backdrop + a per-variant "stage" demo card (a background swatch Figma renders so a mono-white mark is visible against something on the canvas) into the raw SVG/PNG — confirmed by diffing the frame's own child list (get_metadata) against the exported markup. Both were stripped before landing here: what ships is exactly the frame's own vector children (the mark + wordmark paths), nothing else, on a transparent ground for every variant except color-on-indigo (there the indigo field IS one of the frame's own vectors, intentional, kept). The PNG proofs are rasterized locally from these cleaned SVGs rather than from Figma's PNG export, for the same reason: Figma's raster export carries the identical baked-in stage background. brand:sync (below) instead calls the versioned REST /v1/images endpoint, confirmed the same day to already export cleanly (no backdrop, no stage card) for these four nodes — stripFigmaBackdrop() still handles the MCP-shaped bug as a fallback in case a future export reintroduces it, but the common path today needs no stripping beyond tightening the viewBox to the frame's exact (non-rounded) size.

Updated 2026-07-20 (brand rework — canonical logo corrected + single-source components): Felix pinned the final logo set. The canonical color-on-indigo is now the light-grey wordmark (#D9D9D9) on indigo, NOT the earlier white-wordmark treatment — the white lockup is retired to the Figma ⌫ Archive page (never deleted; ids may be referenced). In the same pass every lockup master in the Figma file was converted to a Figma component (Pin 3: each asset exists exactly once as a component; usages are instances), which minted new node ids. The manifest above was repointed accordingly: color-on-indigo 3:2 → 159:3, mono-white 73:4 → 159:5, mono-indigo 73:13 → 159:6, mono-ink 73:22 → 159:7. Verified by brand:sync: the three mono SVGs re-exported byte-identical (same sha256 as 2026-07-14, proving componentization preserved the vectors), while color-on-indigo changed (white → grey). Canonical colors confirmed from the pinned vectors: indigo #000F6A, spire teal #249AB8 → #5FB6C9 → gold #E6C173, mono-ink #17140F.

Updated 2026-07-21 (mirror collapse + color-transparent folded in): brand:sync is now the single fan-out (the competing apps/dev/scripts/copy-brand-assets.mts build-time copy is retired; its dev mirror is now a committed, validator-enforced fan-out). The landing's separately-pinned color-on-page.svg (a hand-derived color-on-indigo-minus-ground, still carrying the old white wordmark) is retired: it is replaced by the derived color-transparent asset, computed from the current canonical color-on-indigo — so the landing nav wordmark now reads grey #D9D9D9, the consistent consequence of the grey-canonical pin (the last white-wordmark instance leaves live surfaces). Two HQ calls carried into this pass:

• color-transparent ships a TRANSPARENT ground — grounds belong to the surface, not the asset (Felix's landing precedent, 2026-07-20: a baked ground seams on real surfaces).
• color-on-light (indigo wordmark + color spire, Figma node 159:4) is decided to ship transparent-ground for the same reason, but is still [unverified vs Marc's .key] and not frozen here — the Ortova - Logo & colors.key Keynote could not be read by the tooling. Marc outstanding: (1) mint a real color-transparent Figma component + ping back its node id (today it is byte-derived from color-on-indigo, not its own node); (2) confirm color-on-light ships transparent (ground supplied by the surface), and flag if the .key differs. See docs/design/brand-figma-file-structure.md.

Re-verified 2026-07-14: re-pulled all four nodes twice (once via Figma MCP, once via brand:sync against the live REST API) to check a report that Figma held a spacing fix not yet in this package. Both pulls are geometrically identical to what was already landed here (every <path d="..."> byte for byte the same; only Figma's auto-generated gradient-id suffixes differ, which is inert). The clearspace between mark and wordmark was already 1x mark width in the landed assets — Figma has not changed since. If a stale lockup is still visible somewhere, the cause is a cached preview/deploy target, not these files.

How apps consume this — the single-source fan-out (2026-07-20)

packages/brand is the one authored home. An app can only serve files that physically live under its own public/, so pnpm brand:sync fans each asset out into per-app mirror dirs, declared in sync-manifest.json under mirrors:

"mirrors": {
  "dev":     { "dir": "apps/dev/public/brand",     "formats": ["svg"] },   # OrtovaLockup wrapper
  "landing": { "dir": "apps/landing/public/brand",  "formats": ["svg"] }    # color-transparent nav mark
}

Each asset lists which mirrors receive it ("mirrors": ["dev", "landing"]). The mirrors are committed, byte-identical, and NOT authorable — real files (never a symlink: a symlink outside the app dir broke Vercel's static collector twice, 2026-07-14; committed real files sidestep that and need no build-time copy step). A hand-edit, a stale copy, or a stray file in any mirror goes red at the next gate via scripts/validate-arch/brand-mirror-parity.ts, which asserts every mirror file is byte-identical to its packages/brand source and the mirror holds nothing else. The one and only fix for a red is to run pnpm brand:sync — never to edit a mirror.

To change a brand asset: edit it in Figma → pnpm brand:sync → commit → push. That is the whole contract ("change it in one place, say update and push, that's it" — Felix). Apps reference assets by served path, e.g. /brand/logo/lockup/horizontal/svg/color-transparent.svg; apps/dev/src/components/panels/wordmark.tsx (OrtovaLockup) is the one dev wrapper, and the landing's nav <img> is the one landing consumer — nothing else should inline these paths.

packages/platform/README.md

@ortova/platform

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

The engine library apps/core and apps/dev build on: the six primitives (subjects, adapters/sources, signals, bundles, products, actions), the brain, orchestration, observability, cost-tracking, and storage. Not a running service — it is consumed in-process.

• What it is: the shared platform SDK; home of the factory contracts that throw at module load (define*) and the schema authority for ingress payloads.
• Input/output boundaries (Layer-1):
  • External-API payloads → per-adapter-kind Zod in src/adapters/kinds/* (arcgis, socrata, ckan, opendatasoft, carto, stac, browser) + per-source schema.ts under apps/core/src/sources/*. See boundary table §D.
  • Product egress → products/*/shape.ts (e.g. InsuranceScoreSchema). See §A v1/score resp.
  • Brain / signal / event types → src/brain, src/signals, src/orchestration.
• Run locally: consumed via workspace import (import { ... } from '@ortova/platform'); pnpm --filter @ortova/platform typecheck. Entry src/index.ts, subpath exports in package.json.
• Deploy: none — published only as a workspace dependency; ships inside apps/dev / worker builds.

packages/twist/README.md

@ortova/twist

Twist client for Overwatch. Wraps @doist/twist-sdk with a factory (createTwistClient), OAuth helpers (buildAuthorizationUrl, exchangeAuthorizationCode), and inbound-webhook primitives (verifyWebhook, parseInboundFormBody).

Vendored from the studio's @nexod/twist (the same package BoI consumes), following the @ortova/platform precedent: engine code is pulled into packages/ and renamed to the @ortova/* namespace so this repo stays self-contained. Keep behavioral changes in sync with the studio package, or note the divergence here.

This package owns client construction, env-var resolution, error normalization, and a thin set of high-level helpers. The consuming app (apps/core, apps/dev) owns which channels to post to, what content to format, OAuth credential storage, and retry policy — that wiring is the Overwatch integration spec, tracked on worksite tile #18.

Install

In-workspace; resolves via pnpm. Add to a consuming package:

// apps/<app>/package.json
"dependencies": { "@ortova/twist": "workspace:*" }

Token-based usage (server-side)

TWIST_BOT_TOKEN=<long-lived OAuth token>     # falls back to TWIST_ACCESS_TOKEN
TWIST_WORKSPACE_ID=<workspace id, integer>

import { createTwistClient } from "@ortova/twist";

const twist = createTwistClient();

const { thread_id } = await twist.postToChannel({
  channelId: Number(process.env.TWIST_CHANNEL_RUNS),
  title: "NDVI run #402 dispatched",
  content: "Verdun baseline scan, 1,284 parcels queued.",
});

await twist.replyToThread({ threadId: thread_id, content: "Gate decision: APPROVE." });

For raw API calls beyond the helpers, use twist.api.* directly.

OAuth flow

import { buildAuthorizationUrl, exchangeAuthorizationCode } from "@ortova/twist";

const authUrl = buildAuthorizationUrl({
  clientId: process.env.TWIST_CLIENT_ID!,
  scopes: ["user:read", "channels:read", "threads:write", "comments:write"],
  state: cryptoRandomState,
  redirectUri: "https://app.example.com/twist/callback",
});

// In the callback, after verifying state:
const { accessToken } = await exchangeAuthorizationCode({
  clientId: process.env.TWIST_CLIENT_ID!,
  clientSecret: process.env.TWIST_CLIENT_SECRET!,
  code: callbackCode,
  redirectUri: "https://app.example.com/twist/callback",
});

Inbound webhooks

Twist signs inbound POSTs only with a verify_token field (no HMAC). Parse the form body, verify the token, and combine with a user_id allowlist:

import { parseInboundFormBody, verifyWebhook } from "@ortova/twist";

const payload = parseInboundFormBody(await req.text());   // TWIST_WEBHOOK_SECRET
const result = verifyWebhook(payload);
if (!result.valid) return new Response(result.reason, { status: 401 });

Configuration responsibility

• Package owns: SDK construction, env-var reading at instantiation, error normalization, OAuth URL/token primitives, webhook parse/verify, the postToChannel / replyToThread / postCard / updateCard helpers.
• App owns: env var population, OAuth credential storage, channel selection per event, content formatting, thread-id persistence, retry policy.

Direct TwistApi instantiation outside this package is a bypass — justify it in a comment citing the gap, per the engine-bypass discipline in CLAUDE.md.