Ortova — Architecture

The intelligence layer for the built world. One spine holds every property's complete truth — permit history, ownership, valuation, physical change — joined on the matricule (Québec's universal parcel ID). An engine + an AI fleet compose intelligence products from that spine.

This is the single container-level map: what runs, what it talks to, and the invariants that hold across those boundaries. Component-level shape lives in each unit's README.md and in the named Zod schemas (src/schemas.ts) — jump-to-definition terminates there, not here.

• What shape is this data? → the named schema in a schemas.ts (colocated-docs Layer 1; docs/architecture/2026-07-22-boundary-schema-table.md).
• What does a unit do / how do I run it? → that unit's README.md.
• How does it connect / where does a fix go? → this file.
• Why is it built this way? → ## Decisions below; git log --follow for the timeline.

Container map

flowchart TD
    subgraph external["External feeds"]
        FEEDS["ArcGIS / Socrata / CKAN / ODS / CARTO / STAC / bulk-geo"]
        RF["Registre foncier (browser)"]
        PDFS["Municipal PDFs (minutes, permits)"]
        LIDARSRC["QC LiDAR / imagery tiles"]
    end

    subgraph runners["ortova-mac self-hosted runner fleet"]
        CRON["launchd watch-heartbeat cron -> watch-drive.yml"]
        WORKERS["apps/worker/* sidecars: geo-ingester, geo-trinity, permit-ingester,
minutes-harvester, permit-doc-parser, governance-parser,
lidar-ingester, tile-builder"]
    end

    subgraph queue["Watch executor"]
        BULLMQ["BullMQ / Redis queue (WatchJobSchema)"]
    end

    subgraph engine["Engine (in-process library)"]
        PLATFORM["packages/platform (@ortova/platform)
six define* factories + four fixed dispatchers"]
        INNGEST["Inngest dispatchers:
source -> canonical-upsert -> signal -> product"]
    end

    subgraph data["Data plane"]
        SUPA["Supabase Postgres
raw / canonical / serving / api / aerial / ops"]
        R2["Cloudflare R2
PMTiles / GeoParquet / COPC / COG objects"]
    end

    subgraph surfaces["Serving surfaces (Vercel)"]
        DEV["apps/dev (@ortova/dev)
operator scenes + app/api/* serving edges"]
        LANDING["apps/landing (@ortova/landing)
ortova.io marketing"]
    end

    subgraph sidecars["HTTP sidecars"]
        LIBPOSTAL["libpostal-sidecar (pydantic)"]
        RFSTAGE["rf-stagehand (Zod)"]
    end

    FEEDS --> WORKERS
    PDFS --> WORKERS
    LIDARSRC --> WORKERS
    RF --> RFSTAGE
    CRON --> BULLMQ
    BULLMQ --> WORKERS
    WORKERS --> SUPA
    WORKERS --> R2
    PLATFORM --> INNGEST
    INNGEST --> SUPA
    WORKERS -. use .-> PLATFORM
    DEV --> SUPA
    DEV --> R2
    DEV --> LIBPOSTAL
    DEV --> RFSTAGE
    RFSTAGE --> SUPA

Containers

Invariants (hold across every boundary; survive all refactors)

1. Every property fact lives in a typed canonical.property_* satellite that owns exactly one fact and carries its own provenance (source_key + external_id + raw_row_hash + SCD-2). canonical.property is identity-only.
2. Identity is normalized once, at ingest. matricule_id is the province-wide join key; derive over store (generated columns / views), never re-store a derivable value.
3. Drawers evolve additively. A canonical satellite in production is never renamed, rebuilt, or superseded — new source classes generalize into the existing drawer so context compounds.
4. Every hand-shaped boundary is a named Zod schema in a schemas.ts, with types derived via z.infer. Verbatim api.* / canonical rows stay to_jsonb passthrough (DB owns that shape).
5. The api.* read-contract is PII-free by construction — SECURITY DEFINER functions, and api_reader holds zero grant on canonical.*.
6. Postgres schemas are restricted to {raw, canonical, serving, aerial, ops, api} — anything else build-fails.
7. All engine data flows through the six define* factories and the four fixed dispatchers. Bypassing a factory throws at module load.
8. Serving is open-format end-to-end. MapLibre + deck.gl project COPC / COG / PMTiles as range-readable R2 objects; compute and UI meet only through those objects plus DB facts.
9. Batch compute runs headless on the ortova-mac fleet / runners, writing objects + facts; the browser only projects them. The machine compounds even with the UI down.
10. DB writes are Felix-gated through scripts/apply-migration.ts --i-am-felix; reads are free.
11. Every data source carries an attribution line everywhere it is shown or delivered.

Module boundaries

• packages/platform exposes the define* contract that apps/core relies on — change the exposed surface deliberately and note it in the commit.
• apps/dev/src/schemas.ts is the only home for hand-Zod'd serving-edge contracts (curated column subsets, GeoJSON envelopes); it never mirrors a canonical table.
• scripts/ops/watch/executor/schemas.ts owns WatchJobSchema — the single cross-worker queue message — and the per-workflow arg bags (WorkflowArgsSchemas).
• Worker input boundaries are stringly-typed env/flag bags; their named schemas live in the executor schema file, not in each worker.

Decisions

Irreversible, architecture-shaping decisions, newest first. Append new entries here (context / decision / consequences, 3–6 lines); split to standalone ADR files only past ten entries. Full archive: decisions/decisions.md; current truth: decisions/canon-now.md.

2026-07-22 — Colocated documentation architecture

• Context. Shape and prose kept drifting from code; docs lived far from what they described.
• Decision. Schemas are the source of shape (named Zod in schemas.ts, types via z.infer); one README.md per deployable unit; one root ARCHITECTURE.md; the same files double as CLAUDE.md agent context. Order: docs/ops/orders/2026-07-21-colocated-docs-architecture.md.
• Consequences. A new boundary means a new named schema in the same commit; a new deployable unit means a new README in the same PR; irreversible decisions land here before the implementing commit.

2026-07-22 — Serving is open-format, never platform

• Context. A stateful tile server / vendor GIS middleware would insert a moat into the serving path.
• Decision. MapLibre + deck.gl over COPC / COG / PMTiles as range-readable static R2 objects; no tile servers, no GIS middleware. The formats are the contract between compute and UI.
• Consequences. Every heavy pipeline runs headless writing objects + facts; the browser only projects. Compute and UI never meet directly, so the machine compounds even with the UI down.

2026-07-22 — Drawers evolve additively (never supersede a production satellite)

• Context. Ratifying the imagery-pipeline architecture; supersede-and-rebuild was the tempting shortcut.
• Decision. A canonical satellite in production is never renamed / rebuilt / replaced; new source classes generalize into the existing drawer (added columns, widened enums).
• Consequences. Context compounds on one spine long-term; migrations widen, never fork, a fact.

2026-07-21 — Sub-metre optical is THE detector; 10 m Sentinel-2 retired

• Context. The 10 m path was carried under a "rural-reserved" reservation for a tier never built.
• Decision. Sub-metre optical (WV-3 × WV-2, 0.5 m) is the sole detector. The Sentinel Lambda, sentinel2-stac / sentinel1-rtc-mpc sources, ndvi-zscore signal, and 10 m dispatchers are excised.
• Consequences. image-change is a pre-production shell awaiting a sub-metre source; no rural tier exists. Tombstone: docs/ops/2026-07-21-repo-staleness-audit.md.

2026-07-13 — GitHub Actions / the mac fleet is the batch cloud-execution path

• Context. Long-running ingests should not sit on the HQ laptop's critical path, and stale-code builds were a recurring failure.
• Decision. Batch jobs run via gh workflow run <wf> -f ... off merged main on the ortova-mac self-hosted fleet (mac-fleet-first, 2026-07-21). Railway is retired; Fly.io is the always-on fallback.
• Consequences. Runners check out main, so jobs always run merged code. Home: docs/architecture/cloud-execution-paths.md; ratchet: validate:wiring cloud-exec-path-wired.

2026-06-25 — Generated-satellite auto-DDL lane (zero-touch onboarding)

• Context. Onboarding a new source required hand-writing the satellite DDL each time.
• Decision. A Rule-6 extension auto-generates the typed satellite DDL for zero-touch source onboarding. Design: docs/architecture/zero-touch-source-onboarding.md.
• Consequences. Adding a source is scaffold-driven; the satellite shape stays catalog-complete by construction. defineService / civic-service was retired the same window (never built).

2026-06-24 — api.* is the published PII-free read-contract (system of record)

• Context. The agency was duplicating the platform's read surface, causing drift.
• Decision. api.* (consumed as @nexod/property) is the sole published reader — SECURITY DEFINER + an api_reader role with zero canonical.* grant.
• Consequences. PII is unreachable by construction; the agency drops its rf_* / agency.* copies. Migrations 0102–0118 applied.

2026-06-23 — Target data model v1: typed temporal logs + reversible identity + as-of layer

• Context. property_observation conflated facts and could not answer as-of questions.
• Decision. Typed canonical.property_* temporal logs (SCD-2), reversible identity normalized at ingest, and an as-of derivation layer. property_observation / recordObservation() / resolveToProperty() retired (migration 0075).
• Consequences. Each fact owns its satellite + provenance; derive-over-store is enforced by validate:arch. Canon: decisions/canon/data/data-shape-v1.md.

2026-04-27 — Six primitives + four fixed dispatchers

• Context. Ad-hoc pipeline code had no enforced topology.
• Decision. Five pipeline primitives (subject / source / signal / bundle / product) + defineAction, all via define* factories; the four-dispatcher topology is fixed and throws at module load on bypass.
• Consequences. The generative loop is uniform and mechanically enforced; new sources/signals/ products slot into fixed seams. Home: decisions/canon-now.md (§Engine paradigm).

Deployable units

README.md

Ortova

The intelligence layer for the built world. Ortova is a property system-of-record: it joins, on one spine, every property's permit history, ownership, assessed/market value, and physical change detected from sub-metre imagery — keyed to the matricule (Québec's universal parcel ID). The spine is the moat; the products are equal views of it, each green/red-lit by what the Atlas holds. There is no single "lead product": roofing leads (pre-production experiment), a municipal compliance worklist (designed), and the further config-products (insurer feeds, lender monitors, pre-closing reports, contractor leads) are all configs on the same spine — the stack and each product's honest maturity live in PRODUCT.md. Entry scope: the Verdun arrondissement of Montréal, with Laval second.

(The company/product is Ortova; the repo, packages, and paths are still named overwatch — legacy, not a second thing.)

• Full product description — vision, the open wedge, kill list, data sources, status: PRODUCT.md.
• The controlling vision + roadmap: docs/founder/the-machine.md.
• The agent operating contract — conventions, per-area guidance, DB/git discipline, execution protocol: CLAUDE.md.

Engine paradigm

The six primitives (subject / source / signal / bundle / product + defineAction) built through the define* factories in @ortova/platform, and the fixed four-dispatcher topology, are canon — single home: decisions/canon-now.md (§Engine paradigm) + decisions/enforced-index.md (build-enforced). defineService was proposed and retired, never built. Not restated here.

This file is the authoritative repo-structure map: when this layout and CLAUDE.md's per-area guidance overlap, this file wins for "what is where," and CLAUDE.md wins for "how to behave there."

Repository layout

apps/

The runtime. Six apps:

• apps/core/ -- the engine implementation: the signal/source/subject/product/bundle declarations and their producers. Subdirectories under apps/core/src/ are signals/, sources/, subjects/, products/, bundles/, canonical/ (reconcilers), atlas/, and inngest/ (dispatcher wiring). This is where domain work lands.
• apps/dev/ -- the Next.js internal operator surface (audit dossier rendering, signal/source/product views, the atlas, the ops cockpit, demo screens). Vercel-deployable. Local dev: pnpm --filter @ortova/dev dev. The survivor knowledge home lives inside it at apps/dev/src/brain/ (see below).
• apps/landing/ -- the public marketing site, live at ortova.io (Next.js, Vercel). Deployed via scripts/ops/release-deploy.sh --i-am-felix landing.
• apps/worker/ -- cloud-side sidecars, run next to the DB off the network-bound laptop (default batch trigger is GitHub Actions, not Railway — Railway is retired, trial exhausted): libpostal-sidecar/ (address canonicalization), rf-stagehand/ (Registre foncier browser scraping, bounded query budget), geo-ingester/ + geo-trinity/ (cloud-side bulk geo ingestion), tile-builder/, minutes-harvester/ (municipal meeting-minutes harvest), governance-parser/ + permit-doc-parser/ (document extraction), and permit-ingester/.
• apps/score-api/ -- a small Next.js app serving property-level score lookups (no current surface routes to it; live-vs-fossil status is open — don't delete pending that call).

packages/platform/

@ortova/platform -- the internal infrastructure: the six define* factories (incl. defineAction), the four-dispatcher topology, storage abstractions, observability with Slack alerts, and the dashboard scanner. apps/core/ is built on this package's contract. Changes to its exposed surface are consequential; make them deliberately and note the contract change in the commit.

pilots/

pilots/verdun-tier1-gee/ -- the Verdun Tier-1 Google Earth Engine pilot (Python/GEE). Validation and exploration surface, separate from the production engine.

data/

data/ -- the atlas: reference material for source and signal design, not runtime-loaded. Subdirectories: atlas/ (sensors, enrichments, transformations, vocabulary), scripts/, seed/, sql/, src/.

migrations/

migrations/ -- sequenced SQL migrations, numbered 0001... Migration order is the file numbering. Applied-migration manifests live in migrations/applied/. The sanctioned apply path is scripts/apply-migration.ts --i-am-felix <file> (Felix-only).

decisions/

The decision surface:

• decisions/canon-now.md -- what's true now (the active canon). Read on cold boot.
• decisions/enforced-index.md -- the canon rules mechanically enforced in the build (generated).
• decisions/decisions.md -- the full decision log (archive: why, not what's true now).
• decisions/source-priority.md -- source onboarding priority.
• decisions/canon/ -- the live canon set (architecture + data-shape).

docs/

• docs/founder/ -- the vision + operating layer. the-machine.md (North Star + roadmap), ortova-ai-native-org.md (how we build), buildout/fleet-zones.md (the 9 standing zones HQ dispatches against), founder-mode.md (the position room).
• docs/briefs/ -- the live process spine; work is dispatched as briefs (the model that replaced the order/receipt ceremony, docs/briefs/2026-05-13-process-rectification.md).
• docs/architecture/ -- architecture notes, incl. enforcement-contract.md (the rules binding agent + human work).
• docs/research/, docs/design/, docs/dossiers/, docs/validation/, etc. -- supporting research, design, and deliverable libraries (reference, not the context layer).
• archive/2026-06-02/ -- cold-archived ceremony (the order/receipt loop + its validators), retained as historical record. Never delete.

Survivor knowledge home

apps/dev/src/brain/ holds the load-bearing institutional knowledge -- inline knowledge cards plus MAP.md. Source-capability truth, compliance/legal notes, cost-model floors, the DR191 NDVI rationale. Treat it as load-bearing, not docs cruft.

Getting oriented

1. PRODUCT.md -- what Ortova is.
2. docs/founder/the-machine.md -- the controlling vision + roadmap.
3. run pnpm state -- prints the live "where we are right now" (branch, migrations, zones); fresh from the system, not a committed file.
4. decisions/canon-now.md + decisions/enforced-index.md -- what's true now + the build-enforced rules.
5. CLAUDE.md -- conventions, per-area guidance, execution discipline.

Build and validate

pnpm install
pnpm typecheck
pnpm validate:arch   # architecture invariants (incl. the wiring guardian)
pnpm validate:wiring # every doc claim / path / command resolves; boot docs are loaded
pnpm state           # regenerate STATE.md

apps/dev/README.md

@ortova/dev

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: the Next.js operator surface — three chrome-free scene surfaces (atlas map /, permit coverage, live /monitor) + the app/api/* serving edges.
• Input/output boundaries (Layer-1): every hand-shaped HTTP boundary is a named Zod schema in apps/dev/src/schemas.ts — request params (LeadScoresRequestSchema, FootprintsRequestSchema, SignalsRequestSchema, PilotRequestSchema, AtlasNeighborsRequestSchema, ScoreRequestSchema, BrainQueryRequestSchema, …), serving-edge responses (LeadScoreCollectionSchema, FootprintCollectionSchema, SignalCollectionSchema), envelopes (V0OkEnvelopeSchema / V0ErrEnvelopeSchema), and emitted events ({Source,Signal,Product}RunEventDataSchema). Product egress InsuranceScoreSchema lives in @ortova/core/products/insurance-score/shape. v0 request params are in src/lib/api-v0/routes.ts. Full map: boundary table §A. Verbatim api.* rows are to_jsonb passthrough (DB-owned, deliberately not hand-Zod'd).
• Run locally: pnpm --filter @ortova/dev dev (port 3030; needs .env.local with SUPABASE_POSTGRES_POOLER_URL).
• Deploy: Vercel — scripts/ops/release-deploy.sh --i-am-felix dev (Release Agent, gated). Live ortova.nexod.ca.

Operator dashboard for the Nexod Platform. Dark Palantir-style UI reading live state from Postgres (dytmsttyixbkltadddok, ca-central-1).

Routes

• / — pipeline overview
• /sources, /sources/[key] — source adapter health, runs, config
• /signals, /signals/[key] — signal producers + model versions
• /products, /products/[key] — products + delivery clients + recent leads
• /workers — Inngest function inventory grouped by primitive
• /bundles — scoring bundle versions
• /health — adapter health + recent failures
• /report — sprint data report with live KPIs + print-to-PDF
• Demo routes under (demo)/*: /gis, /permis, /use-cases, /opportunities, etc.

Operational scripts

# Local dev (requires .env.local with SUPABASE_POSTGRES_POOLER_URL)
pnpm --filter @nexod/overwatch-dev dev

# Smoke-test every route against a running URL
SMOKE_BASE_URL=https://overwatch.nexod.ca \
  pnpm --filter @nexod/overwatch-dev test:smoke

# Assert the Verdun/Laval cascade baselines
SUPABASE_POSTGRES_POOLER_URL=... \
  pnpm --filter @nexod/overwatch-dev cascade:replay

# Fire a synthetic alert to verify Slack wiring
SUPABASE_POSTGRES_POOLER_URL=... SLACK_WEBHOOK_URL=https://hooks.slack.com/... \
  pnpm --filter @nexod/overwatch-dev alert:smoke

Slack observability setup

1. Create a Slack incoming webhook in the target channel.
2. Set SLACK_WEBHOOK_URL on the Vercel project (Settings → Environment Variables → Production).
3. Redeploy (vercel --prod) — env vars are snapshotted at build time.
4. Run pnpm --filter @nexod/overwatch-dev alert:smoke to confirm the pipe works.

Once wired, the observability-digest-cron Inngest function (runs 05:00 UTC daily) scans raw.observability_alerts for unposted rows and ships them to Slack.

CI

• .github/workflows/dashboard-smoke.yml — Playwright smoke runs on every successful Vercel production deploy.
• .github/workflows/cascade-replay.yml — cascade baseline check runs nightly 04:00 UTC.

apps/landing/README.md

@ortova/landing

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

The public marketing site (ortova.io). Next.js, static/marketing-only.

• What it is: the outward brand + narrative site. No spine data, no operator surface.
• Input/output boundaries (Layer-1): none. This unit has no runtime data boundary — no HTTP API, no DB read, no queue message. Nothing here resolves to a schemas.ts name by design (marketing content, not a serving edge). Any future form/lead-capture endpoint gets a named schema in a local src/schemas.ts in the same commit (standing rule).
• Run locally: pnpm --filter @ortova/landing dev (port 3032).
• Deploy: Vercel — scripts/ops/release-deploy.sh --i-am-felix landing (Release Agent, gated). Live domain ortova.io.

apps/worker/geo-ingester/README.md

geo-ingester

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: cloud-side bulk-ingest of a national/regional geo source into raw staging, off the HQ laptop's critical path.
• Input boundary (Layer-1): WorkflowArgsSchemas['geo-ingest'] = { source, phase? } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema when Watch-driven. External payloads validated per adapter-kind schema.ts (boundary table §D).
• Output: raw geo staging rows → canonical upsert. DB-owned shape (not hand-Zod'd, §C).
• Run locally: pnpm --filter @ortova/geo-ingester plan (dry) / ingest.
• Deploy: gh workflow run geo-ingest.yml -f source=<key> (runs-on ortova-mac).

Reusable bulk-geo ingestion job: pulls a large external vector layer into a raw Postgres table, province-wide. Built to run on a cloud box with a fat pipe to the DB — not a laptop. Province-scale geometry (cadastre ≈ 3.7 GB, footprints similar) is upload-bandwidth-bound; from a home uplink (330 KB/s) the cadastre took ~3 h, with the DB sitting idle in ClientRead the whole time. Cloud-to-cloud in/near ca-central-1 (where Supabase lives), the DB write speed is the floor instead of the pipe — **20–30 min for the full cadastre**.

What it does (per INGEST_SOURCE)

1. Reads the source from sources.ts (ArcGIS FeatureServer URL, OID field, fields, target table, scope).
2. Harvests the layer in OID-range chunks — orderByFields=<oid> + FEATURE_SERVER_PAGING (stable paging; without it ArcGIS silently dupes+gaps), validating every chunk COUNT(*) == COUNT(DISTINCT oid) == range size, with 503 backoff retries.
3. Loads each chunk → staging via ogr2ogr COPY (-append), statement timeout disabled, one ~3 MB chunk on disk at a time.
4. Applies staging → target in WAL-safe ogc_fid batches (a single multi-M-row INSERT can PANIC the instance), scope-tagged.
5. Verifies the target row count equals the source count.

Idempotent: the apply refuses if the scope is already loaded.

Lessons baked in (so we never re-fight them)

• ArcGIS pagination is unstable without orderByFields → silent dupes + gaps. Always ordered.
• GlobalID is not unique in some QC layers — validate on OBJECTID.
• The public service 503s under sustained pulls → per-chunk retries + backoff.
• The pooler statement_timeout kills long COPYs → disabled on the connection.
• A single province-size INSERT/COPY blows WAL → chunked apply, chunk-at-a-time load.
• ogr2ogr -overwrite on the first chunk DROPs + recreates the staging table, wiping any table/column comments a prior catalog seed left there — which used to leave the staging table uncataloged after every run and trip validate:catalog red until someone re-ran scripts/seed-dataset-catalog.py by hand. Fixed (Felix decision 2026-07-08): the worker owns its catalog — stampCatalog() in ingest.ts re-applies the meta.dataset_catalog row + native COMMENTs for both the staging table and the target table at the end of every successful harvest, sourced from the stagingCatalog/targetCatalog fields on each sources.ts entry. Best-effort/non-fatal by design — a catalog-stamp failure never fails an otherwise-successful harvest. A new source's catalog content lives with its other declarative fields in sources.ts.

Run it

Sanctioned cloud-execution tiering — full home: docs/architecture/cloud-execution-paths.md. Every trigger honours the same principle: never run stale laptop code; run merged main (the 2026-07-08 build-model decision). What changed 2026-07-13: the free GitHub Actions runner, not the paid Railway image, is the DEFAULT.

PRIMARY (default): GitHub Actions — .github/workflows/geo-ingest.yml. A run-to-completion job on a free GitHub runner with a fat pipe to the DB (~20–30 min vs ~3 h from a laptop). actions/checkout@v4s main, so the code that runs is always merged main — the same never-run-stale guarantee, for free. Dispatch:

gh workflow run geo-ingest.yml -f source=cadastre_qc
gh run watch                                # tail to completion; read the final VERIFY row-count line

INGEST_PHASE — split harvest from apply (resumable, INV-4)

The pipeline is harvest → stage → apply. INGEST_PHASE (workflow input, or the env var locally) splits it so a long run can't strand a full stage table before the apply. timeout-minutes is 300 (a 414-tile province Overture harvest overran the old 120 cap at tile 28 and left 2.8M rows stranded in raw.*_stage with a zero target — the failure this mode exists to make un-repeatable).

• full (default) — harvest + apply, one run. Unchanged behaviour.
• harvest — stage only, apply deferred. Use to keep a huge harvest under a timeout, then finish with an apply run.
• apply — promote ALREADY-STAGED rows → the target without re-harvesting. Idempotent + resumable: the apply is ON CONFLICT (<target PK>) DO NOTHING in 200k-row fid batches, so a re-run (or a crash mid-apply) converges to the full set without double-writing — the fid-ordered batching is the watermark. Only for sources that declare applyConflictTarget in sources.ts (e.g. footprints_qc).

gh workflow run geo-ingest.yml -f source=footprints_qc -f phase=apply   # finish a stranded stage in minutes

source is a key from sources.ts (cadastre_qc, footprints_qc, …). The workflow runs ingest.ts --plan (read-only) then ingest.ts --i-am-felix (the Felix-gated write) with SUPABASE_POSTGRES_POOLER_URL from repo secrets. gh workflow run executes the workflow as it exists on the default branch (main), so a workflow edit must LAND on main before a dispatch can use it. Felix-gated: the Release Agent dispatches on Felix's approved set. Dry-run locally first if you like:

INGEST_SOURCE=cadastre_qc npm run plan      # prints the plan, no writes

LEGACY / fallback: Railway (scripts/ops/release-worker.sh). The git-connected Railway image still exists and still runs merged main, but its trial is effectively exhausted (2026-07-13) — do not provision new Railway services. Build model (decided Felix 2026-07-08): this service builds its Railway image directly from GitHub main (railway up's upload-from-cwd model is what let a merged fix run stale in prod once already; see scripts/ops/release-worker.sh's header). One-time per service, railway service source connect --repo <owner>/<repo> --branch main --service geo-ingester, Root Directory = apps/worker/geo-ingester, Dockerfile path = Dockerfile (mirrors railway.json's build.watchPatterns: ["apps/worker/geo-ingester/**"]), region ca-central-1. Triggering: bash scripts/ops/release-worker.sh --i-am-felix geo-ingester INGEST_SOURCE=cadastre_qc. Raw railway up / railway redeploy are not an allowlisted agent path. Use only an already-connected service; prefer GitHub Actions above.

FALLBACK for an always-on job: Fly.io (ca-central-1, scale-to-zero) — for a standing service rather than a run-to-completion dispatch. Not needed for batch geo (that is GitHub Actions).

After cadastre_qc lands (raw.qc_cadastre_lot, scope qc), run scripts/reconcile-parcel-qc.ts --scope qc --apply --i-am-felix to resolve canonical.property.parcel_id province-wide.

Adding a layer

Add an entry to SOURCES in sources.ts — no code change for another ArcGIS FeatureServer. (Overture footprints need a second kind — DuckDB over S3 GeoParquet by tile; the staging→apply→verify stages here are reused.)

US metro footprints (tile 401) — one table, many scopes

footprints_us_<metro> (20 sources, US_METRO_AOIS in sources.ts) is the sibling of footprints_qc generalized to disjoint AOIs instead of one province bbox: one national raw table (raw.us_overture_building, migration 0245), scope-tagged per metro (us_nyc, us_austin, …) — mirrors raw.qc_overture_building being "one table per DATASET KIND, scope-tagged," not "one table per city." Each metro source gets its own staging table (raw.us_overture_building_stage_<metro>), unlike footprints_qc's single shared stage — that's deliberate: 20 distinct sources sharing ONE stage table would let two concurrent dispatches clobber each other's in-flight -overwrite; isolated per-metro stage tables make concurrent metro dispatch safe. Adding a metro = add a row to US_METRO_AOIS (bbox + tileSize + measured building count) + its scope_code to migration 0245's reconciler list — no new code path. AOI bboxes are derived from the US Census TIGERweb "Incorporated Places" extent for each metro's GEOID (canonical.property_permit.municipality_code), padded 0.03°; see the US_METRO_AOIS header comment for the full citation and the measured canonical.property.centroid join-feasibility caveat (7 of the 20 metros have near-zero centroid coverage today — an upstream geocoding gap, not an ingest defect).

apps/worker/geo-trinity/README.md

geo-trinity

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: the release-sync geo pipeline — bakes GeoParquet + PMTiles and attaches footprints.
• Input boundary (Layer-1): WorkflowArgsSchemas['geo-trinity'] = { job: 'release-sync', release, shard? } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema when Watch-driven.
• Output: GeoParquet/PMTiles → R2, plus canonical.property_footprint rows. DB/R2-owned (§C).
• Run locally: pnpm --filter @ortova/geo-trinity release-sync (needs R2_* in .env.local).
• Deploy: gh workflow run geo-trinity.yml (runs-on ortova-mac).

The continental buildings trinity (tile 431, Felix's architecture decision 2026-07-14): three jobs that pin, join, and serve Overture Maps' NA buildings layer at national scale — off the laptop, cloud-side, next to R2 and the DB. Supersedes apps/worker/geo-ingester's per-city AOI approach for buildings (footprints_us_<metro>, tile 401): instead of ~20+ hand-declared metro bboxes landing rows straight into Postgres, this pins the WHOLE continent to object storage once per release, and only SELECTIVELY joins matched buildings onto properties — a new city's buildings are already in the pin (its grid tile is already covered), no new source entry, no unmatched-building row ever lands in the DB at all.

The three jobs

1. release-sync.ts — extracts Overture's theme=buildings/type=building for one release, tiled over a 5°×5° North-America grid (lib/na-grid.ts, 288 tiles), and publishes partitioned GeoParquet + MANIFEST.json (release tag, per-partition row counts, byte sizes, sha256s) to R2 (or a local dir for testing). Read-only against Overture's public S3 (unsigned).
2. attach-join.ts — exports property centroids + QC parcel geometries from canonical.* (read-only), spatial-joins them against a pinned buildings partition in DuckDB (QC rows: parcel-contains-building-centroid; everyone else: 0249's centroid-in-building rule), and upserts MATCHES ONLY into canonical.property_footprint + canonical.building_height — same idempotent ON CONFLICT keys as migration 0249 ((property_id, footprint_id, source) / (toit_id, lidar_source)), so a monthly re-run of a newer release is a pure GERS-stable diff, not a rebuild.
3. tile-bake.ts — bakes ONE buildings.pmtiles from the pinned NA parquet via tippecanoe (see tile-bake.ts's header for why tippecanoe over planetiler) and publishes it to R2 alongside the release.

Every job is --plan (read-only, real counts) verified, --local-only (writes locally, never touches R2), and Felix-gated on the actual R2/DB write (--i-am-felix) — same discipline as apps/worker/geo-ingester and apps/worker/tile-builder, whose patterns this worker reuses (DuckDB spatial+httpfs preamble, PGOPTIONS pooler-timeout override, the psql-COPY-to-stdout-into-tippecanoe pipe).

Destination: R2-or-local (lib/dest.ts)

resolveDest() picks R2 when R2_ENDPOINT + R2_ACCESS_KEY_ID + R2_SECRET_ACCESS_KEY + R2_BUCKET are all set, else falls back to --local-dir <path> (or LOCAL_DIR env, or a WORK_DIR-relative default) — so every job runs and proves itself with zero R2 credentials. As of 2026-07-14 the R2 env DOES exist (Felix's Cloudflare account, bucket ortova-geo, region auto, in .env.local + GitHub Actions secrets), but this worker's local proofs (below) were deliberately run against --local-dir — the actual R2 --i-am-felix publish is a Felix/Release-Agent op (Rule 6 / the Blob-upload class of write), not something this drop executes itself, per the SCALE agent's own draft-and-prove mandate.

Local proof (2026-07-14, run against the real overturemaps-us-west-2 S3 bucket + the real SUPABASE_POSTGRES_POOLER_URL)

release-sync, partition atlantic_test (bbox [-64.5,45.9,-61.9,47.1], PEI + eastern Nova Scotia — chosen small/fast, sits over Ortova's not-yet-covered geography so it proves the mechanics without touching real spine data):

• --plan: 112,181 buildings (COUNT pushdown, no data read) — reconciled against a hand-run duckdb query against the same S3 path (bfnu85sft): identical count.
• Real extract (--local-only): 112,181 rows, 14.0 MB parquet, 2.6 min. Independent shasum -a 256 on the output file matched the sha256 the script wrote into MANIFEST.json (95618d3a...513502e).

release-sync, partition verdun_test (bbox [-73.6044,45.423,-73.5158,45.4816], added specifically because it's the ONE tile that overlaps live canonical.property rows — 10,080 Verdun properties, queried live via arrondissement ILIKE '%verdun%'): 26,082 buildings (--plan).

tile-bake, fed the atlantic_test parquet (--local-only): 112,181 buildings → 12.5 MB mbtiles → 12.2 MB PMTiles, 6 seconds wall-clock. pmtiles show confirms a valid tileset: zoom 2–12, 366 addressed tiles, bounds matching the source bbox.

attach-join fixture test (attach-join.test.ts, pnpm test / tsx --test attach-join.test.ts): builds 3 synthetic buildings + 2 synthetic properties (one with a QC parcel whose geometry contains a building's centroid while the property's OWN centroid is deliberately OUTSIDE that building — proving the parcel-contains rule is a genuinely different match path, not a redundant special case of centroid-in-building; one with no parcel_id, centroid-in-building only) and runs the EXACT joinSql() used in production through the real DuckDB binary. Asserts: 1 QC match, 1 centroid match, 2 distinct buildings, the untouched 3rd building excluded. Passing.

attach-join live reconciliation: canonical.property_footprint's constraint (UNIQUE (property_id, footprint_id, source)) and canonical.building_height's (UNIQUE (toit_id, lidar_source)) were read LIVE via pg_get_constraintdef and confirmed to match this worker's loaderSql() byte-for-byte against migration 0249's already-proven INSERT shape (0249 committed 2,661,560 US property_footprint rows this exact way). A full DuckDB-join-to-DB-write dry-run against a real metro's staged data (the tile's ask) needs one pinned buildings partition covering that metro — verdun_test's partition is the first one that overlaps live spine data; attach-join.ts --scope verdun --release 2026-06-17.0 --buildings <verdun_test.parquet> --dry-run is the exact ready-to-run command (see --dry-run output shape below) once that partition finishes extracting.

Sharding + resume (tile 431 follow-up, 2026-07-14)

Why: atlantic_test's measured tile (112,181 buildings, 2.6 min) and verdun_test's (26,082 buildings, 3.3 min) are both BELOW a full 5°×5° grid cell's average density (~150M NA buildings / 288 land tiles ≈ 520k/tile) — neither is even a full-size grid cell (both are smaller custom test bboxes, lib/na-grid.ts's TEST_PARTITIONS, not the production grid). A full-NA release-sync sizes to somewhere between "14-24 runner-hours" (if wall-clock is dominated by DuckDB's per-tile S3-scan overhead, which my two samples' near-constant 2.6-3.3 min regardless of row count hints at) and "40-60 runner-hours" (if wall-clock scales with row count at average density, scaling my faster sample's 43,146 buildings/min up to 520k buildings/tile ≈ 12 min/tile × 288 tiles ≈ 58 hours) — either way, one unsharded dispatch cannot fit a single GitHub-hosted job's 360-minute hard cap.

• --shard <index>/<count> (release-sync.ts, lib/na-grid.ts's shardTiles()) slices the 288-tile grid into count disjoint, order-stable slices — tiles[i] goes to shard i % count. Fixture-tested (lib/na-grid.test.ts): every shard 0..count-1 is disjoint, their union is the full grid, and the split is stable across re-computation (no hidden randomness). At count=12, 288/12 = exactly 24 tiles/shard (verified live: --shard 3/12 reports tiles=24).
• Resume: before extracting a tile, release-sync.ts does a cheap destination check (lib/dest.ts's existsAt/fetchJson — a HEAD/GET on a tiny per-partition sidecar JSON, NEVER a re-download of the parquet itself). If the tile is already pinned, it's skipped and its recorded metadata is reused. Proven live: ran --partition atlantic_test twice against the same local destination — first run: 4.7 min (real extract, sidecar published); second run (identical command): 0.0 min, logged already pinned (resume) — 112,181 rows ... skipping re-extract, Total: 1 non-empty partitions (1 resumed from a prior pin, 0 newly extracted). This is what turns a cap-death/eviction on shard N into a fast no-op tail on re-dispatch, not a redo of already-finished tiles.
• Manifests, sharded: a global MANIFEST.json assumed ONE writer; N concurrent shards writing it directly would race the same object key. Each --shard <i>/<m> run instead publishes its OWN partial manifest to overture/<release>/manifest/shard-<i>-of-<m>.json. lib/merge-manifests.ts reads every shard's partial manifest and combines them into the final overture/<release>/MANIFEST.json — it refuses a partial merge (non-zero exit, unless --plan) if any shard 0..count-1 is missing, rather than silently publishing a manifest that under-claims coverage. Proven live with synthetic 2-shard fixtures: merged 2 partitions/30 rows correctly; a 3rd-shard-missing run correctly listed all 3 as MISSING and refused. An UNSHARDED run (no --shard) is unchanged — it's the sole writer, so it still publishes MANIFEST.json directly.

Per-shard wall-clock estimate (honest range, not false precision): at the CONSTANT-overhead-dominated floor (my two samples' 2.6-3.3 min/tile regardless of row count), 24 tiles/shard ≈ 79-96 min/shard — comfortably inside the 300-min hosted cap. At the ROW-COUNT-scaled ceiling (43,146 buildings/min applied to the 520k/tile NA average), 24 tiles/shard ≈ **290 min/shard** — inside the 300-min cap but with almost no margin, and a single denser-than-average tile could push it over. Recommendation: dispatch the first 1-2 shards on ubuntu-latest and WATCH the real wall-clock (gh run watch); if any shard approaches the cap, either re-dispatch the remaining shards on a self-hosted runner (-f runner=ortova-mac, see the caveat below) or raise shardCount (more, smaller shards) — resume makes either recovery path free (already-pinned tiles never re-extract).

Runner input — correcting the record: the workflow's runner choice (ubuntu-latest default, ortova-mac self-hosted option) is a NEW input in this drop, not a mirror of an existing apps/worker/permit-ingester GitHub Actions pattern — permit-ingest.yml has no runner input today (verified: runs-on: ubuntu-latest is hardcoded there). ortova-mac is listed as a CANDIDATE label only; it does nothing unless a self-hosted runner is actually registered under that exact label (repo Settings > Actions > Runners) — registering/operating that runner is infrastructure ops this drop does not perform or verify.

Adding a partition / release

• New NA tile: nothing to add — lib/na-grid.ts's 5° grid already covers the continent; a new city's buildings land automatically on the next release-sync run for a new release tag.
• New release: RELEASE=<new-tag> — no code change; MANIFEST.json namespaces every partition under overture/<release>/.
• New test partition (for a local proof over a specific metro before a full NA run): add a { key, bbox } to TEST_PARTITIONS in lib/na-grid.ts.

The canon wiring

• meta.dataset_catalog's external-parquet locator shape + the drafted companion-table migration for pinned-layer rows: see migrations/0363_external_layer_catalog.sql (DRAFT, not applied).
• pnpm validate:wiring's external-layer-pinned check (scripts/validate-arch/wiring.ts): asserts a MANIFEST.json exists at the configured destination and its checksums match, skippable-with-warning when no R2 env / no manifest yet exists (so CI stays green pre-first-run).
• The retirement note for raw.us_overture_building / raw.qc_overture_building is drafted in tile 431's outcome, not executed — their drop is a LATER gated op, after the trinity is verified end-to-end in production.

Deploy (Felix-gated — drafted, not run)

Primary: GitHub Actions, .github/workflows/geo-trinity.yml (mirrors geo-ingest.yml's shape — free runner, ca-central-1-adjacent AWS/Cloudflare edge, always runs merged main). Dispatch:

gh workflow run geo-trinity.yml -f job=release-sync -f release=2026-06-17.0 -f shard=0/12
gh workflow run geo-trinity.yml -f job=attach-join -f release=2026-06-17.0 -f scope=us_nyc -f buildings=<r2-key-or-local-path> -f dryRun=true
gh workflow run geo-trinity.yml -f job=tile-bake -f release=2026-06-17.0 -f buildings=<r2-key-or-local-path>
gh workflow run geo-trinity.yml -f job=merge-manifests -f release=2026-06-17.0 -f shardCount=12

The full 12-shard NA release-sync dispatch (one gh workflow run per shard, all disjoint, each independently resumable — the Release Agent runs these, not this drop):

for i in 0 1 2 3 4 5 6 7 8 9 10 11; do
  gh workflow run geo-trinity.yml -f job=release-sync -f release=2026-06-17.0 \
    -f shard=${i}/12 -f runner=ortova-mac
done
# then, once all 12 have completed (watch each with `gh run watch`):
gh workflow run geo-trinity.yml -f job=merge-manifests -f release=2026-06-17.0 -f shardCount=12

(-f runner=ortova-mac per the coordinator's ask — see the caveat above: this only works once that label is a real registered runner; drop the flag, or set -f runner=ubuntu-latest, to dispatch on the GitHub-hosted default instead, watching the first shard or two before trusting the 300-min cap at scale.)

R2 secrets (R2_ENDPOINT, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET) are already in .env.local + GitHub Actions secrets (Felix, 2026-07-14) — this workflow is ready to dispatch on Felix/Release-Agent's go. No new Railway service (trial exhausted, no new services per docs/architecture/cloud-execution-paths.md); a Dockerfile is still provided here for the always-on-fallback Fly.io path if a standing service is ever warranted, but GitHub Actions run-to-completion is the primary path for all three jobs (none of them is a long-running service).

apps/worker/governance-parser/README.md

governance-parser — QC council résolutions + règlements, parsed from held PVs

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: extracts council résolutions/règlements from held minutes PVs (LLM + rules).
• Input boundary (Layer-1): CLI --muni / --limit / --apply. Hand types PvMeta / Extracted* in types.ts — no Zod schema yet (FLAGGED, boundary table §C; Zod is follow-up).
• Output: governance facts → DB. DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/governance-parser dry-run (also classify-dry-run, extract-join-dry-run).
• Deploy: not in WORKER_SPECS / no dedicated workflow yet; runs on the ortova-mac fleet.

DEPTH play of the QC Municipal Governance Corpus (master map Tier 1: docs/research/qc-minutes-lane/2026-07-18-qc-municipal-governance-corpus-map.md). Turns the 66,617 procès-verbaux we already hold (raw.qc_minutes_documents, plain text at text_object_uri) into the higher-value résolution (numbered council decisions) and règlement (by-law lifecycle) folders — ZERO new scraping.

• Decision it informs: are the résolution + règlement folders fillable at quality from the PV text we already have, and for how many munis?
• Action it unblocks: filling the resolution + reglement folders (foundation canon 2026-07-19-qc-governance-corpus-foundation.md) for the 635 PV-covered municipalities, feeding canonical.entity_resolution / canonical.entity_reglement (migrations 0386 / 0387).

Lives in apps/worker/ (self-contained, non-workspace — like minutes-harvester) because the work is bulk text parsing + fetch. Install standalone: pnpm install --ignore-workspace.

Files

• patterns.ts — the French council-minutes regexes (résolution number schemes, proposé/appuyé, adoption markers; règlement number + stage keywords).
• parse-pv.ts — pure, deterministic core: parsePv(text, meta) → résolutions + règlements. Idempotent (same text → same external_id + content_hash).
• types.ts — the record shapes (mirror the raw tables 0386/0387).
• run-dryrun.ts — read-only measurement (samples PVs, parses, reports yield/precision + a hand-check sample). No writes.
• ingest.ts — Felix-gated writer + reconcile (--apply --i-am-felix); default dry-run. UPSERTs raw + reconciles to canonical on the MAMH code.
• parse-pv.test.ts — unit test (three schemes, stages, precision guard, idempotency).

How résolutions are anchored

A résolution is a block led by a resolution-number token at the start of a line. Numbering schemes vary by municipality; the parser matches (in order): slash_nnnn_mm_yy (0064/02/21), alpha_yyyy_n (VS-AJ-2020-1, CM-2026-123), spaced_alpha (CM26 02 064), yyyymmdd_seq (20250811-01), yyyy_mm_dd_dot (2021-11-22.01), date_seq (2025-11-17-29), dot_seq_yymmdd (150504.094), yyyy_mm_seq (2022-03-44, 2025-07-202), yy_mm_nnn (21-11-901, 24-04-11265), keyword (RÉSOLUTION 2024-80). Precision guard: a block is accepted only if it carries a resolution signal (il est résolu / sur proposition / attendu que / considérant / an adoption marker).

How règlements are gated

A règlement number is emitted as a lifecycle event only when a STAGE keyword (avis de motion / adoption du règlement / projet de règlement / second projet / abrogation) sits with it. A bare reference — dérogation mineure au règlement de zonage no 4300 — is NOT an event and is excluded. That gate is the drawer's precision guard (verified: 30/30 hand-checked règlements were real proceedings, zero bare references leaked).

Measured (dry-run, 40 munis × 6 PVs, 2026-07-19, extractor v1)

• Résolutions: 2,135 across 240 PVs = 8.9 / PV avg (median 1; the mean is carried by full council munis, tens per PV); 33 / 40 munis yielded ≥1. Precision: 30/30 hand-checked = 100% real numbered council resolutions.
• Règlements: 243 events = ~1.0 / PV; 31 / 40 munis yielded ≥1; stages avis_motion/projet/projet_2/adoption/abrogation. Precision 30/30 = 100% real proceedings; number field exact 25/30 (5 truncate a compound number or grab a referenced by-law — a v2 item).
• Muni → spine join: 635/635 = 100% (deterministic MAMH-code resolve).

Honest limits (the ~18% of munis that yield little/nothing)

1. Misclassified raw docs — some doc_type='proces-verbal' rows are actually lab-quality certificates or avis-publics filed under the PV type (e.g. East Angus 41060). The parser correctly yields zero; this is a corpus-quality issue (ZONE 1 Pristine / harvester classifier), not a parser gap.
2. Narrative-style minutes without a per-item number at the margin (résolutions embedded inline) — genuinely low-yield, mostly small munis.
3. OCR-degraded scans — the dry-run sampled text_method='text-layer'; OCR munis parse worse and are excluded from the numbers above.
4. Number-field truncation on compound règlement numbers — a documented v2 fix.

Projection (order-of-magnitude, caveated)

8.9 rés/PV × 66,617 held PVs ≈ ~450k–600k résolution records (lower than the naive product: OCR munis + the misclassified/narrative tail parse below the text-layer sample average). 1.0 règl/PV ≈ **50k–70k règlement events**. Of the 635 PV-covered munis, an estimated ~500–525 gain a populated résolution folder and ~480–500 a règlement folder (from 33/40 and 31/40 in-sample).

Attribution: Municipal council minutes (procès-verbaux), municipality of origin.

apps/worker/libpostal-sidecar/README.md

libpostal HTTP sidecar

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: a Python/FastAPI address-parsing sidecar wrapping libpostal.
• Input/output boundary (Layer-1): HTTP POST /parse — ParseRequest / ParseResponse defined as pydantic models (Python). FLAGGED (boundary table §C): out of Zod scope — pydantic is this unit's schema authority.
• Run locally: docker build then docker run (see body); container serves /parse.
• Deploy: container image (was Railway — retired; runs as a container next to the DB).

FastAPI wrapper around libpostal + pypostal bindings. Deployed on Railway Hobby as a long-running process shared across all Nexod platform source adapters that need address canonicalization.

API

• POST /parse — body { "address": "<string>" } returns { "components": { "road": "...", "house_number": "...", ... } }.
• GET /health — Railway health check endpoint.

Auth: set LIBPOSTAL_API_KEY on Railway; clients pass it in X-API-Key. Without the env var the endpoint accepts any caller (local dev only).

Running locally

cd apps/overwatch-worker/libpostal-sidecar
docker build -t nexod-libpostal .
docker run --rm -p 8080:8080 nexod-libpostal
# Smoke:
curl -X POST http://localhost:8080/parse \
  -H 'content-type: application/json' \
  -d '{"address":"100 boulevard Saint-Laurent Montréal"}'

Expected: status 200 with a components dict containing road, house_number, and city.

Deploying to Railway

1. railway login
2. railway init in this directory and link it to a new service.
3. Add the env var LIBPOSTAL_API_KEY (any random 32+ char string). Keep it matched with LIBPOSTAL_API_KEY configured for the Nexod platform.
4. railway up — the build will compile libpostal from source (~8 minutes first time, cached thereafter). Health check at /health must return 200 before Railway marks the deploy ready.
5. Copy the public domain into LIBPOSTAL_URL on the Nexod dashboard env.

Wiring into the platform

import { createHttpLibpostalClient, canonicalizeAddress } from '@nexod/platform/adapters';

const libpostal = createHttpLibpostalClient({
  baseUrl: process.env.LIBPOSTAL_URL!,
  apiKey: process.env.LIBPOSTAL_API_KEY!,
});

const addr = await canonicalizeAddress('100 boulevard Saint-Laurent', {
  cityKey: 'montreal',
  libpostal,
});
// addr.method === 'libpostal_hybrid' when sidecar reachable
// addr.method === 'rule_fallback' when sidecar down or env vars absent

The client returns null on any network/auth failure so the adapter falls through to rule_fallback cleanly. Sidecar downtime degrades accuracy from ~96% to ~93% on the 50-address fixture; it does NOT break ingestion.

Memory footprint

libpostal loads its ~2 GB trained model into memory on process start. The Railway Hobby plan allocates 8 GB RAM; a single sidecar replica fits comfortably. Cold start is ~5–8 seconds while the model hydrates from disk cache (first start after deploy is slower — the healthcheck timeout is set to 300 s to accommodate this).

apps/worker/lidar-ingester/README.md

@ortova/lidar-ingester — the LiDAR index-drawer loader

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: acquires + stamps the QC LiDAR tile index (the height/point-cloud substrate).
• Input boundary (Layer-1): CLI --i-am-felix / --plan; reads canonical.lidar_tile. No Zod schema (FLAGGED, boundary table §C): env-only input, low shape surface.
• Output: LiDAR index rows → canonical.lidar_tile. DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/lidar-ingester acquire (also stamp-tiles).
• Deploy: gh workflow run lidar-acquire.yml (runs-on ortova-mac).

The metadata loader for migration 0397 (canonical.lidar_acquisition + canonical.lidar_tile

• their raw.* landings). It seeds the index drawer that answers "what LiDAR covers parcel X" — LiDAR mission P0a (docs/ops/2026-07-21-lidar-raw-layer-mission.md). Metadata only. No point-cloud bytes are ever downloaded or stored (INV-0; the drawer carries the fetch URI, never the tile).

The three doors (source_program)

Contract

• --dry-run is the DEFAULT. It parses the source, builds the raw + canonical UPSERT rows, prints the counts + a sample, and writes the full .sql to --out. It writes nothing to the DB — safe to run before migration 0397 is applied.
• --apply --i-am-felix pipes the generated SQL to psql over SUPABASE_POSTGRES_POOLER_URL in keyset batches (idempotent ON CONFLICT on the natural keys). Felix-gated (Rule 6); requires 0397 applied. Sub-agents never run this.
• Idempotent. Natural keys — acquisition (source_program, project_key), tile (source_program, project_key, tile_name) — so a re-run is an UPSERT, never a duplicate.
• After a bulk load: refresh the coverage gauge (Felix-gated): REFRESH MATERIALIZED VIEW CONCURRENTLY serving.lidar_coverage_by_acquisition;

Usage

# federal dry-run (uses/caches the two small index files under --cache-dir):
python3 load_index.py --source federal --dry-run --out /tmp/lidar_federal.sql

# provincial acquisition roster (live WFS), dry-run:
python3 load_index.py --source provincial --dry-run --out /tmp/lidar_provincial.sql

# apply (Felix only, after 0397 applied):
python3 load_index.py --source federal --apply --i-am-felix

Fidelity notes (carried into the drawer)

• Density is two numbers. Federal LDR_AGGREGATE_DENSITY = all-returns aggregate (CMM 2023 = 20.83 pts/m²); provincial DENSITE = nominal spec (CMM = 15). Both stored; density_basis says which. Never compare across bases as equal.
• Same flight, two doors. A CMM tile exists under both federal-copc (COPC) and mrnf-laz (LAZ). The drawer holds both, keyed by (source_program, ...); not collapsed.
• Federal tile footprints live in the 407 MB Index_LiDARtiles_tuileslidar.gpkg (over the metadata fence) — federal tile rows land with geom NULL; the acquisition footprint carries coverage. Full provincial per-tile footprint ingest is the lidar-acquire front's bulk job.

P1 ACQUIRE — acquire.ts (federal COPC → R2)

Tile 467 (Felix, 2026-07-21): R2 approved (~$15/mo metro tier). acquire.ts copies federal COPC tiles from the free anon S3 bucket into our R2 bucket, byte-for-byte (no format conversion — federal tiles are already COPC). Provincial (LAZ, not COPC) needs an untwine/PDAL convert step first — the seam for that is documented in acquire.ts's header but not built; only federal-copc tiles are acquired today.

Key layout (forever)

lidar/<source_program>/<project_key>/<basename-of-source_uri>
e.g. lidar/federal-copc/600023_52_CMM_2024/24_2495039f08_dc.copc.laz

source_program first (a CMM tile exists as both a federal-copc COPC row and an mrnf-laz LAZ row — the prefix keeps the two doors from ever colliding), project_key next (mirrors the federal bucket's own layout 1:1 — eyeball-mappable, prefix-listable per acquisition), tile filename last, unchanged. Full justification in acquire.ts's header comment.

Usage

# read-only plan (DB read + R2 HEAD only, no transfer):
tsx acquire.ts --project 600023_52_CMM_2024 --limit 10 --plan

# execute (Felix-gated — an R2 PUT is an outbound spend, Rule 6):
tsx acquire.ts --project 600023_52_CMM_2024 --limit 10 --i-am-felix

# bypass the DB (proof/testing against tiles a still-loading drawer doesn't hold yet):
tsx acquire.ts --manual /tmp/tiles.json --i-am-felix

# stage (never apply) the canonical.lidar_tile UPDATE that stamps the ledger's outcome:
tsx stage-db-stamp.ts --out /tmp/lidar-tile-stamp.sql

Resumable (a HEAD check against R2 skips a tile already landed at the expected size), paced at ≤3 concurrent transfers (hard cap in acquire.ts), per-tile ledger at .acquire-ledger.json (gitignored, runtime state).

Proof (2026-07-21, 3 CMM_2024 tiles, R2 bucket ortova-geo)

All three confirmed present in R2 at the exact source byte count (aws s3 ls against the R2 endpoint). HTTP range-read proof (the actual point of landing COPC in our own bucket — a presigned GET against the R2-hosted object, Range: bytes=0-1023 and a mid-file Range: bytes=100000-100511):

{ "status": 206, "contentRange": "bytes 0-1023/2867985", "bodyBytes": 1024 }
{ "status": 206, "contentRange": "bytes 100000-100511/190510", "bodyBytes": 512 }

Real 206 Partial Content + correct Content-Range, confirming R2 serves COPC octree range-reads without any public-read/custom-domain change — a presigned SigV4 GET (via the aws CLI's own s3 presign, not a new SDK dependency — see lib/r2.ts's header) is enough.

The one DB write this pipeline stages (never applies)

acquire.ts never touches canonical.lidar_tile. stage-db-stamp.ts reads the local ledger and writes an idempotent UPDATE ... SET our_store_uri, checksum, ingest_status='acquired' statement per acquired/verified tile to a .sql file for the release lane to review and run separately (Rule 6 — DB writes are Felix-gated).

Bulk numbers (2026-07-21 measurement + projection)

Measured this session (single-stream, this laptop, under heavy load tonight — not representative of a clean environment): 57.87 MB in 100.2 s ≈ 0.58 MB/s; a smaller 11.3 MB tile ran at 0.38 MB/s (fixed per-tile overhead dominates on small files). At the pipeline's 3-concurrency cap, and assuming roughly linear scaling, call it ~1.5–1.7 MB/s aggregate on this machine tonight.

Recommendation: run the bulk transfer cloud-side via GitHub Actions, not the self-hosted ortova-mac runner. The ortova-mac runner is this same physical laptop (.env.local access is its only advantage) — exactly the machine measured as bandwidth/load-constrained tonight. A hosted GH Actions runner has no direct .env.local, but the only credentials this pipeline needs (R2_ENDPOINT / R2_ACCESS_KEY_ID / R2_SECRET_ACCESS_KEY / R2_BUCKET) are easy to hold as encrypted repo secrets — the federal S3 read is anonymous, no credentials at all. Per docs/architecture/cloud-execution-paths.md, GH Actions is the default batch-geo-ingest path. Built (2026-07-21, tile 467 follow-up): --tier cmm-metro --shard <i>/<m> on acquire.ts (deterministic tiles[i] -> shard i % m partitioning, lib/shard.ts — geo-trinity's release-sync.ts/lib/na-grid.ts precedent mirrored verbatim; fixture-tested in lib/shard.test.ts against a synthetic 4,920-tile set: every shard 0..count-1 disjoint, union = full set, stable re-computation, both even (4920/8) and uneven (4920/7) splits proven) + .github/workflows/ lidar-acquire.yml (workflow_dispatch, inputs tier/shard_index/shard_total).

Per-shard wall time (8 shards, cmm-metro tier)

4,920 tiles / 1.17 TB (exact, measured) ÷ 8 shards = 615 tiles/shard, ≈146.5 GB/shard. At the README's own conservative cloud-runner rate (30 MB/s aggregate, 3-way concurrency, ~10 MB/s/stream — the same number the "Bulk numbers" table above used for the full-tier 11-hour estimate):

146,488.6 MB / 30 MB/s = 4,883 s ≈ 81 minutes/shard

Comfortably inside the workflow's 200-minute hard cap (the spend ceiling by construction — GitHub Actions bills per runner-minute) with wide margin for real-world variance (S3 throttling, transient retries, per-tile fixed overhead on the smaller tiles). Total wall time for all 8 shards run in parallel: ≈81-100 minutes (not 8x serial — every shard is an independent job). Total GitHub Actions runner-minutes billed: ≈8 × 90 min ≈ 720 min ≈ $5.76 at the $0.008/min Linux overage rate (cloud-execution-paths.md) — comfortably inside the $15 cap Felix approved (tile 467), with margin for a slower-than-projected real run or a retry.

Fire commands (the Release Agent runs these on Felix's go — NOT fired by this drop)

for i in 0 1 2 3 4 5 6 7; do
  gh workflow run lidar-acquire.yml -f tier=cmm-metro -f shard_index=$i -f shard_total=8
done
gh run watch   # tail the first shard or two before trusting the projection above

After all 8 shards land (watch gh run list --workflow=lidar-acquire.yml for 8 green runs), reconcile the DB against the ACTUAL R2 listing with the stamp loader (below) — never a hand-assembled SQL file from the 8 scattered per-shard ledger artifacts.

The STAMP LOADER — stamp-tiles.ts (production path, supersedes stage-db-stamp.ts)

The durable fix for the one-off-SQL-file pattern: stage-db-stamp.ts reads ONE local .acquire-ledger.json, which a sharded cloud dispatch scatters across 8 separate runners (each uploading its own ledger as a run artifact) — reassembling those by hand before every stamp is the exact ceremony this loader kills. stamp-tiles.ts instead reconciles canonical.lidar_tile against the ACTUAL R2 bucket listing (ListObjectsV2, ground truth) — no ledger needed at all, from any shard, any manual run, any future re-run.

# read-only: DB read + R2 listing only, prints exact would-stamp / already-correct / not-in-r2 counts
tsx stamp-tiles.ts --dry-run

# apply (Felix-gated — UPDATE canonical.lidar_tile, Rule 6):
tsx stamp-tiles.ts --apply --i-am-felix

# scope the DB read to one tier (default is unscoped — cheap even so, one SELECT):
tsx stamp-tiles.ts --tier cmm-metro --dry-run

Batch-committed (500 rows/transaction), natural-key UPDATE (idempotent — a re-run over already-correct rows is a no-op). Proof against prod (2026-07-21, before any bulk shard has run): already-correct=5 would-stamp=0 not-in-r2=34994 — the exact 3 proof tiles from the table above plus 2 more from an earlier local proof run, all already correctly stamped by stage-db-stamp.ts; zero new stamps until the bulk shards actually land tiles in R2.

apps/worker/minutes-harvester/README.md

minutes-harvester — Stage 2 of the QC meeting-minutes lane

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: harvests QC municipal meeting-minutes PDFs into raw landing (the live Watch front).
• Input boundary (Layer-1): WorkflowArgsSchemas['minutes-ingest'] = { municipality } and WorkflowArgsSchemas['minutes-discovery'] = { codes } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema on the queue.
• Output: raw.qc_minutes_* rows. DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/minutes-harvester plan / ingest / local.
• Deploy: gh workflow run minutes-ingest.yml -f municipality=<code> (runs-on ortova-mac).

Pulls every council-minutes / avis-public / permit-bulletin PDF a QC municipality publishes on its own open web archive, and stores the bytes verbatim in durable object storage, registering each one in raw.qc_minutes_documents (migration 0244_qc_minutes_documents.sql, DRAFT, Felix-gated apply).

This worker does NOT extract. It does not itemize dérogation-mineure / PIIA / démolition records, does not resolve an address to property_id, and does not write any canonical satellite. That is Stage 3 — future SPINE work (see "Path to Stage 3" below). Felix's directive for Stage 2 is explicit: no segmentation, pull every municipality, full history, store verbatim, stop there.

• Decision it informs: is the council-minutes lane's supply real at full-history scale (not just the 1-2-document Stage-1 census sample)?
• Action it unblocks: a Stage 3 extraction pass (dérogation-mineure / PIIA / démolition itemization → a canonical satellite) has a durable, verbatim corpus to run against instead of re-fetching the web on every experiment.

Census this worker operationalizes: docs/research/qc-minutes-lane/2026-07-13-qc-meeting-minutes-census.md (35 no-open-permit-feed municipalities + the Gatineau permit-bulletin bonus).

Run modes

# read-only dry-run: discover candidate PDFs + resume-check against the DB,
# print counts. NO fetch of document bytes, NO writes anywhere. Always safe.
MUNICIPALITY=Beloeil npx tsx ingest.ts --plan

# proof mode: fetch + hash + OCR + store to LOCAL DISK (OUT_DIR), write a
# local manifest.json. NO Blob upload, NO DB write. This is how the harvester
# is verified end-to-end without a Felix-gated write.
MUNICIPALITY=Beloeil OUT_DIR=./out npx tsx ingest.ts --local-only

# the real run: Blob upload + raw.qc_minutes_documents INSERT. Felix-gated —
# the Release Agent runs this (via .github/workflows/minutes-ingest.yml),
# never a SCALE agent against prod.
MUNICIPALITY=ALL SUPABASE_POSTGRES_POOLER_URL=... BLOB_READ_WRITE_TOKEN=... \
  npx tsx ingest.ts --i-am-felix

MUNICIPALITY accepts a municipality name (Beloeil, Vaudreuil-Dorion) or its MAMH code (57040), or ALL to run every configured entry in the registry in one process (fine for --plan; for --i-am-felix the sanctioned path is one municipality per GitHub Actions dispatch — see below — so a bad regex on one town fails small, not the whole province).

The registry (municipalities.ts)

One entry per censused municipality. Two confidence tiers, both real inventory (Felix's directive is "pull EVERY municipality" — the registry covers all 35 + Gatineau):

• verified: true (8 entries) — seeds were derived from an ACTUAL live fetch of the archive page during this build (2026-07-13) and the PDF hrefs were inspected by hand: Drummondville, Rimouski, Beloeil, Vaudreuil-Dorion, Mascouche, Saint-Eustache, Sainte-Julie, Saguenay (the last two copied verbatim from apps/worker/permit-doc-parser/city-configs.ts, already proven there). Safe to dispatch as-is.
• verified: false (27 entries) — a real portal URL (root domain, or the actual conseil/séances page where hand-navigation found it) but the exact PDF-link include regex was NOT live-derived in this pass (seeds: []). Dispatching one of these with --plan is safe (read-only) and is exactly how a future session completes the entry: a --plan that discovers 0 PDFs is the honest signal to go add a real seed (mirror a verified: true entry above), never a silent wrong write. notes on each stub carries the Stage-1 census's own hint.

Hard cases (5, flagged not silently dropped): Repentigny + Shawinigan (scanned-ocr — image-only PDFs, this worker's OCR path still ACQUIRES them fine, text_method reads ocr/mixed), Granby (waf-blocked — every automated fetch 403s despite permissive robots.txt, needs a headless-browser fetch strategy this worker does not implement), Trois-Rivières + Victoriaville (js-portal — minutes sit behind a JS document-search widget, no link-guessable index page; each has a documented possible side-channel in its notes). All five are follow-up SCALE tiles, not silent gaps.

Measured (2026-07-13, MUNICIPALITY=ALL --plan against the live pooler, real counts)

Full MUNICIPALITY=ALL --plan run also correctly reported the 27 registry stubs by name (each printing its census hint / "needs a real seed" note) and the 5 hard cases by name + reason — one process pass over the ENTIRE 35-town

• Gatineau registry, zero crashes, zero writes.

These are discovery counts (candidate PDF urls found), not yet acquisition counts — see "Proof-pull" below for the 3 municipalities actually downloaded + hashed + stored + would-be-registered end to end. The scale here (hundreds of documents per city, not the 1-2 the Stage-1 census sampled) is the concrete answer to "is full history real" — yes, and it is far bigger than a spot-check suggested.

Proof-pull (2026-07-13, --local-only, real files on disk — not just written, opened)

Three municipalities acquired end to end (discover → download → sha256 hash → OCR/text-extract → store → would-be-registration record in a local manifest — everything --i-am-felix does except the Blob upload + the real DB write):

INV-4 resumability demonstrated live, not just described: re-running the IDENTICAL Vaudreuil-Dorion --max 15 command a second time reported already known: 15 (skipped), net-new: 781 (down from 796), and acquired the NEXT 15 documents — proving both (a) a re-dispatch skips everything already held and (b) it makes forward progress rather than reprocessing the same content. The local manifest after both runs holds 30 entries, 30 distinct content_hash values, 60 files on disk (30 PDFs + 30 cached .txt siblings).

Content verified, not just byte-counted: the cached text of a Beloeil document contains real itemized council content — grep -i "dérogation" <cached .txt> returns:

12. AUDITION DES PERSONNES INTÉRESSÉES ET AUTORISATION D'UNE DEMANDE DE DÉROGATION
        CONSIDÉRANT la réception d'une demande de dérogation mineure (DM-2026-9037) pour la
dérogations mineures;
        D'autoriser la dérogation mineure numéro DM-2026-9037 telle que demandée pour le

— confirming the Stage-1 census's finding (itemized, addressed dérogation- mineure records) survives the actual harvest, not just the sample read.

A real bug the proof-pull caught (documented, not hidden): Vaudreuil- Dorion's archive page declares <base href="/" />; the first pass ignored it and resolved every relative href against the wrong path segment, so all 15 downloads 404'd. extractLinks() now honors a page's own <base> tag — exactly the kind of defect a real end-to-end proof-pull surfaces that a regex-only "looks right" read would not.

Storage choice: Vercel Blob (justified)

storage.ts uploads to Vercel Blob (@vercel/blob), the SAME backend apps/worker/tile-builder already uses in production for its PMTiles uploads, with a live BLOB_READ_WRITE_TOKEN secret already provisioned. Reasons this beats standing up a new S3 bucket for this specific corpus:

1. Zero new infrastructure. No new AWS IAM role/bucket/policy for Felix to provision — the secret already exists and is already used by a sibling worker in this exact repo.
2. Right-sized for THIS corpus. Even at full 35-municipality/full-history scale, this is thousands of PDFs at a few hundred KB–2 MB each — a few GB total, nowhere near the province-scale (hundreds of GB, cadastre ≈ 3.7 GB) pivot point the SCALE playbook reserves for S3/GDAL bulk-geo work. Blob's per-object simplicity (a URL, no bucket/key IAM ceremony) fits a document-per-row corpus better than S3's multipart/streaming machinery, which tile-builder needs only because a single .pmtiles file there can be 600+ MB.
3. Content-addressed keys make re-uploads free. documentKey() names every object qc-minutes/<municipality_code>/<sha256>.pdf — the SAME content always lands at the SAME key, so allowOverwrite: false on the Blob put() call is a correctness backstop (an attempted re-upload of already-stored bytes is a structural no-op, not a race).

s3:// is the documented fallback shape (matching canonical.imagery_acquisition.s3_uri / canonical.property_scene.s3_uri, migrations 0073/0009) if a future migration standardizes all raw-document storage on S3 — storage.ts's ObjectStore interface does not leak the Blob SDK outside this one file, so swapping backends later is a one-file change, not a rewrite.

--local-only writes to local disk instead (explicitly durable: false in storage.ts) — this is proof-mode only, never how the real harvest runs.

Idempotent + resumable (INV-4), by construction

• Resume-check (knownUrlsFromDb): before fetching ANY document byte, the worker reads every source_url (+ also_seen_at_urls alias) already registered for this municipality_code — a free SELECT (Rule 6) — and skips re-downloading/re-uploading/re-writing anything already held. A re-dispatch after a mid-run crash resumes exactly where it left off. --local-only uses a local manifest.json for the same purpose (so the proof-pull is resumable too, without a DB).
• content_hash is the DB's own idempotency key (UNIQUE constraint, migration 0244) even if the URL-skip were somehow bypassed: ON CONFLICT (content_hash) DO UPDATE never inserts a duplicate document row — it only ever appends an alias URL to also_seen_at_urls when the SAME bytes are found at a NEW url (a doc re-posted under a new path).
• Chunked writes: DOCS_PER_INSERT = 50 — the register step never issues one whole-municipality INSERT, even for Vaudreuil-Dorion's 796 documents (mirrors the geo/permit ingesters' per-batch-transaction discipline, never a province-size single statement).
• 429/5xx backoff: fetchTextBackoff / fetchBytesBackoff retry up to 5 times with exponential backoff (Retry-After honored on 429, capped at 30s), on both the index-page crawl and the document download. A 4xx other than 429 (404, 403) is treated as permanent and NOT retried — that document is recorded as failed, the run continues.
• Silent-0 guard: a --i-am-felix run against every attempted municipality (one with seeds.length > 0) that discovers 0 candidates on ALL of them fails the job (main()'s zero-yield check) — a broken regex or a site change must never exit green as "nothing to harvest."

Deploy config (drafted for Felix — GitHub Actions, no Railway)

Per the 2026-07-13 build-model decision (docs/architecture/cloud-execution-paths.md, overwatch.2026-07-13.001) — GitHub Actions is the default sanctioned batch runner; do not provision new Railway services. This worker therefore has no railway.json / Dockerfile — .github/workflows/minutes-ingest.yml IS its deploy config:

# one municipality at a time, plan first (always safe, read-only):
gh workflow run minutes-ingest.yml -f municipality=Beloeil -f mode=plan
gh run watch

# then the real harvest (Felix-gated — Release Agent only):
gh workflow run minutes-ingest.yml -f municipality=Beloeil -f mode=i-am-felix

The workflow installs poppler-utils + tesseract-ocr (+ fra/eng models)

• postgresql-client directly on the ubuntu-latest runner (no Docker image build), npm installs this worker's own package.json (it is NOT a pnpm-workspace member — see the repo-root pnpm-workspace.yaml comment — so a plain npm install here mirrors geo-ingester/tile-builder), then runs --plan (gate) then --i-am-felix (only when mode=i-am-felix).

Ordered plan for the Release Agent (after Felix approves the migration)

1. Apply migration 0244: scripts/apply-migration.ts --i-am-felix migrations/0244_qc_minutes_documents.sql (raw table + the one catalog row; no canonical satellite — see the migration's header for why).
2. Set secrets (if not already present from tile-builder): SUPABASE_POSTGRES_POOLER_URL, BLOB_READ_WRITE_TOKEN on the repo's GitHub Actions secrets.
3. Dispatch the 8 verified: true municipalities first, one at a time, mode=plan then mode=i-am-felix: Drummondville, Rimouski, Beloeil, Vaudreuil-Dorion, Mascouche, Saint-Eustache, Sainte-Julie, Saguenay.
4. For each of the 27 verified: false stubs: dispatch mode=plan first. A nonzero discovery count means the stub's portal URL was enough to at least reach a page (still confirm the regex is right before i-am-felix); a zero count means the seed needs a real regex — mirror one of the 8 validated entries in municipalities.ts (fetch the stub's feed_url, find the real archive page + PDF link pattern, same 15-30 line pattern as the entries above) before dispatching i-am-felix for that town.
5. The 5 hard cases (Repentigny, Shawinigan, Granby, Trois-Rivières, Victoriaville) need a follow-up SCALE tile (a headless-browser fetch strategy) before they can be dispatched at all — flagged, not silently scheduled.
6. Verify: SELECT municipality_code, count(*), min(fetched_at), max(fetched_at) FROM raw.qc_minutes_documents GROUP BY 1 ORDER BY 1; (read-only, Rule 6) after each dispatch — the same silent-0 discipline permit-ingest.yml uses.

Path to Stage 3 (future, NOT this worker's job)

apps/worker/permit-doc-parser already prototypes the itemization step (HARVEST → EXTRACT → JOIN → MEASURE) for a DIFFERENT 10-town list from an earlier research pass — its extract-records.ts (French-language DM/PIIA/ démolition record extraction) and join.ts (address/lot resolve to property_id, read-only) are the shape a Stage 3 pass over THIS worker's 35-municipality corpus would reuse. Wiring that up — reading raw.qc_minutes_documents, running the extractor, designing the canonical satellite this ONE table currently deliberately omits — is a SPINE (Zone 4) tile, not a SCALE one: SCALE runs the canonical-upsert at scale, it does not design what it writes.

apps/worker/permit-doc-parser/README.md

permit-doc-parser — lossless permit extraction from QC municipal PDFs

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: LLM extraction of structured permit records + feed classification from municipal PDFs.
• Input/output boundaries (Layer-1): PermitRecordSchema, DocExtractionSchema, CompletenessReportSchema, FeedClassificationSchema — rich Zod already in types.ts (boundary table §C).
• Run locally: pnpm --filter @ortova/permit-doc-parser parse (also classify, pipeline).
• Deploy: container / CLI on the ortova-mac fleet.

Most Québec municipalities do not publish permits as a data feed. Of ~40 cities surveyed, only 4 have a machine-readable feed (Montréal, Québec City, Laval, Longueuil — see docs/research/qc-permits/2026-05-28-municipality-coverage.md). The rest publish permit decisions inside French-language PDFs: council procès-verbaux (minutes), avis publics (public notices), and some permit-statistics reports. Those PDFs list address-level dérogation mineure, PIIA, démolition, construction, lotissement, and changement d'usage cases — the exact permit events the spine wants.

This worker turns those PDFs into structured permit records, and classifies each municipality's publication as a candidate feed.

• Decision it informs: do the ~14 QC towns previously judged "closed" become tappable permit feeds via document parsing?
• Action it unblocks: expanding QC permit coverage beyond Montréal / Laval / Québec by parsing council minutes + avis-publics PDFs.

It lives in apps/worker/ because the heavy work is PDF text extraction + OCR (the same reason geo-ingester / tile-builder are here). This is parser logic — it is NOT yet wired into the engine dispatchers. The output is a clean canonical.property_permit-shaped superset that a future source adapter feeds through the canonical-upsert; see "Path to engine wiring" below.

The full pipeline (BUILD 3, below) is HARVEST → EXTRACT → JOIN → MEASURE, per city, driven by city-configs.ts. Scaling to the other ~10 closed towns is adding a config, not writing code. Jump to "BUILD 3" for the runbook + the two-city end-to-end numbers.

BUILD 1 — the lossless extraction parser

Input: one municipal PDF (French). Output: a complete structured set of permit records + a completeness proof.

Pipeline (parse.ts → the modules):

1. Acquire, raw-native-first (fetch-doc.ts). Fetch the URL (or read a local path), hash it, and store the verbatim PDF + the verbatim extracted text under out/ before any structured extraction. Extraction derives over the stored raw, never the other way round (data-shape canon P2 / feedback_bought_imagery_persists_to_db).
2. Full-text extraction, every page, never capped (extract-text.ts). pdftotext -layout for the native text layer, split per page on the form-feed poppler emits. Pages whose text layer is empty are detected as scanned and OCR'd individually (pdftoppm -r 300 → tesseract -l fra+eng). No token or page limit anywhere; long docs are handled page-by-page so nothing is dropped.
3. Doc classification (classify-doc.ts): doc_type (proces-verbal | avis-public | ordre-du-jour | permit-stats) + doc_date from the filename/header.
4. Record extraction (extract-records.ts, behind the Extractor seam in llm-seam.ts). An anchor model reads both document shapes:
   • procès-verbal / ordre-du-jour: each item anchored by a demand id (2025-DM-289, 2026-PIIA-006, ranges …225 à …228 expanded), a numbered urbanism line with a locus, or a resolution number (CM26 02 064, 2025-11-17-29). Agenda + resolution mentions of the same demand merge.
   • avis public: one subject per notice — objet + IMMEUBLE VISÉ carry the locus; the meeting-venue address in the body is explicitly ignored.
5. Completeness proof (completeness.ts) — see below.

The superset schema (types.ts)

Every field is optional; anything unexpected lands in a raw_fields bag, so nothing valuable is truncated:

municipality, source_url, doc_type, doc_date,
item_type (derogation-mineure | piia | demolition | construction | renovation
           | lotissement | usage | other),
address, lot_number, matricule, cadastre, resolution_number, applicant,
declared_value, units, description, demand_id,
raw_excerpt (verbatim), page_no, extraction_confidence, raw_fields{}

The losslessness proof (the gap check)

completeness.ts scans the raw lines independently of the extractor for every permit signal (civic address, permis, dérogation, démolition, PIIA, lotissement, construction, usage, demand-id) and classifies each:

• matched — the line is inside an extracted record's source span.
• reviewed-not-permit — a by-law / procedural / venue line, or a signal word with no civic locus (a real permit is always locatable; discussion prose is not). E.g. "Règlement de démolition numéro 334" is a by-law, not a permit.
• UNMATCHED — a locatable permit signal the extractor missed. unmatched > 0 is a gap = a bug and is reported verbatim.

lossless = (unmatched === 0). Every record is also re-validated against the superset Zod schema before it is emitted.

BUILD 2 — the feed classifier (classify-feed.ts)

Per municipality, from its sampled docs:

refresh_rate (monthly | per-council-session | irregular | unknown)   ← from doc dates
format_consistency (structured-table | semi-structured | prose | scanned-image)
doc_types_published[]
permit_types_present[]     ← THE most important field: taxonomy ACTUALLY found
joinability (address-level | aggregate-only | mixed | none)
est_volume_per_period, parse_confidence, docs_sampled, notes

parse_confidence is discounted when any sampled doc failed the completeness check, so a lossy parse can never masquerade as a trustworthy feed.

BUILD 3 — the end-to-end pipeline (HARVEST → EXTRACT → JOIN → MEASURE)

pipeline.ts runs the whole chain per municipality, driven by city-configs.ts:

1. HARVEST (harvest.ts) — a generalized municipal-PDF crawler. From a city's seed archive URL(s) it discovers every permit-relevant PDF (an include/ exclude regex on the link), downloads the verbatim bytes (raw-native- first), and writes a manifest.json of {url, doc_type, doc_date, sha256, bytes, local_path}. Idempotent + resumable (keyed by URL; a re-run skips held docs; kill it mid-run and re-run), content-deduped (identical bytes under a new URL collapse to one file, both URLs recorded), and polite (rate_limit_ms between fetches). Both proven cities list their whole multi- year archive on one index page, so no pagination is needed; follow_index handles deeper archives when they paginate.
2. EXTRACT — the existing lossless parser (parse-one.ts) over the whole harvested corpus; the independent completeness scan gates each doc.
3. JOIN (join.ts) — resolve each addressed record to canonical.property_id, READ-ONLY, reusing the spine's OWN keys: Tier 1 deterministic address_key = canonical.address_key(addr) AND mamh_code = canonical.mamh_of(muni) (the exact QC join the live permit sources use, DR-190), Tier 2 pg_trgm similarity ≥ 0.82 scoped to the city. Batched: a whole city is 2 queries, not one-per-record.
4. MEASURE — per city: PDFs harvested, records extracted, completeness gaps, join-rate to property_id, permit-type mix. Writes report.json + property-permit-load.jsonl (the load-ready canonical.property_permit shape).

# full pipeline, all configured cities (harvest full archive → extract → join → measure)
pnpm --filter @ortova/permit-doc-parser pipeline
#   one city:            pnpm --filter @ortova/permit-doc-parser pipeline -- "Mont-Tremblant"
#   bounded proof run:   … pipeline -- "Saint-Georges" --max 8
#   re-measure offline:  … pipeline -- --offline           (reuse the on-disk manifest)

# harvest only (populate/refresh the manifest, download nothing twice)
pnpm --filter @ortova/permit-doc-parser harvest -- "Mont-Tremblant"

Per-city output lands under out/<city>/: raw/ (verbatim PDFs), manifest.json, extractions/<slug>.json, join.json (per-record verdicts + summary), property-permit-load.jsonl (load-ready, one canonical.property_permit row per record), report.json. A top-level out/pipeline-summary.json rolls up all cities.

What the pipeline proves end-to-end (2026-07-02, FULL archives)

570 extracted records attach deterministically to a real canonical.property_id (Tier-1 address_key, confidence 1.0) — the proof that PDF-only towns land on the spine. pnpm --filter @ortova/permit-doc-parser pipeline reproduces it.

Honest rough edges the scale run surfaced (the fixture was clean; a 5-year archive is not):

• Completeness is NOT uniform at scale. 126/221 MT docs are lossless; the other 95 produce 423 gap lines (402 addresses, the rest construction/permis/ lotissement). The check is doing its job — flagging that the demand-id anchor model misses addresses that live in resolution bodies / prose of older & agglo PVs (e.g. "…au 1303-1305, rue Labelle…"). Closing these is the next parser pass; the gap report names every missed line.
• Address extraction truncates on line-wrapped streets — "1001 rue de" instead of "1001 rue de Saint-Jovite" when pdftotext wraps the street across a line. ~168 of the 404 distinct MT misses are this shape.
• Genuinely unjoinable-by-address: 159 street-only refs (a chemin with no civic number) + 76 civic ranges ("2971-3067 chemin de la Chapelle" — the spine stores single civics). These need the cadastre/lot→parcel join (pilot-only), not an address fix.
• Lot "derrière le" case: an avis whose subject is a lot behind a civic address joins to the reference address, not the true parcel (semantic, not a bug).

Two province-general normalizations were added at the join boundary (join.ts normalizeForKey, exactly what a source adapter's normalize.ts does): strip trailing procedural clauses ("…en vertu du Règlement") and fold a trailing cardinal to the roll's single letter ("Avenue Nord" → "N", stored "AVENUE N"). Those lifted Saint-Georges from 60% → 100% of addressed on the bounded run.

Adding a city (the config format — this is how it scales)

A new town is ~15 lines in city-configs.ts, no code:

{
  municipality: 'Sainte-Adèle',
  municipality_code: '78030',   // MAMH code — the JOIN key (canonical.mamh_of arg). VERIFY it:
                                //   SELECT count(*) FROM canonical.property WHERE mamh_code='78030';
  feed_url: 'https://…/proces-verbaux',   // provenance
  rate_limit_ms: 800,
  seeds: [{
    doc_type_hint: 'proces-verbal',
    index_urls: ['https://…/proces-verbaux'],      // the archive listing page(s)
    include: /\/pv[^"]*\.pdf$/i,                    // which links are permit PDFs
    exclude: /reglement|budget/i,                   // optional — parser is the real gate
  }],
}

If the new town uses a French phrasing the taxonomy misses, that is a taxonomy.ts extension (province-general) — never a per-city hack. The join key is always the MAMH code; verify it returns the town's properties before trusting its join rate.

Run it (parser-only, offline)

# whole committed corpus, offline (uses the fixtures/*.pdf)
pnpm --filter @ortova/permit-doc-parser corpus
#   add --online to also fetch the remote-only siblings (Saguenay, La Pocatière)

# one document (URL or local path)
pnpm --filter @ortova/permit-doc-parser parse -- <url|path> --municipality "Mont-Tremblant"

# fixture-driven tests (offline, via poppler)
pnpm --filter @ortova/permit-doc-parser test
pnpm --filter @ortova/permit-doc-parser typecheck

Outputs land under out/: raw/ (verbatim PDFs), text/ (verbatim text), extractions/<slug>.json (per-doc DocExtraction), corpus-summary.json.

System deps: poppler (pdftotext, pdftoppm) + tesseract with the fra model. The Dockerfile installs them. Locally on macOS: brew install poppler tesseract tesseract-lang.

What the corpus run proves today (2026-07-02)

Four committed fixtures, real docs pulled from the live sites:

4/4 docs lossless, 18 records. Feed classification:

• Mont-Tremblant → address-level, structured-table, permit types [derogation-mineure, piia], ~15 items/session, parse_confidence 1.0. This is the strongest "closed → tappable" result: a rich, joinable permit feed hiding in the minutes.
• Saint-Georges → address-level, avis publics carry one dérogation each, address in the notice (often in the filename too). A clean, low-volume feed.
• Richmond → address-level but this session was almost all by-law work; the parser correctly extracts nothing spurious and proves it lost nothing.
• (--online) Saguenay / La Pocatière parse cleanly to 0 records on the sampled docs — the sampled sessions had no address-level urbanism items; worth sampling more dates before a verdict (feedback_never_claim_exhausted).

What works vs what's still rough

Works (proven by the run + tests):

• Full-document, every-page text extraction with per-page OCR fallback; no cap.
• Raw-native-first persistence (verbatim PDF + text) before extraction.
• The demand-id-anchored PV extractor (Mont-Tremblant is essentially perfect).
• The avis-public single-subject reader (venue address correctly rejected).
• The independent completeness proof; 4/4 fixtures lossless.
• The superset schema + raw_fields catch-all; Zod-validated on emit.

Still rough / not yet done:

• OCR is untested on a truly scanned doc — all four fixtures have text layers. The code path exists and is guarded, and the local tesseract has no fra model (would run eng with a logged confidence penalty; the Dockerfile adds fra). A scanned fixture is the next test to add.
• permit-stats tier is stubbed — classified but no aggregate-number extractor yet (Trois-Rivières / Gatineau bulletins). The seam is there.
• applicant and declared_value rarely appear in these docs and are lightly covered; extend taxonomy.ts as more phrasings surface.
• Coverage is a line-span heuristic, not a semantic guarantee — a permit whose address failed to parse would surface as an UNMATCHED gap (correct), but a malformed record could over-cover. The completeness check is the backstop, not a proof of per-field correctness.
• Not wired into the engine (deliberate — see below). No DB writes, no migration. Felix-gated.

The LLM seam (llm-seam.ts)

Extraction is behind one Extractor contract. ruleBasedExtractor is the default and is what the corpus + pipeline runs prove. buildExtractionPrompt() is the exact prompt an LLM extractor would receive — the same superset schema, so the same completeness check + Zod validation adjudicate an LLM's output identically.

Checked 2026-07-02: no LLM key is present (the only key in .env.local is SUPABASE_POSTGRES_POOLER_URL; no ANTHROPIC_API_KEY / OPENAI_API_KEY). So the rule engine stays the default and llmExtractor throws until a provider is wired.

The exact wiring (drop-in, when a key lands — the seam, not a rewrite): add @anthropic-ai/sdk to package.json, put ANTHROPIC_API_KEY in the worker env, and replace llmExtractor.extract in llm-seam.ts with:

import Anthropic from '@anthropic-ai/sdk';
const anthropic = new Anthropic(); // reads ANTHROPIC_API_KEY

export const llmExtractor: Extractor = {
  name: 'llm-claude-v1',
  async extract(pages, meta) {
    const msg = await anthropic.messages.create({
      model: 'claude-opus-4-8',       // or claude-haiku for the cheap first pass
      max_tokens: 8192,
      messages: [{ role: 'user', content: buildExtractionPrompt(pages, meta) }],
    });
    const json = (msg.content[0] as { text: string }).text;
    const raw = JSON.parse(json.slice(json.indexOf('['), json.lastIndexOf(']') + 1));
    // adjudicate identically: parse each through PermitRecordSchema; derive
    // covered_lines from each record's raw_excerpt against the page text, then
    // checkCompleteness() gates it exactly as it gates the rule engine.
    return coverFromExcerpts(raw, pages, meta);
  },
};

pickExtractor('llm') then selects it (pipeline run … --extractor llm). The one new helper, coverFromExcerpts, maps each returned raw_excerpt back to its source line indices so the independent completeness scan can still prove losslessness — that is the whole point of the seam: the adjudicator does not change.

Path to engine wiring (the next tile, Felix-gated)

Follow docs/architecture/source-onboarding-kit.md: one pdf-kind source per municipality (or one parameterised source keyed by municipality) whose adapter yields these records as native raw rows into raw.<muni>_permit_docs; the canonical-upsert then resolves each address / lot_number to property_id and lands canonical.property_permit — the identical shape every existing permit city already writes. The parser output here is deliberately that shape so the adapter is thin. The bulk PDF fetch runs cloud-side in this worker; the migration apply + canonical join stay Felix-gated.

Provenance

The four fixtures were fetched 2026-07-02 from the live municipal sites; each carries its live URL in corpus.ts (feedback_source_attribution_always). Documents are public municipal records. When a feed is onboarded, its meta.dataset_catalog.source records authority + license + the per-doc URL.

apps/worker/permit-ingester/README.md

permit-ingester — cloud bulk-ingest runtime for the permit canonical join

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: cloud-side bulk canonical-upsert of a permit feed (config-driven, resumable).
• Input boundary (Layer-1): WorkflowArgsSchemas['permit-ingest'] = { source_key, phase? } in scripts/ops/watch/executor/schemas.ts; wrapped by WatchJobSchema when Watch-driven.
• Output: canonical permit upsert (platform config-driven). DB-owned shape (§C).
• Run locally: pnpm --filter @ortova/permit-ingester plan / ingest.
• Deploy: gh workflow run permit-ingest.yml -f source=<key> (runs-on ortova-mac).

A run-to-completion worker that drives the permit canonical-upsert (and an optional resumable raw sweep) from a box sitting next to the database, instead of from a laptop over home wifi.

Why it exists (worksite tile 213, Zone 5 SCALE)

The canonical-upsert dispatcher is sound — keyset paging + a single LEFT JOIN identity resolve, flat ~9 MB heap at scale (tile 205). The cost of finishing a big-city join is network: Montréal's 554k permit rows are ~hundreds of thousands of round-trips to the Supabase pooler, which over home wifi is hours of pure latency tax — and the pooler's forced 2-minute statement_timeout turns any straggler query into a hard failure. The durable home is a worker near the DB. Same posture as the geo-ingester / tile-builder siblings.

It is also the runtime the national permit feeds (Calgary, Toronto, Edmonton…) ingest through: every permit city lands the same canonical shape on the same property subject, so one source-parameterised worker serves them all.

What it runs

--plan is read-only: prints the raw/canonical counts and what would run.

It does not re-stamp rows into synthetic chunk run_ids. Tile 205's keyset paging made each batch an O(limit) seek regardless of a run_id's size, so the old "split one big run_id into 5k-row chunks" dance is obsolete — and that dance also tripped raw.<source>.run_id's FK to adapter_run_log and an accidental string-vs-bigint chunk loop in the laptop script. Running over the existing run_ids sidesteps both.

Run it

# read-only preview
SOURCE_KEY=montreal-ckan-permits tsx ingest.ts --plan

# finish the Montréal canonical join (Felix-gated write)
SOURCE_KEY=montreal-ckan-permits tsx ingest.ts --i-am-felix

# a fresh national city: raw sweep then canonical
SOURCE_KEY=calgary-socrata-permits tsx ingest.ts --i-am-felix --phase all

Env: SOURCE_KEY (one of the registered permit sources in ingest.ts) and SUPABASE_POSTGRES_POOLER_URL. Writes require --i-am-felix (Rule 6).

Deploy (Felix-gated)

This worker runs the TypeScript engine, so unlike the self-contained siblings the Docker build context is the repo root, not this dir.

Build model (decided Felix 2026-07-08): git-connected, not laptop-uploaded. Data-ingestion workers under apps/worker/** build their Railway image directly from GitHub main — the deployed image is always merged main, never an arbitrary laptop checkout. The prior model (manual railway up uploads) is exactly what let the typed-watermark fix (commit cabdcca) sit merged on main while the running service kept executing pre-fix code (source: null on the service — no gate between "merged" and "running"). See scripts/ops/release-worker.sh for the run-trigger vehicle.

Root Directory is . (repo root) for THIS service — not the subdir convention the self-contained siblings use. This worker's Dockerfile COPYs packages/, apps/, data/, and scripts/ from repo-root-relative paths (it needs the whole pnpm workspace to run the TS engine), so Root Directory MUST stay the repo root or those COPYs resolve to nothing and the build fails. dockerfilePath stays repo-root-relative (apps/worker/permit-ingester/Dockerfile) to match. Path-filtering the rebuild trigger to only apps/worker/** (like the self-contained siblings) would be WRONG here and would silently reintroduce a narrower version of the exact staleness bug this build model exists to kill — permit-canonical-upsert.config.ts lives under apps/core/, not apps/worker/permit-ingester/, and the Dockerfile copies packages/ and scripts/ too. railway.json's build.watchPatterns is therefore scoped to everything this Dockerfile actually depends on (apps/worker/permit-ingester/**, apps/core/**, packages/**, scripts/**, data/**, the root workspace manifests) — narrower would be unsafe, wider (no filter at all) would also be a legitimate, safer default if this list ever drifts from the Dockerfile's real COPY surface.

1. One-time setup (Release Agent, per service): railway service source connect --repo <owner>/<repo> --branch main --service permit-ingester. Set Root Directory = . (repo root) and Dockerfile path = apps/worker/permit-ingester/Dockerfile (mirrors railway.json, which also carries the watchPatterns rebuild scope — no separate dashboard step needed for that part).
2. Pick a region close to the Supabase database — that proximity is the entire point.
3. Service variables: SOURCE_KEY, SUPABASE_POSTGRES_POOLER_URL. Optionally a start-command override for --phase/--plan, and CANONICAL_UPSERT_BATCH_SIZE to tune the canonical-phase chunk size. Set/update these via scripts/ops/release-worker.sh --i-am-felix permit-ingester SOURCE_KEY=... (never hand-set through the dashboard — the script logs evidence).
4. restartPolicyType: NEVER (it's a one-shot job; the canonical phase is idempotent so a manual re-run is always safe).
5. Triggering a run (every subsequent ingest, after step 1's one-time connect): bash scripts/ops/release-worker.sh --i-am-felix permit-ingester — pulls the latest commit from main and reruns. Raw railway up / railway redeploy are not an allowlisted agent path; release-worker.sh is the only sanctioned trigger (mirrors release-push.sh / release-deploy.sh).

The connect step, the variable sets, and every run trigger are Felix-gated (outward + spends money) — the Release Agent runs them on Felix's approved set.

Fast cohort load (tile 314) — the canonical-upsert throughput fix

The permit canonical join floored at ~140 rows/s (SF's 1.29M ≈ 2.5 h; NYC's 5M ≈ 10 h). Measured on prod (EXPLAIN (ANALYZE, BUFFERS), live SF load): the identity resolve — LEFT JOIN canonical.property ON (address_key, mamh_code) to fetch property_id — is ~85% of per-batch wall time (2,238 ms of ~2,600 ms / 500 rows), and 100% of that is the random heap fetch for property_id (not in the key index): 1,759 ms / 500 probes with 327 heap disk reads vs 2.9 ms / 0 when the identical probes stay index-only. The 253 MB key index fits t4g.small cache; the 1,334 MB heap does not, so every property_id fetch is a cold random read.

Two complementary levers make the cohort load fast (both preserve byte-identical join results — proven: 30,000 / 30,000 SF permits recompute to the exact persisted property_id):

1. Covering index migrations/0186_property_addrkey_mamh_covering_index.sql — (address_key, mamh_code) INCLUDE (property_id) → the resolve plans as an Index Only Scan; the 290 MB covering index stays cache-resident and the heap fetch disappears. Projected resolve 2,238 ms → ~155 ms / 500 (14×), lifting the server-side resolve ceiling from ~223 to ~3,000+ rows/s for the whole permit cohort (QC + US). Felix-gated apply (CREATE INDEX CONCURRENTLY, direct/ session connection — see the migration header for mechanics).
2. CANONICAL_UPSERT_BATCH_SIZE (default 500) — set to e.g. 2500 for the cloud-side cohort load. Larger WAL-safe chunks fold the same work into ~5× fewer round-trips + set-based statements. INV-4-safe: still chunked per-batch transactions (clamped ≤ 20 000 so it can never become a WAL-PANIC province-size INSERT), keyset-paged, idempotent, resumable. Result-invariant (batching is pure pagination over a deterministic join).

Order of operations for the redeploy: apply 0186 first (so the resolve is index-only), then bash scripts/ops/release-worker.sh --i-am-felix permit-ingester CANONICAL_UPSERT_BATCH_SIZE=2500 and re-run the canonical phase per source (idempotent — already-loaded rows hash-match and skip).

apps/worker/rf-stagehand/README.md

rf-stagehand — Registre foncier lookup worker

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: a Fastify + Stagehand HTTP worker that looks up Registre foncier ownership.
• Input/output boundary (Layer-1): HTTP POST /registre-foncier/lookup — LookupRequest, LookupResult, ExtractedOwnershipSchema (Zod already, src/lookup.ts / src/server.ts; boundary table §C).
• Run locally: pnpm --filter @ortova/worker-rf-stagehand dev (Fastify server).
• Deploy: Docker container.

Railway-hosted HTTP sidecar that scrapes registrefoncier.gouv.qc.ca via Stagehand + Browserbase. Fills the canonical.property_ownership table that drives the ownership_change signal.

API

POST /registre-foncier/lookup

{
  "cadastralLot": "1 234 567",
  "lookupId": "abc123"
}

Headers:

• X-API-Key: <RF_WORKER_API_KEY> (required if the worker has the env var set)

Response 200:

{
  "lookupId": "abc123",
  "cadastralLot": "1 234 567",
  "found": true,
  "ownerName": "Jean Tremblay",
  "acquisitionDate": "2023-09-14",
  "registrationNumber": "25 379 544",
  "subsequentCreditors": ["Desjardins Caisse Centrale"],
  "declarationResidenceFamiliale": false,
  "costCad": 0.20,
  "sessionId": "sess_abc",
  "durationMs": 12350
}

GET /health → { ok: true, service: "rf-stagehand" }

Environment

Deployment

# First time:
railway login
railway init    # create a new Railway project (Hobby tier, $5/mo)
railway link    # if re-linking

# Set secrets:
railway variables set \
  BROWSERBASE_API_KEY="bb_..." \
  BROWSERBASE_PROJECT_ID="prj_..." \
  ANTHROPIC_API_KEY="sk-ant-..." \
  RF_WORKER_API_KEY="$(openssl rand -base64 32)" \
  LOG_LEVEL="info"

# Deploy:
railway up

# Get the public URL:
railway domain

After deploy, set LAMBDA_BROWSER_URL + LAMBDA_BROWSER_API_KEY on the overwatch-dev Vercel project. The qc-registre-foncier-ownership source adapter reads both to fire lookups.

Cost model

Each session burns $0.05 Claude (Stagehand planner, ~3 turns) + ~$0.15 Browserbase (30s session) = **$0.20 / session**. Platform-side preFlightCostCap() enforces a $200 CAD/month cap against raw.rf_cost_log before each session fires. Sessions that succeed report their actual cost back and the ledger updates.

Stagehand + Browserbase caching

The Stagehand selector cache means Day 1 of a new lookup burns Claude tokens for page interpretation; Days 2-30 execute cached selectors at $0 LLM cost. Browserbase compute still accrues per session. Expect total cost to drop ~80% after the first week of production runs.

apps/worker/tile-builder/README.md

tile-builder

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

• What it is: bakes the map substrate — PMTiles from canonical.* layers.
• Input boundary (Layer-1): CLI --layer / --sql-filter; reads canonical.*. No Zod schema (FLAGGED, boundary table §C): batch CLI, env/flag input.
• Output: PMTiles → R2. R2-owned artifact (§C).
• Run locally: pnpm --filter @ortova/tile-builder plan / build.
• Deploy: gh workflow run tile-builder.yml (runs-on ortova-mac).

Reusable vector-tile build job: turns canonical.* layers into PMTiles and uploads them to Vercel Blob. Built to run on a cloud box with a fat pipe to the DB — not a laptop. Province-scale geometry (cadastre ~3.7 GB, property centroids ~2.78M points) is stream-bound; from a home uplink the parcels layer alone would take hours, while cloud-to-cloud in ca-central-1 (where Supabase lives) it runs to completion in tens of minutes.

Sibling to apps/worker/geo-ingester/ — same shape (run-to-completion, declarative manifest, Felix-gated writes, --plan dry-run, optional --local-only).

What it does (per enabled layer in layers.ts)

1. Streams features from canonical.* as GeoJSONSeq via psql -At COPY (...) TO STDOUT (one JSON Feature per line).
2. Pipes the stream into tippecanoe with the per-layer flags (zoom range, simplification, clustering, drop/coalesce policy), writing <layer>.mbtiles.
3. Converts to PMTiles with pmtiles convert.
4. Uploads to Vercel Blob as tiles/<layer>.pmtiles (public, no random suffix, allowOverwrite), capturing the public URL.
5. Rewrites the TILE_URLS block in apps/dev/src/lib/tile-urls.ts so the /home MapLibre canvas reads stable, committed URLs.

Read-only on canonical.* (Rule 6). The only state-changing side effects are the Blob upload and the manifest rewrite, both gated by --i-am-felix.

Per-layer config (verbatim from the brief)

footprints is declared but disabled: true until the Overture province ingest (docs/briefs/2026-05-26-qc-object-graph-scaleout.md Wave 2) lands.

Run it

Deploy to Railway (matches the other apps/worker/* sidecars) in a region close to Supabase, or run as an AWS Fargate/EC2 task in ca-central-1 (optimal — same region as the DB; this is what the geo-ingester uses for cadastre/footprint ingests of similar size). Set:

SUPABASE_POSTGRES_POOLER_URL = <session-pooler url>
BLOB_READ_WRITE_TOKEN        = <Vercel Blob token from the linked project>
WORK_DIR                     = /tmp/tile-builder      # optional, default

Then it runs once on start (CMD = tsx build.ts --i-am-felix).

Local sanity checks (no cloud needed)

# Dry run — counts + the exact tippecanoe command per layer, no streaming.
SUPABASE_POSTGRES_POOLER_URL=... tsx build.ts --plan

# Build one layer locally (territories is fastest — ~1.4k features).
# Leaves WORK_DIR/<layer>.pmtiles on disk for inspection. Skips Blob.
SUPABASE_POSTGRES_POOLER_URL=... tsx build.ts --layer territories --local-only

--local-only and --plan do NOT touch the manifest or Blob.

Generated artifacts

• WORK_DIR/<layer>.mbtiles — tippecanoe output (intermediate).
• WORK_DIR/<layer>.pmtiles — final tile bundle.
• tiles/<layer>.pmtiles — Blob object key (overwritten on each run).
• apps/dev/src/lib/tile-urls.ts — manifest, rewritten in place per run.

Felix-gating posture

Same as geo-ingester:

• --plan — read-only.
• --local-only — read-only on the DB; only side effect is files in WORK_DIR.
• --i-am-felix — required for Blob upload + manifest rewrite.

The DB connection has statement_timeout=0 idle_in_transaction_session_timeout=0 applied via PGOPTIONS for the long-running COPY streams (parcels and property take minutes each).

Adding a layer

Add an entry to LAYERS in layers.ts — no code change. To pin tile parameters to the brief, paste the tippecanoe flags verbatim from the per-layer table above. The runner is layer-agnostic.

packages/brand/README.md

@ortova/brand

The frozen, versioned home for Ortova's design assets — the repo side of the contract in docs/design/design-asset-management.md. Figma (Ortova — Brand & Design System, fileKey oNDdUL9rGunX7Yuh9VRSi7) is the design source of truth; this package holds the exported, approved files apps actually consume. The app never reads Figma live, and never keeps a per-app inline copy — this is the one place.

Naming

kind/variant/context/size, lower-kebab, mirrored 1:1 as the folder path (contract Layer 2). Today:

logo/lockup/horizontal/svg/color-on-indigo.svg   — full color lockup ON the indigo ground rect
logo/lockup/horizontal/svg/color-transparent.svg — the SAME lockup, indigo ground stripped (derived)
logo/lockup/horizontal/svg/mono-white.svg        — dark chrome
logo/lockup/horizontal/svg/mono-indigo.svg       — light marketing chrome
logo/lockup/horizontal/svg/mono-ink.svg          — light operator chrome
logo/lockup/horizontal/png/<variant>@1x.png      — raster proofs (@1x/@2x), only where SVG can't go
logo/lockup/horizontal/png/<variant>@2x.png

color-transparent is a derived asset, not a separate Figma node: brand:sync computes it by stripping the single flat-#000F6A ground <path> from color-on-indigo (every glyph + wordmark path byte-identical), so it always tracks the pinned grey (#D9D9D9) wordmark and never carries the ~2ppm control-point jitter a Figma clone() would. Grounds belong to the surface, not the asset (Felix, 2026-07-20) — a page supplies its own background and drops in color-transparent, no rectangular seam. The mono trio shares one aspect ratio (280 × 115.5); the two color lockups share 500 × 206.25. Set height, width follows.

Keeping this fresh

Figma (oNDdUL9rGunX7Yuh9VRSi7) is the source; this directory is a frozen mirror, never a place to hand edit. Run pnpm brand:sync to pull the current state of every asset in sync-manifest.json (the node id -> path wiring), strip Figma's export artifacts, and rewrite the SVG + PNG proofs here in place. Needs FIGMA_TOKEN (a personal access token, figma.com -> Settings -> Personal access tokens) in .env.local locally or as a GitHub Actions secret in CI; pnpm brand:sync --dry-run validates the manifest without one.

Provenance (2026-07-14, tile 411)

Exported from Figma nodes 3:2 / 73:4 / 73:13 / 73:22 (frame names match the paths above verbatim). The Figma MCP/desktop-app export path bled the Foundations page backdrop + a per-variant "stage" demo card (a background swatch Figma renders so a mono-white mark is visible against something on the canvas) into the raw SVG/PNG — confirmed by diffing the frame's own child list (get_metadata) against the exported markup. Both were stripped before landing here: what ships is exactly the frame's own vector children (the mark + wordmark paths), nothing else, on a transparent ground for every variant except color-on-indigo (there the indigo field IS one of the frame's own vectors, intentional, kept). The PNG proofs are rasterized locally from these cleaned SVGs rather than from Figma's PNG export, for the same reason: Figma's raster export carries the identical baked-in stage background. brand:sync (below) instead calls the versioned REST /v1/images endpoint, confirmed the same day to already export cleanly (no backdrop, no stage card) for these four nodes — stripFigmaBackdrop() still handles the MCP-shaped bug as a fallback in case a future export reintroduces it, but the common path today needs no stripping beyond tightening the viewBox to the frame's exact (non-rounded) size.

Updated 2026-07-20 (brand rework — canonical logo corrected + single-source components): Felix pinned the final logo set. The canonical color-on-indigo is now the light-grey wordmark (#D9D9D9) on indigo, NOT the earlier white-wordmark treatment — the white lockup is retired to the Figma ⌫ Archive page (never deleted; ids may be referenced). In the same pass every lockup master in the Figma file was converted to a Figma component (Pin 3: each asset exists exactly once as a component; usages are instances), which minted new node ids. The manifest above was repointed accordingly: color-on-indigo 3:2 → 159:3, mono-white 73:4 → 159:5, mono-indigo 73:13 → 159:6, mono-ink 73:22 → 159:7. Verified by brand:sync: the three mono SVGs re-exported byte-identical (same sha256 as 2026-07-14, proving componentization preserved the vectors), while color-on-indigo changed (white → grey). Canonical colors confirmed from the pinned vectors: indigo #000F6A, spire teal #249AB8 → #5FB6C9 → gold #E6C173, mono-ink #17140F.

Updated 2026-07-21 (mirror collapse + color-transparent folded in): brand:sync is now the single fan-out (the competing apps/dev/scripts/copy-brand-assets.mts build-time copy is retired; its dev mirror is now a committed, validator-enforced fan-out). The landing's separately-pinned color-on-page.svg (a hand-derived color-on-indigo-minus-ground, still carrying the old white wordmark) is retired: it is replaced by the derived color-transparent asset, computed from the current canonical color-on-indigo — so the landing nav wordmark now reads grey #D9D9D9, the consistent consequence of the grey-canonical pin (the last white-wordmark instance leaves live surfaces). Two HQ calls carried into this pass:

• color-transparent ships a TRANSPARENT ground — grounds belong to the surface, not the asset (Felix's landing precedent, 2026-07-20: a baked ground seams on real surfaces).
• color-on-light (indigo wordmark + color spire, Figma node 159:4) is decided to ship transparent-ground for the same reason, but is still [unverified vs Marc's .key] and not frozen here — the Ortova - Logo & colors.key Keynote could not be read by the tooling. Marc outstanding: (1) mint a real color-transparent Figma component + ping back its node id (today it is byte-derived from color-on-indigo, not its own node); (2) confirm color-on-light ships transparent (ground supplied by the surface), and flag if the .key differs. See docs/design/brand-figma-file-structure.md.

Re-verified 2026-07-14: re-pulled all four nodes twice (once via Figma MCP, once via brand:sync against the live REST API) to check a report that Figma held a spacing fix not yet in this package. Both pulls are geometrically identical to what was already landed here (every <path d="..."> byte for byte the same; only Figma's auto-generated gradient-id suffixes differ, which is inert). The clearspace between mark and wordmark was already 1x mark width in the landed assets — Figma has not changed since. If a stale lockup is still visible somewhere, the cause is a cached preview/deploy target, not these files.

How apps consume this — the single-source fan-out (2026-07-20)

packages/brand is the one authored home. An app can only serve files that physically live under its own public/, so pnpm brand:sync fans each asset out into per-app mirror dirs, declared in sync-manifest.json under mirrors:

"mirrors": {
  "dev":     { "dir": "apps/dev/public/brand",     "formats": ["svg"] },   # OrtovaLockup wrapper
  "landing": { "dir": "apps/landing/public/brand",  "formats": ["svg"] }    # color-transparent nav mark
}

Each asset lists which mirrors receive it ("mirrors": ["dev", "landing"]). The mirrors are committed, byte-identical, and NOT authorable — real files (never a symlink: a symlink outside the app dir broke Vercel's static collector twice, 2026-07-14; committed real files sidestep that and need no build-time copy step). A hand-edit, a stale copy, or a stray file in any mirror goes red at the next gate via scripts/validate-arch/brand-mirror-parity.ts, which asserts every mirror file is byte-identical to its packages/brand source and the mirror holds nothing else. The one and only fix for a red is to run pnpm brand:sync — never to edit a mirror.

To change a brand asset: edit it in Figma → pnpm brand:sync → commit → push. That is the whole contract ("change it in one place, say update and push, that's it" — Felix). Apps reference assets by served path, e.g. /brand/logo/lockup/horizontal/svg/color-transparent.svg; apps/dev/src/components/panels/wordmark.tsx (OrtovaLockup) is the one dev wrapper, and the landing's nav <img> is the one landing consumer — nothing else should inline these paths.

packages/platform/README.md

@ortova/platform

Deployable unit (colocated-docs Layer 2 — see root ARCHITECTURE.md).

The engine library apps/core and apps/dev build on: the six primitives (subjects, adapters/sources, signals, bundles, products, actions), the brain, orchestration, observability, cost-tracking, and storage. Not a running service — it is consumed in-process.

• What it is: the shared platform SDK; home of the factory contracts that throw at module load (define*) and the schema authority for ingress payloads.
• Input/output boundaries (Layer-1):
  • External-API payloads → per-adapter-kind Zod in src/adapters/kinds/* (arcgis, socrata, ckan, opendatasoft, carto, stac, browser) + per-source schema.ts under apps/core/src/sources/*. See boundary table §D.
  • Product egress → products/*/shape.ts (e.g. InsuranceScoreSchema). See §A v1/score resp.
  • Brain / signal / event types → src/brain, src/signals, src/orchestration.
• Run locally: consumed via workspace import (import { ... } from '@ortova/platform'); pnpm --filter @ortova/platform typecheck. Entry src/index.ts, subpath exports in package.json.
• Deploy: none — published only as a workspace dependency; ships inside apps/dev / worker builds.

packages/twist/README.md

@ortova/twist

Twist client for Overwatch. Wraps @doist/twist-sdk with a factory (createTwistClient), OAuth helpers (buildAuthorizationUrl, exchangeAuthorizationCode), and inbound-webhook primitives (verifyWebhook, parseInboundFormBody).

Vendored from the studio's @nexod/twist (the same package BoI consumes), following the @ortova/platform precedent: engine code is pulled into packages/ and renamed to the @ortova/* namespace so this repo stays self-contained. Keep behavioral changes in sync with the studio package, or note the divergence here.

This package owns client construction, env-var resolution, error normalization, and a thin set of high-level helpers. The consuming app (apps/core, apps/dev) owns which channels to post to, what content to format, OAuth credential storage, and retry policy — that wiring is the Overwatch integration spec, tracked on worksite tile #18.

Install

In-workspace; resolves via pnpm. Add to a consuming package:

// apps/<app>/package.json
"dependencies": { "@ortova/twist": "workspace:*" }

Token-based usage (server-side)

TWIST_BOT_TOKEN=<long-lived OAuth token>     # falls back to TWIST_ACCESS_TOKEN
TWIST_WORKSPACE_ID=<workspace id, integer>

import { createTwistClient } from "@ortova/twist";

const twist = createTwistClient();

const { thread_id } = await twist.postToChannel({
  channelId: Number(process.env.TWIST_CHANNEL_RUNS),
  title: "NDVI run #402 dispatched",
  content: "Verdun baseline scan, 1,284 parcels queued.",
});

await twist.replyToThread({ threadId: thread_id, content: "Gate decision: APPROVE." });

For raw API calls beyond the helpers, use twist.api.* directly.

OAuth flow

import { buildAuthorizationUrl, exchangeAuthorizationCode } from "@ortova/twist";

const authUrl = buildAuthorizationUrl({
  clientId: process.env.TWIST_CLIENT_ID!,
  scopes: ["user:read", "channels:read", "threads:write", "comments:write"],
  state: cryptoRandomState,
  redirectUri: "https://app.example.com/twist/callback",
});

// In the callback, after verifying state:
const { accessToken } = await exchangeAuthorizationCode({
  clientId: process.env.TWIST_CLIENT_ID!,
  clientSecret: process.env.TWIST_CLIENT_SECRET!,
  code: callbackCode,
  redirectUri: "https://app.example.com/twist/callback",
});

Inbound webhooks

Twist signs inbound POSTs only with a verify_token field (no HMAC). Parse the form body, verify the token, and combine with a user_id allowlist:

import { parseInboundFormBody, verifyWebhook } from "@ortova/twist";

const payload = parseInboundFormBody(await req.text());   // TWIST_WEBHOOK_SECRET
const result = verifyWebhook(payload);
if (!result.valid) return new Response(result.reason, { status: 401 });

Configuration responsibility

• Package owns: SDK construction, env-var reading at instantiation, error normalization, OAuth URL/token primitives, webhook parse/verify, the postToChannel / replyToThread / postCard / updateCard helpers.
• App owns: env var population, OAuth credential storage, channel selection per event, content formatting, thread-id persistence, retry policy.

Direct TwistApi instantiation outside this package is a bypass — justify it in a comment citing the gap, per the engine-bypass discipline in CLAUDE.md.