development-plan.adoc16.6 KBhistory comment on this file

Development Plan

Build order, complexity, and agent assignment for the thirteen crates in docs/spec/overview.adoc. Phases follow the dependency graph; crates within a phase have no edges between them and build in parallel. The pre-redo tag is the salvage path throughout: correctness that was hard-won once (CAS discipline, anchor projection, docker readiness, sprite quirks, backend conformance, Postgres/Tigris store code) is ported and re-verified, not rediscovered.

Tools and abstractions before scale-out govern the ordering below: a phase earns its place by exercising a real abstraction (the gate, the query engine, receive, sync) against stock git, not by anticipating load git itself would have to buckle under. gix-receive is the clearest instance — a custom wire-framing crate structurally required only once Postgres-backed refs make git’s own receive-pack impossible to run directly, so it moves out of the foundation and into the scale-out phase alongside the Postgres RefStore, Tigris objects, and the durable queue it ships with. Until that phase, git’s own receive-pack is the one hosted and local serving transport; the single-node root at git.ents.cloud (Phase 6) is stock git wearing the same gate everything else runs, not a bespoke protocol.

Agent scheme

sonnet — bulk and medium-complexity implementation, ports from pre-redo, template/CLI surface breadth.
opus — high-complexity subsystems: wire protocols, merge algorithms, store backends with subtle atomicity.
fable — the two crates where a quiet bug is a security hole or a wrong design (ents-gate, ents-query core semantics), public-API review of every crate at its phase boundary, and integration.

Every crate lands with Tracey annotations mapping its spec rules (.config/tracey/config.styx already covers docs/spec/), tests run via cargo nextest run, and a fable review of the public API before anything in the next phase depends on it.

Phase 0: bootstrap

Before any crate lands, redeploy the existing Fly app (.config/fly.toml) as a headless machine serving two things behind one host — no gix-receive, no gate, no CI or Sprite executor:

stock git’s smart-HTTP backend (git http-backend) against a bare repo on the odb volume, so git.ents.cloud is a real origin from day one; and
tracey web, reading a plain working tree a post-receive hook checks out on every push (the same updateInstead trick Phase 6 formalizes for real), so the same host shows live spec coverage on HEAD.

Every phase number below is one higher than in any prior ordering of this document. Nothing here is salvaged or reusable code — the single-node hosted root in Phase 6 (git-ents, the CLI-complete milestone) replaces the git backend wholesale, at the same hostname; whether it keeps serving the coverage dashboard is a later call.

Crate schedule

Crate Phase Complexity Agent Notes

Crate	Phase	Complexity	Agent	Notes
`facet-git-tree`	1	Low	sonnet	Exists (external repo). Stays domain-blind. Land the `#[facet(transparent)]` fix upstream and drop the local `[patch]`.
`gix-ref-store`	1	Medium	sonnet	Read/CAS trait split per `arch.refstore-read-cas-split`; loose-ref impl ports the CAS discipline from `pre-redo` (`arch.loose-cas-discipline`: no shelling to `git update-ref`). Fable reviews the trait before phase 3 consumes it.
`ents-model`	2	Low	sonnet	Entity structs, namespaces, trailers, taxonomy — declarative, but it is the vocabulary every crate imports, so spec fidelity and the public API review matter more than the code. Trailer parsing is new work, not a port: no `git-metadata` crate exists at `pre-redo`.
`ents-gate`	3	Medium	fable	Small in lines, security-critical in every line: tip signature, refname binding, DAG-FF, epoch, bootstrap fail-closed. Pure over the read half of `RefStore`. Exit test: identical verdicts at all three call sites.
`ents-query`	3	High	fable	The critical path: grammar/parser (easy), static footprint extraction, incremental set-entry via generation numbers, work set `trigger − results(self, any)` (hard). Property-test monotone entry-only semantics against synthetic ref histories.
`ents-anchor`	3	Medium	sonnet	Mostly a port: projection (blame + fuzzy context match) survives at `pre-redo`; re-home it on `ents-model` and the new retention rule (embed blob + context, no gitlinks).
`ents-receive`	4	Medium	sonnet	Orchestration above traits that already exist by now: gate policy (mandatory/advisory), footprint matching, enqueue, `(effect, oid)` dedup, redaction at ingest. Defines `EventSink`, reconstructible from repository state as the work set `trigger − results(self, any)` (`query.workset`); the in-memory impl plus a boot-time reconciliation scan is the reference implementation, durability deferred as a performance property (`receive.reconstructible`). Fable reviews the `receive()` signature — every frontend is downstream of it.
`ents-sync`	4	High	opus	Fetch/push plumbing is routine; the schema-aware three-way merge over typed trees (divergence + adoption on one machinery) is the hard part. Absorbs git-store’s merge.
`ents-effect`	5	Medium	sonnet	`Executor` trait + Docker/Sprite backends (feature-gated), toolchain materialization, run loop, write-back as a receive client. Ports docker readiness probes and sprite quirks from `pre-redo`; the design is settled, the environment is the risk.
`git-ents` (bin)	6	Medium	sonnet	Local root wiring (~50 lines) plus the real cost: subcommand surface and the `denyCurrentBranch=updateInstead` worktree edge. Headless — no `serve` subcommand yet (arrives Phase 7 with `ents-web`). Doubles as the single-node hosted root (`roots.single-node-hosted`): loose refs and a real odb on a Fly volume, served behind git’s own `receive-pack` — the same stock-git transport Phase 0 bootstraps, now invoking the gate from a hook — with an in-memory `EventSink` and a boot-time reconciliation scan, and the Sprite executor, deployed at `git.ents.cloud`. Milestone: CLI-complete. Two gaps ship unclosed, neither assigned a phase: no porcelain command or root bootstrap sets refs/meta/config’s verification epoch, so the hosted root’s Mandatory gate stays archival until one is designed (`gate.epoch-bootstrap); and meta-ref.tip-invariant’s generic escape-hatch display for an unrecognized meta-ref, and anchor resolution, have no command anywhere in this binary, despite `ents-model (phase 2) forward-referencing them to "the `git-ents` binary" by name.
`ents-web`	7	Medium	sonnet	Broad surface, low algorithmic depth. Lands `serve` in `git-ents` (`roots.local`): the local root’s own wiring, plus sessions/CSRF (`roots.web-session`) and signed mutations (`roots.web-signing`) layered on top, the user’s own key standing in for whichever identity a composition root injects. Signing identity is injected by the composition root; the crate assumes nothing about network reachability, so in-process webview embedding stays a supported deployment (`roots.web-agnostic`). The gap this phase shipped with — no composition root wiring `ents-web` onto a hosted root, the server-key indirection proven only against fixture identities — was closed after phase 10 by `git ents serve --hosted` (roots.single-node-hosted’s web-UI clause): the single-node hosted root mounts this same crate behind nginx with the mandatory gate, the server’s enrolled member key as signing identity, and member sign-in via the CLI-driven challenge flow (`roots.web-signin, `receive.attributed-author` — edits land as "member via the web").
`gix-receive`	8	High	opus	Scale-out only: structurally necessary once the Postgres `RefStore` leaves no on-disk repo for git’s own `receive-pack` to act on, not before. Smart-HTTP wire framing + pack ingestion, zero policy. Protocol-v2 and sideband subtleties; conformance-test against real `git` clients. Quarantine semantics per `receive.object-access`.
`git-ents-server` (bin)	8	High	opus	Postgres `RefStore` (multi-ref CAS in a transaction), Tigris object store behind gitoxide traits, a durable-queue `EventSink`, and `gix-receive` replacing git’s own `receive-pack` as transport — a new composition root swapped in with zero library-crate edits. This phase IS the honesty test (`roots.honesty-test`): entry is triggered by measured storage/load pressure (the kiln-cache growth analysis), not a date. Reuses the Sprite executor and sessions/CSRF from `git-ents`; salvage the store code and gix-receive framing from the scale-out work at `pre-redo`.
`ents-forge` conversations	9	Medium	fable	Comments become the universal conversational primitive (`model.comment` broadened, `model.comment-state`, `model.comment-context`, `model.comment-thread`, `model.review`): the Comment struct broadens as a clean break (experimental repo, no on-disk data to preserve, so no back-compat reader), working-tree capture and projection in `ents-anchor` (`anchor.working-tree`), the Review entity plus its retention pin ref (`model.review-pin` — `refs/meta/pins/reviews/<target>/<member>`, an empty-tree signed commit whose parents include the reviewed commit; verify the gate’s tip-signed and fast-forward checks accept the merge-shaped pin advance), and porcelain — `comment reply/resolve/reopen`, `comment list --worktree` with a machine-readable form (`lens.parity`), `issue` and `review` actions.
`ents-lens`	9	High	opus	The LSP frontend (`lens.adoc`, all rules): `git ents lsp` reusing the local root’s wiring the way `serve` does, code lenses + hint diagnostics + hover derived per-request from anchor projection onto the working tree, and the editor-file compose flow. No network, no state of its own.
`ents-zed` (extension)	9	Low	sonnet	Zed extension at `editors/zed`, outside the workspace (wasm target): registers the `ents-lsp` language server running `git ents lsp`. Verify current Zed LSP capability coverage against the real `zed_extension_api` — `lens.diagnostics` exists precisely so a client without code-lens rendering still shows the conversation.
`ents-web` conversations	9	Medium	opus	Issues index and detail with threads as context queries (`model.comment-context`), review creation and verdict display on commit pages (`model.review`), reply/resolve on the existing comment views. Reuses the registry-driven meta rail and session/CSRF machinery already in place.
kernel identity binding (`ents-model`, `ents-gate`, `ents-receive`, `ents-sync`, `ents-effect`, `ents-testutil`)	10	High	fable	One atomic migration — the kernel cannot build half-converted. Retire `trailer.rs` (`Advance-ref` and the never-used `Schema-Version`) in favor of `meta-ref.identity-binding`: the gate recomputes each refname from signed content — the parentless-roots walk for hash-identified namespaces (never applied to pins, whose ancestry reaches code history), natural-key tree fields (`Member` and `Effect` gain their name field), composite review segments, and result trees gaining effect + target (`model.result-identity`, closing a result-forgery replay). Strict genesis decode for the gate’s own types (a result) plus the pairwise schema-disjointness test across every genesis-borne struct; `gate.owner-mutation`; `propose_*` grows the sign-then-name genesis flow (the ref named from the signed commit’s own oid — no circularity, since no commit names its own ref anymore). Two gaps ship unclosed, neither assigned a phase: a comment or an issue’s type lives outside the gate’s own crate, so strict decode never runs for those namespaces (`gate.non-kernel-strict-decode`); and the binding’s redaction-vouching clause has no implementation (`gate.redaction-vouching-undefined`).
forge and surface identity migration (`ents-forge`, `git-ents`, `ents-web`, `ents-lens`, skills)	10	Medium	sonnet	`uuid` leaves the workspace: comment and issue ids are genesis commit oids, reviews move to the composite `reviews/<target>/<member>` key with re-review’s fast-forward advance now a reachable CLI path; the CLI’s plain display and the web UI abbreviate ids the way git abbreviates oids, while the `--porcelain` and lens machine-readable forms keep the full id (`lens.parity`); lens and agent skills updated for the id format.

facet-git-tree

Low

sonnet

Exists (external repo). Stays domain-blind. Land the #[facet(transparent)] fix upstream and drop the local [patch].

gix-ref-store

Medium

sonnet

Read/CAS trait split per arch.refstore-read-cas-split; loose-ref impl ports the CAS discipline from pre-redo (arch.loose-cas-discipline: no shelling to git update-ref). Fable reviews the trait before phase 3 consumes it.

ents-model

Low

sonnet

Entity structs, namespaces, trailers, taxonomy — declarative, but it is the vocabulary every crate imports, so spec fidelity and the public API review matter more than the code. Trailer parsing is new work, not a port: no git-metadata crate exists at pre-redo.

ents-gate

Medium

fable

Small in lines, security-critical in every line: tip signature, refname binding, DAG-FF, epoch, bootstrap fail-closed. Pure over the read half of RefStore. Exit test: identical verdicts at all three call sites.

ents-query

High

fable

The critical path: grammar/parser (easy), static footprint extraction, incremental set-entry via generation numbers, work set trigger − results(self, any) (hard). Property-test monotone entry-only semantics against synthetic ref histories.

ents-anchor

Medium

sonnet

Mostly a port: projection (blame + fuzzy context match) survives at pre-redo; re-home it on ents-model and the new retention rule (embed blob + context, no gitlinks).

ents-receive

Medium

sonnet

Orchestration above traits that already exist by now: gate policy (mandatory/advisory), footprint matching, enqueue, (effect, oid) dedup, redaction at ingest. Defines EventSink, reconstructible from repository state as the work set trigger − results(self, any) (query.workset); the in-memory impl plus a boot-time reconciliation scan is the reference implementation, durability deferred as a performance property (receive.reconstructible). Fable reviews the receive() signature — every frontend is downstream of it.

ents-sync

High

opus

Fetch/push plumbing is routine; the schema-aware three-way merge over typed trees (divergence + adoption on one machinery) is the hard part. Absorbs git-store’s merge.

ents-effect

Medium

sonnet

Executor trait + Docker/Sprite backends (feature-gated), toolchain materialization, run loop, write-back as a receive client. Ports docker readiness probes and sprite quirks from pre-redo; the design is settled, the environment is the risk.

git-ents (bin)

Medium

sonnet

Local root wiring (~50 lines) plus the real cost: subcommand surface and the denyCurrentBranch=updateInstead worktree edge. Headless — no serve subcommand yet (arrives Phase 7 with ents-web). Doubles as the single-node hosted root (roots.single-node-hosted): loose refs and a real odb on a Fly volume, served behind git’s own receive-pack — the same stock-git transport Phase 0 bootstraps, now invoking the gate from a hook — with an in-memory EventSink and a boot-time reconciliation scan, and the Sprite executor, deployed at git.ents.cloud. Milestone: CLI-complete. Two gaps ship unclosed, neither assigned a phase: no porcelain command or root bootstrap sets refs/meta/config’s verification epoch, so the hosted root’s Mandatory gate stays archival until one is designed (`gate.epoch-bootstrap); and meta-ref.tip-invariant’s generic escape-hatch display for an unrecognized meta-ref, and anchor resolution, have no command anywhere in this binary, despite `ents-model (phase 2) forward-referencing them to "the git-ents binary" by name.

ents-web

Medium

sonnet

Broad surface, low algorithmic depth. Lands serve in git-ents (roots.local): the local root’s own wiring, plus sessions/CSRF (roots.web-session) and signed mutations (roots.web-signing) layered on top, the user’s own key standing in for whichever identity a composition root injects. Signing identity is injected by the composition root; the crate assumes nothing about network reachability, so in-process webview embedding stays a supported deployment (roots.web-agnostic). The gap this phase shipped with — no composition root wiring ents-web onto a hosted root, the server-key indirection proven only against fixture identities — was closed after phase 10 by git ents serve --hosted (roots.single-node-hosted’s web-UI clause): the single-node hosted root mounts this same crate behind nginx with the mandatory gate, the server’s enrolled member key as signing identity, and member sign-in via the CLI-driven challenge flow (`roots.web-signin, receive.attributed-author — edits land as "member via the web").

gix-receive

High

opus

Scale-out only: structurally necessary once the Postgres RefStore leaves no on-disk repo for git’s own receive-pack to act on, not before. Smart-HTTP wire framing + pack ingestion, zero policy. Protocol-v2 and sideband subtleties; conformance-test against real git clients. Quarantine semantics per receive.object-access.

git-ents-server (bin)

High

opus

Postgres RefStore (multi-ref CAS in a transaction), Tigris object store behind gitoxide traits, a durable-queue EventSink, and gix-receive replacing git’s own receive-pack as transport — a new composition root swapped in with zero library-crate edits. This phase IS the honesty test (roots.honesty-test): entry is triggered by measured storage/load pressure (the kiln-cache growth analysis), not a date. Reuses the Sprite executor and sessions/CSRF from git-ents; salvage the store code and gix-receive framing from the scale-out work at pre-redo.

ents-forge conversations

Medium

fable

Comments become the universal conversational primitive (model.comment broadened, model.comment-state, model.comment-context, model.comment-thread, model.review): the Comment struct broadens as a clean break (experimental repo, no on-disk data to preserve, so no back-compat reader), working-tree capture and projection in ents-anchor (anchor.working-tree), the Review entity plus its retention pin ref (model.review-pin — refs/meta/pins/reviews/<target>/<member>, an empty-tree signed commit whose parents include the reviewed commit; verify the gate’s tip-signed and fast-forward checks accept the merge-shaped pin advance), and porcelain — comment reply/resolve/reopen, comment list --worktree with a machine-readable form (lens.parity), issue and review actions.

ents-lens

High

opus

The LSP frontend (lens.adoc, all rules): git ents lsp reusing the local root’s wiring the way serve does, code lenses + hint diagnostics + hover derived per-request from anchor projection onto the working tree, and the editor-file compose flow. No network, no state of its own.

ents-zed (extension)

Low

sonnet

Zed extension at editors/zed, outside the workspace (wasm target): registers the ents-lsp language server running git ents lsp. Verify current Zed LSP capability coverage against the real zed_extension_api — lens.diagnostics exists precisely so a client without code-lens rendering still shows the conversation.

ents-web conversations

Medium

opus

Issues index and detail with threads as context queries (model.comment-context), review creation and verdict display on commit pages (model.review), reply/resolve on the existing comment views. Reuses the registry-driven meta rail and session/CSRF machinery already in place.

kernel identity binding (ents-model, ents-gate, ents-receive, ents-sync, ents-effect, ents-testutil)

High

fable

One atomic migration — the kernel cannot build half-converted. Retire trailer.rs (Advance-ref and the never-used Schema-Version) in favor of meta-ref.identity-binding: the gate recomputes each refname from signed content — the parentless-roots walk for hash-identified namespaces (never applied to pins, whose ancestry reaches code history), natural-key tree fields (Member and Effect gain their name field), composite review segments, and result trees gaining effect + target (model.result-identity, closing a result-forgery replay). Strict genesis decode for the gate’s own types (a result) plus the pairwise schema-disjointness test across every genesis-borne struct; gate.owner-mutation; propose_* grows the sign-then-name genesis flow (the ref named from the signed commit’s own oid — no circularity, since no commit names its own ref anymore). Two gaps ship unclosed, neither assigned a phase: a comment or an issue’s type lives outside the gate’s own crate, so strict decode never runs for those namespaces (gate.non-kernel-strict-decode); and the binding’s redaction-vouching clause has no implementation (gate.redaction-vouching-undefined).

forge and surface identity migration (ents-forge, git-ents, ents-web, ents-lens, skills)

Medium

sonnet

uuid leaves the workspace: comment and issue ids are genesis commit oids, reviews move to the composite reviews/<target>/<member> key with re-review’s fast-forward advance now a reachable CLI path; the CLI’s plain display and the web UI abbreviate ids the way git abbreviates oids, while the --porcelain and lens machine-readable forms keep the full id (lens.parity); lens and agent skills updated for the id format.

Phase gates

Phase 0 exit: git.ents.cloud resolves over HTTPS; git clone and git push round-trip against it using stock git; this repo’s origin points at it; the tracey web dashboard at the same host reflects HEAD’s coverage after each push.
Phase 1 → 2: gix-ref-store passes a CAS conformance suite (concurrent writers, crash injection).
Phase 3 → 4: gate verdicts proven identical across call sites; query evaluator handles the staged-pipeline and fan-in idioms incrementally on a synthetic repo.
Phase 4 → 5: end-to-end local write path (receive in-process, advisory gate, null EventSink) green; sync merge fuzz-tested against divergent typed trees.
Phase 6 exit: the CLI-complete milestone — every porcelain command green against the single-node hosted root; the boot-time reconciliation scan regenerates obligations correctly after a kill -9 of the in-memory queue. The two gaps this row’s own notes name (gate.epoch-bootstrap; `meta-ref.tip-invariant’s missing reader surface) are known exceptions, not silent ones.
Phase 7 exit: git ents serve (roots.local) is loopback-only with no smart-HTTP surface added, drives every state-changing route through a per-session CSRF check (roots.web-session), and signs mutations through an identity the composition root injects, never one this crate resolves itself (roots.web-signing, roots.web-agnostic); the same router is provably reachable with no bound socket, driven in-process via tower::ServiceExt::oneshot. This row’s gap — no hosted deployment mounting the web UI, the server-key half proven only against fixtures — closed after phase 10: git ents serve --hosted wires it onto the single-node hosted root at git.ents.cloud, with member sign-in (roots.web-signin) gating mutations.
Phase 8 exit: the honesty test (roots.honesty-test) — the Postgres/Tigris/durable-queue root, now served by gix-receive in place of git’s own receive-pack, is wired as a new composition root with zero library-crate modification; gix-receive round-trips a push from stock git, and a diff of library-crate contents between phase entry and phase exit MUST be empty.
Phase 9 exit: the comment loop — a comment composed through the lens’s editor-file flow against a dirty working tree (lens.compose, anchor.working-tree) is listed open by the machine-readable git ents comment list --worktree form (lens.parity), resolved through the CLI (model.comment-state), and gone from the lens’s next publish; an issue and a review each round-trip CLI ↔ web with their threads rendered as context queries (model.comment-context, model.review). This being an experimental repository with no on-disk data to preserve, a struct change is a clean break: entities read only as their current shape, and no back-compat reader is kept — `meta-ref.migration’s forward rewrite applies whenever data actually needs carrying, not a permanent legacy code path.
Phase 10 exit: identity is derivation — no minted id and no binding trailer exists anywhere in the workspace. The doppelgänger replay (a signed mutation commit proposed as the genesis of a new entity) and the result replay (a signed pass proposed for a different effect or commit) are refused by tests driving receive (gate.identity-binding, model.result-identity); entity structs prove pairwise disjoint under strict decode; a comment created, replied to, and resolved, and a commit reviewed then re-reviewed (the pin advancing fast-forward), round-trip through CLI and web under genesis-oid and composite ids (model.comment, model.review). Same clean-break rule as phase 9: no legacy reader, ids regenerate with the data. The two gaps this row’s own notes name (gate.non-kernel-strict-decode; gate.redaction-vouching-undefined) are known exceptions, not silent ones: a comment or issue genesis is not yet refused from double-admission across namespaces, and the binding’s redaction-vouching clause is unimplemented.

Backend conformance tests are shared suites written once against each trait (RefStore, EventSink, Executor) and run per implementation — the pre-redo backend-conformance pattern, resurrected.

docs/development-plan.adoc

Development Plan

Agent scheme

Phase 0: bootstrap

Crate schedule

Phase gates