git ents: docs: add new abstraction for revsets

commit e1a2291

docs: add new abstraction for revsets

Assisted-by: Claude:claude-fable-5

Joseph D. Carpinelli · 1 month ago

tree browse at HEAD · parents: b34811e · comment on this commit

Reviews

No reviews of this commit yet — record a verdict below.

Start a review

docs/abstractions.adoc @@ -1,14 +1,13 @@ = git-ents Abstractions -The load-bearing abstractions, stated as invariants; everything else in -the project is an instance or a consequence. +The load-bearing abstractions, stated as invariants; everything else in the project is an instance or a consequence. -*Admission rule:* an abstraction appears here only if it already carries -weight in multiple crates or commands. Nothing speculative. +*Admission rule:* an abstraction appears here only if it already carries weight in multiple crates or commands, or closes a gap those abstractions exposed. +Where this document runs ahead of shipped code, the gap is marked. ''''' -== The five abstractions +== The six abstractions === 1. Meta-ref @@ -16,167 +15,228 @@ * *Storage* — the ref points to a commit whose tree is the entity. * *Synchronization* — fetch or push only the entities you care about. -* *Authorization* — pre-receive rules gate access by refname. -* *History* — the ref’s commit history is the audit trail. +* *Authorization* — refname-keyed rules gate who may advance the ref. +* *History* — the ref's commit history is the audit trail. -*Granularity rule:* one ref per independently-authored entity -(`refs/meta/member/*`, `refs/meta/comments/*`, `refs/meta/effects/*`, -`refs/meta/results/*`); one ref for repo-global state -(`refs/meta/account`, `refs/meta/config`). Entities that different -actors write concurrently must not share a ref. Writes stay -conflict-free; reads aggregate refs into views. +*Granularity rule:* one ref per independently-authored entity (`refs/meta/member/*`, `refs/meta/comments/*`, `refs/meta/effects/*`, `refs/meta/results/*`); one ref for repo-global state (`refs/meta/account`, `refs/meta/config`). +Entities that different actors write concurrently must not share a ref. +Writes stay conflict-free; reads aggregate refs into views. + +Two namespaces are consequences of the granularity rule plus the gate (5): + +* `refs/meta/inbox/*` — entities authored by someone not authorized for a canonical ref, awaiting adoption. +* `refs/meta/results/~<member>/*` — results produced by a member's own executor rather than a designated worker. + +Both hold the same typed trees as their canonical counterparts; only the refname rule differs. === 2. Typed tree A Rust struct annotated `#[derive(Facet)]` *is* the storage schema. -`facet-git-tree` maps struct ↔ Git tree directly; no serialization -format exists to version. +`facet-git-tree` maps struct ↔ Git tree directly; no serialization format exists to version. -Changing a struct is a storage migration, not a refactor — and a -migration is itself a signed push: rewrite the tree under the new -struct, commit on the ref’s old tip. History keeps the old encoding as -archive. Trees stay pure struct representations — no version marker -entry. If explicit version detection is ever needed, it belongs in a -`Schema-Version:` commit-message trailer: the commit is already the -storage unit, so versioning the encoding there pollutes neither the -tree nor the merge path. +Changing a struct is a storage migration, not a refactor — and a migration is itself a signed commit: rewrite the tree under the new struct, commit on the ref's old tip. +History keeps the old encoding as archive. +Trees stay pure struct representations — no version marker entry. -*Tip invariant:* the tip of a meta-ref is always readable by the -current binary; history is archival. +Ref-level metadata that is not entity content belongs in commit-message trailers, because the commit is already the storage unit. +Two trailers are reserved: `Schema-Version:` for explicit encoding detection if it is ever needed, and `Ents-Ref:` for refname binding (see 4). +Versioning or binding in the tree would pollute both the schema and the merge path. + +*Tip invariant:* the tip of a meta-ref is always readable by the current binary; history is archival. === 3. Anchor -A durable pointer into source: blob, optional line range, specific -commit. +A durable pointer into source: blob, optional line range, specific commit. -* *Retention invariant:* the tree storing an anchor embeds the anchored -blob, plus a context blob of the surrounding lines, as ordinary entries -— content addressing makes this free. The anchored content is reachable -from `refs/meta/*` and survives force-push, branch deletion, and gc — no -gc special-casing, and no pinned ancestry: the anchored commit’s oid is -recorded as data only. (Gitlinks are not reachability edges and retain -nothing; embedding is the only mechanism that works.) -* *Projection:* anchors project onto newer commits at read time — blame -plus fuzzy matching against the context blob; anchor data is never -mutated. When the anchored commit has been gc’d, projection degrades to -context matching instead of breaking. +* *Retention invariant:* the tree storing an anchor embeds the anchored blob, plus a context blob of the surrounding lines, as ordinary entries — content addressing makes this free. +The anchored content is reachable from `refs/meta/*` and survives force-push, branch deletion, and gc — no gc special-casing, and no pinned ancestry: the anchored commit's oid is recorded as data only. +(Gitlinks are not reachability edges and retain nothing; embedding is the only mechanism that works.) +* *Projection:* anchors project onto newer commits at read time — blame plus fuzzy matching against the context blob; anchor data is never mutated. +When the anchored commit has been gc'd, projection degrades to context matching instead of breaking. -Anchors are independent of any consumer; comments use them, but reviews, -TODOs, and blame overlays can too. +Anchors are independent of any consumer; comments use them, but reviews, TODOs, and blame overlays can too. -=== 4. Signed push +=== 4. Signed commit -`git push` is the *only write path*. Every mutation, including metadata, -is a signed push certificate verified by pre-receive against the member -refs. +Every meta-ref mutation is an author-signed commit. +The signature is a data artifact, not a transport artifact: it replicates with the repo and verifies offline in every clone, so verification evidence is itself repository state. +Push certificates are demoted to transport concerns; they carry no meta-ref semantics. -* Web-UI edits become real pushes signed by the server’s key, which must -itself appear in the member database. -* Effect workers write results with their own member keys. *No -privileged write path exists.* -* Auth state is repository state: no session database, no token table. -Revocation is a state on the member entity, not deletion — a revoked key -must be explicitly rejected, whereas deletion would merely make old -signatures unverifiable. -* Authorization is refname-keyed pre-receive rules. -`refs/meta/effects/*` is admin-writable only: authoring an effect -schedules code execution, which requires more trust than pushing a -branch. This rule must exist explicitly; it is not the default. -* Bootstrap: an empty member list admits every push so the first member -can enroll. Hosted, this leaves “first push owns the repo” open per -repository; closing it — repository creation requires an existing -account — needs a server-level key→account registry, because bare repos -are created on the first `info/refs` request, before any push -certificate exists. Known gap, direction chosen, not yet enforced. +A commit signature proves authorship; it does not prove placement. +The difference is recovered explicitly: -=== 5. Effect +* *Refname binding* — an `Ents-Ref:` trailer names the ref the commit was authored for, checked at verification; without it, a signed commit could be replayed as the tip of a different meta-ref. +* *Anti-replay* — meta-refs advance fast-forward-only in the DAG sense (the new tip descends from the old), enforced by atomic CAS on the ref store; the parent hash is the freshness binding, so no nonce is needed. -Push-triggered side effects are the *only side-effect path*. An effect -is repository data under `refs/meta/effects/<name>`: +*Tip invariant:* the tip of a meta-ref is signed by a member authorized for that refname. +This is checkable after the fact by anyone with a clone, not only by whoever ran the server. + +*Adoption, never rewrite:* when author and placer differ — an inbox entity accepted by an admin, a user-run result accepted by a maintainer — the authorized member merges the contributor's commit onto the canonical ref. +The merge commit satisfies the tip invariant; the contributor's signed commit sits in ancestry with attribution cryptographically intact. +Cherry-picking creates a new commit object and destroys the author's signature; it is forbidden as an adoption mechanism. +Fast-forwarding directly to a contributor's commit would put an unauthorized signature at the tip; adoption is always a merge, even a trivial one. + +*The principled split:* content signatures carry authorization only where mutations are author-signed single-writer appends — which the granularity rule guarantees for meta-refs. +On `refs/heads/*`, pushing commits you did not author is legitimate, so branch refs keep transport-level auth. +"Signed push is the only write path" is therefore retired; the invariant is the tip invariant above, which is strictly stronger where it applies. + +=== 5. Gate + +Verification is a pure function over ref-store reads: + +[arabic] +. The new tip is signed by a member authorized for this refname. +. The `Ents-Ref:` trailer matches the refname. +. The new tip descends from the old tip. +. The update commits via atomic CAS. + +Because members and refname rules live under `refs/meta/*`, the policy is repository state: any frontend evaluates the *actual* policy, offline, staleness bounded by last fetch. + +*The gate is a property of the store, not of writing:* + +* The hosted store runs the gate at CAS time; failure aborts the transaction (*mandatory*). +* The local store accepts any write and runs the gate as a verdict; failure annotates (*advisory*). + +Enforcing the gate locally would destroy offline-first: you could not author while unenrolled or work against an unfetched member list. +You can always write locally what hosted will never accept; the sync path's answer to a rejected canonical push is an offer to push to your inbox ref instead. + +Three call sites, one function: hosted CAS, local UI verdict, push pre-flight. +The hosted server is not where policy lives; it is the one place where the verdict has teeth. + +The local web UI signs as the user, with the user's own member key, indistinguishable from CLI-authored commits. +Server-key signing of UI edits is a hosted necessity only (a browser cannot hold the git signing key) and must not be imported locally, where it would be strictly worse provenance. + +=== 6. Effect + +An effect is a declarative, content-addressed subscription to a commit-set query, whose execution is sandboxed and whose output re-enters the repository as verified data. +It is repository data under `refs/meta/effects/<name>`: [source,rust] ---- #[derive(Facet)] struct Effect { - trigger: RefPattern, // e.g. "refs/heads/*" + trigger: CommitQuery, // a commit set as a function of ref state toolchains: Vec<ToolchainRef>, // e.g. ["rust-1.88"] run: Command, results: RefName, // e.g. "refs/meta/results/<name>/*" } ---- +*Trigger semantics:* the trigger denotes a set of commits; the effect fires once per commit that *enters* the set. +The query algebra is deliberately tiny: + +* `rev(expr)` — any revspec or ref glob: `refs/heads/*`, `main ^release`, ancestry, merge-base. +* `results(effect, status)` — commits having a result of that status; cheap to resolve because the results refname encodes the tested oid. +* Set operations — union, intersection, difference. + +`RefPattern` survives as the degenerate query `rev(<glob>)`; nothing shipped changes meaning. +*(Gap: `CommitQuery` beyond the degenerate case is design, not code.)* + +*Monotone, entry-only:* a force-push can shrink a set, but a commit leaving the set retracts nothing — results are immutable history; the commit merely stops being an obligation. +Monotone semantics is what makes distributed evaluation safe with no coordination beyond the existing CAS. + +*No pipeline state:* the work set is `trigger − results(self, any)` — the results ref is the materialization marker. +The dedup key `(effect, oid)` over an at-least-once queue yields exactly-once outcomes with zero state outside the repository. + +*Recursion is structure, not a rule:* downstream-of-effects is syntactically visible — a query names `results(...)` or it does not. +Closure is opted into by writing the query you mean; the accidental fork bomb is unreachable. + Pipeline invariants: [arabic] -. `post-receive` matches `(refname, old, new)` against -`refs/meta/effects/*`; matches enqueue `(effect, refname, new_oid)`. -That tuple is the dedup key — the queue is at-least-once; git’s content -addressing makes idempotency nearly free. -. Pushes are never blocked; the durable enqueue is the entire -synchronous cost. -. A worker dequeues, materializes declared toolchains from -`refs/meta/toolchains/*`, executes in a sandbox — one trait, two -backends: Fly.io Sprite hosted, Docker local. Host-direct exists only -behind an explicit `--unsandboxed`. -. Results return *only* via signed push to the effect’s results ref, one -ref per tested commit (`refs/meta/results/<effect>/<short-oid>`), so -concurrent results never conflict. +. `post-receive` remains a dumb matcher: a query's ref footprint is statically extractable, so a transition maps to affected queries, which enqueue re-evaluation. +Set entry is computed incrementally from the `old..new` frontier, bounded by commit-graph generation numbers. +. Pushes are never blocked; the durable enqueue is the entire synchronous cost. +. A worker dequeues, materializes declared toolchains from `refs/meta/toolchains/*`, and executes in a sandbox — one trait, multiple backends: Fly.io Sprite, Docker. +Host-direct exists only behind an explicit `--unsandboxed`. +. Results return *only* as signed commits pushed to the effect's results ref, one ref per tested commit (`refs/meta/results/<effect>/<short-oid>`), so concurrent results never conflict. -*Recursion rule:* triggers never match `refs/meta/results/*` or -`refs/meta/index/*`. Results and indexes are pushes, so effects could -trigger effects; that closure must be opted into per effect, never -arrived at by accident. +*Identity discipline:* the runner is a member, not an ambient authority. +A result signature proves who pushed the result, not that the run was faithful; official results are official because canonical results refs are writable only by designated worker keys — a refname rule, not a runtime property. +Any member may run any effect on their own executor and account; their results land in `refs/meta/results/~<member>/*` or the inbox, adoptable by merge like everything else, with the trust decision explicit in the adoption. +Anyone can run CI; nobody can impersonate the verdict. -The sandbox bounds the runtime blast radius; the admin-only write rule -on `refs/meta/effects/*` bounds who can schedule execution at all. +*Where the line holds:* no content predicates beyond `results(…, status)`, no time atoms, no external-event atoms. +A query language that reads file contents is a build system's dependency scanner in the trigger layer; that work belongs inside effects, not around them. +The bet, stated as a bet: DAG membership plus results membership is a sufficient trigger basis, and composition happens in the repository, not in a workflow language. + +The sandbox bounds the runtime blast radius; the admin-only write rule on `refs/meta/effects/*` bounds who can schedule execution on canonical infrastructure at all. Hosted mode requires both. ''''' == The loop -Abstractions 4 and 5 are dual: one write path in, one side-effect path -out — and the side-effect path closes back into the write path, because -results are signed pushes by a worker that is just another member. All -state changes, human or machine, flow through the same verified, audited -channel. This closure is the design’s central property. +Abstractions 4, 5, and 6 close into each other: signed commits are the only way state changes, the gate is the only admission judgment, effects are the only side-effect path — and effect results are signed commits by workers that are just members, judged by the same gate. +All state changes, human or machine, flow through one verified, audited channel, and the repository is the message bus. +This closure is the design's central property. + +''''' + +== Composition (consequence of the abstractions) + +No code knows where it is running; the core is handed trait objects and never asks. +Any `if hosted` branch inside the library is the design failing. + +The seams: + +* `RefStore` — reads plus atomic multi-ref CAS. +Local: loose refs written by our code, not `git update-ref`, so local mutations honor the same CAS discipline. +Hosted: Postgres rows. +* `ObjectStore` — local: the odb; hosted: Tigris. +* `EventSink` — where post-receive matches go. +Local: null; hosted: a durable queue. +* `Executor` — Docker, Sprite, or unsandboxed, chosen by whoever built it. + +"Push" conflates object transfer with a verified ref transaction; locally, transfer is vacuous. +The unit the library exposes is `receive(refs, objects, events, proposal)`: gate evaluation, effect matching, and enqueue live inside it, above the traits. +Frontends construct a `Proposal` and call it — the CLI and local UI in-process against the odb, smart-HTTP by unpacking the wire pack into the `ObjectStore` first. +Local and hosted do not share a push path; they share `receive`, with only the trait impls swapped. +That is the correctness anchor for writes, parallel to `git effect run` for execution. + +Composition roots are the only place deployment exists — roughly fifty lines each of trait construction and wiring: + +* `git-ents` (CLI): file `RefStore`, odb, Docker executor, null `EventSink`, advisory gate. +`git ents serve` is the same wiring plus the smart-HTTP frontend on loopback. +Local effect execution is pull (`git effect run`), never a daemon watching refs; the queue is the only skipped component, and it carries no correctness content. +* `git-ents-server` (hosted): Postgres, Tigris, durable queue, Sprite executor, mandatory gate. +* The worker — queue consumer, executor, push-back client — is a real seam in the library and a packaging decision in the binary; it stays inside `git-ents-server` until scale forces the split, but never shares in-process state with receive. + +What to run is repo data; how to run it is a deployment property. +An effect must not be able to demand its own executor or `--unsandboxed`. +Config selects trait impls at the root and never leaks past it; core code reading config is a trait that should exist and does not. + +Store and executor are orthogonal axes, so the matrix has quadrants nobody designs: local store plus Sprite executor is `git effect run --executor sprite` with your own Fly token, and it works because everything the run needs is repo data. +The honesty test for the seams: a third root — say single-node self-hosting on SQLite, local fs, and Docker — must be constructible without touching the library. ''''' == Derived, not fundamental -Instances and consequences of the five, listed for orientation: +Instances and consequences of the six, listed for orientation: -* *CI checks* — the first shipped effect: trigger on branch pushes, run -a command, publish results. Future effects (comment notifications, -member-cache rebuilds, ATProto mirroring) are registrations, not -subsystems. -* *Toolchains* — typed trees under `refs/meta/toolchains/*`; a resource -effects declare, not a trigger. The repo carries its own execution -environment with provenance, as ~1KB hash-pinned manifests; only the -sandbox touches the bytes. They keep a subcommand only because -import/activation logic is nontrivial; if that shrinks, the subcommand -dies. -* *Members, accounts, comments* — typed trees behind meta-refs (1+2), -written via signed pushes (4). -* *Fanout indexes* — discovery without ref enumeration: -`refs/meta/index/*` maps object oids to the entities anchored to them, -rebuilt by an effect (5) and written via the worker’s signed push (4). -Clients read the index; a stale or absent index degrades to scanning -ref tips, never to wrong answers. -* *Rendering registry* — documents render by MIME type through one -lookup table (HTML web, plain-text CLI; unknown types pass through). -Implementation choice. -* *Embeddable server* — `git-ents-server` is a library first; the serve -command, standalone binary, and hooks-as-subcommands are thin wrappers. +* *CI checks* — the first shipped effect: trigger on branch pushes, run a command, publish results. +* *Pipelines* — query composition, not a subsystem: staged CI is `rev(refs/heads/main) ∩ results(unit, pass)`, fan-in is intersection, conditional edges are difference. +The pipeline state is the results namespace. +* *The inbox workflow* — the email-patch model rebuilt on the primitives: inbox ref as mailing list, adoption merge as the maintainer applying a signed patch. +* *User-funded CI* — hosted CI is not load-bearing: a contributor with no canonical push access gets full CI on the real toolchains, self-executed, results shareable and verifiable. +The server's monopoly reduces to custody of the canonical refs. +* *Toolchains* — typed trees under `refs/meta/toolchains/*`; a resource effects declare, not a trigger. +The repo carries its own execution environment with provenance, as ~1KB hash-pinned manifests; only the sandbox touches the bytes. +They keep a subcommand only because import/activation logic is nontrivial; if that shrinks, the subcommand dies. +* *Members, accounts, comments* — typed trees behind meta-refs (1+2), written as signed commits (4), admitted by the gate (5). +* *Fanout indexes* — discovery without ref enumeration: `refs/meta/index/*` maps object oids to the entities anchored to them, rebuilt by an effect (6) and written via the worker's signed commit (4). +Clients read the index; a stale or absent index degrades to scanning ref tips, never to wrong answers. +* *Rendering registry* — documents render by MIME type through one lookup table (HTML web, plain-text CLI; unknown types pass through). Implementation choice. +* *Embeddable server* — `git-ents-server` is a library first; the serve command, standalone binary, and hooks-as-subcommands are thin wrappers. +This is not merely packaging: it is what keeps all authorization and effect-matching logic in the library rather than smeared across subprocess boundaries. ''''' == Command surface (consequence of the abstractions) -Primitives are local plumbing; `git ents` adds exactly one capability: -remote synchronization (fetch the relevant `refs/meta/*` first). +Primitives are local plumbing; `git ents` adds exactly one capability: remote synchronization (fetch the relevant `refs/meta/*` first). .... git store show refs/meta/member/joey # generic escape hatch: pretty-prints @@ -190,8 +250,11 @@ git effect list git effect add ci --on 'refs/heads/*' --toolchain rust-1.88 -- cargo test +git effect add integ --on 'rev(refs/heads/main) & results(ci, pass)' \ + --toolchain rust-1.88 -- cargo test --features integ git effect show ci [<commit>] # definition + results for a commit -git effect run ci [--at <commit>] # local execution: identical toolchain +git effect run ci [--at <commit>] [--executor docker|sprite] + # local execution: identical toolchain # materialization and sandbox path, # queue skipped — nothing else differs git effect log ci # results ref history @@ -201,8 +264,7 @@ git toolchain log rust-1.88 .... -`git effect run` is the correctness anchor: local and hosted execution -share the identical path or the abstraction is decoration. +Two correctness anchors: `git effect run` shares the identical materialization and sandbox path with the hosted worker, and every mutation frontend shares the identical `receive` with the hosted server — or the abstractions are decoration. Porcelain: @@ -213,10 +275,13 @@ git ents effect ... git ents toolchain ... git ents comment ... +git ents inbox list|adopt git ents login git ents serve | server .... +Revocation is a state on the member entity, not deletion — a revoked key must be explicitly rejected, whereas deletion would merely make old signatures unverifiable. + ''''' == Deployment (consequence of the abstractions) @@ -226,19 +291,19 @@ | |Local (`git ents serve`) |Hosted (`git-ents-server`) |Repositories |Real working repositories |Bare repositories |Discovery |Current repo or directory scan |Created on first push -|Auth |Signed pushes optional by default |Signed pushes required -|Effects |Execute inside a Docker container |Execute inside a Fly.io -Sprite +|Ref store |Loose refs via `RefStore`, CAS-disciplined |Postgres rows, atomic CAS +|Object store |The odb |Tigris +|Gate |Advisory — writes annotated, never blocked |Mandatory — failure aborts the CAS +|Effects |Pull, via `git effect run` (Docker default) |Push-triggered, durable queue, Fly.io Sprite |Purpose |Personal forge, development, demos |Production forge |=== -Working repos reject pushes to the checked-out branch; the local server -sets `receive.denyCurrentBranch=updateInstead` so pushes also update the -working tree. Known edge: `updateInstead` fails on a dirty worktree, so -branch-push behavior is not perfectly identical to hosted mode — -metadata pushes are. +Working repos reject pushes to the checked-out branch; the local server sets `receive.denyCurrentBranch=updateInstead` so pushes also update the working tree. +Known edge: `updateInstead` fails on a dirty worktree, so branch-push behavior is not perfectly identical to hosted mode — metadata behavior is. +Worktree update is frontend business, after `receive` accepts; core never touches a worktree. -Pushes to `refs/meta/*` never touch the working tree, so all metadata -behaves identically in both modes: the deployment model is an -implementation detail of the data model — a direct consequence of -abstraction 1. +Pushes to `refs/meta/*` never touch the working tree, so all metadata behaves identically in both modes: the deployment model is an implementation detail of the data model — a direct consequence of abstractions 1 and 5. + +Bootstrap remains the known gap: an empty member list admits every push so the first member can enroll. +Hosted, this leaves "first push owns the repo" open per repository; closing it — repository creation requires an existing account — needs a server-level key→account registry, because bare repos are created on the first `info/refs` request, before any push certificate exists. +Direction chosen, not yet enforced.

docs/design.adoc @@ -1,0 +1,99 @@ += git-ents: The Shape of the System + +`abstractions.adoc` states the invariants; this document states what they add up to. +It is the design as a whole — the thesis, the stories that follow from it, and what the end application is. + +''''' + +== Thesis + +A forge is a repository plus a store with an admission policy. +Everything else a forge appears to be — identity, authorization, discussion, CI, audit — is repository state, and therefore portable, offline-verifiable, and owned by whoever holds a clone. + +The consequence stated bluntly: the hosted server is not the forge. +It is custody of the canonical refs, a gate with teeth, and a queue. +Delete it and every clone still contains the members, the policy, the discussion, the CI verdicts, and the cryptographic evidence that all of it is authentic. + +''''' + +== The write story + +"Push" conflates two things: object transfer and a verified ref transaction. +Locally, transfer is vacuous — so the system's real write primitive is the transaction: `receive(refs, objects, events, proposal)`, a library function pure over storage traits. +Every mutation frontend — CLI, local web UI, smart-HTTP — constructs a proposal and calls the same function. +Local and hosted do not share a push path; they share `receive`. + +Mutations are author-signed commits, not push certificates. +A push cert signs a transition and evaporates at the transport layer; a commit signature replicates with the repo and verifies in every clone, forever. +Verification evidence is repository state, like everything else. +Refname binding lives in an `Ents-Ref:` trailer; anti-replay is fast-forward-plus-CAS, with the parent hash as the freshness binding. + +Because writing and verifying are separated, local is genuinely offline-first: the local store accepts any write and the gate merely annotates. +You can author while unenrolled, work against an unfetched member list, and accumulate meta-refs the canonical store would reject. +The gate is the same pure function everywhere; only its consequence differs — advisory locally, mandatory at the hosted CAS. + +''''' + +== The trust story + +A commit signature proves authorship; refname rules prove placement; the two are deliberately distinct. +In the normal case one person is both, and the tip invariant — every meta-ref tip is signed by a member authorized for that refname — is checkable after the fact by anyone with a clone. + +When author and placer differ, the mechanism is adoption: an authorized member merges the contributor's signed commit onto the canonical ref. +The merge satisfies the tip invariant; the contributor's signature survives intact in ancestry. +Cherry-pick is forbidden as an adoption verb because it destroys the signature it is supposed to honor. + +This resurrects the email-patch workflow on native primitives. +The inbox ref is the mailing list; the adoption merge is the maintainer applying a signed patch; the trust decision is explicit, recorded, and attributed on both sides. +A rejected canonical push degrades to an inbox offer, not an error. + +Workers are inside the same trust model. +An effect runner is a member with a key, scoped by refname rules, revocable as repository state — never an ambient authority. +"Official CI" is not a runtime property; it is a refname rule saying canonical results refs are writable only by designated worker keys. + +''''' + +== The execution story + +The repository is an event-sourced system: refs are the log, effects are the consumers, membership is the access control, and the queue is the only component outside the data model — pure plumbing with no correctness content. + +An effect subscribes to a commit-set query and fires once per commit entering the set. +Pipelines are query composition, not orchestration: staged CI is an intersection with a results set, fan-in is intersection, conditional edges are difference. +The pipeline's state is the results namespace; the work set is `trigger − results(self, any)`; exactly-once outcomes fall out of content addressing with zero state outside the repo. +Semantics are monotone and entry-only, which is what makes distributed evaluation safe with nothing but the existing CAS. + +Execution is orthogonal to storage. +What to run is repo data; how to run it is a deployment property chosen at a composition root — so "local store, cloud executor" is a quadrant that exists without being designed: `git effect run --executor sprite` on your own account. +Results you run yourself land in your own namespace, cryptographically distinct from official verdicts, adoptable by merge when a maintainer decides to trust them. + +The economic consequence: hosted CI is not load-bearing. +A contributor with no push access gets full CI on the real toolchains, self-funded and self-executed, with shareable, verifiable results. +Anyone can run CI; nobody can impersonate the verdict. + +''''' + +== The deployment story + +No code knows where it is running. +The core is handed four traits — `RefStore`, `ObjectStore`, `EventSink`, `Executor` — and deployment exists only in composition roots of roughly fifty lines. +Local wires files, the odb, Docker, a null sink, and an advisory gate; hosted wires Postgres, Tigris, a durable queue, Sprites, and a mandatory gate. +The honesty test is that a third root — single-node self-hosting, say — is constructible without touching the library. + +Policy travels with the repo, so the local UI does not approximate the remote's rules; it evaluates them, offline, staleness bounded by last fetch. +The hosted server is not where policy lives — it is the one place where the verdict has teeth. + +''''' + +== What it is to use + +*A solo developer* has a complete forge on a laptop: comments anchored into source, effects run on demand in the same sandbox path production uses, a local web UI signing with their own key — no daemon, no account, no server. + +*A team* adds exactly one thing: a canonical store. +Enrollment is a signed commit; revocation is a state change; the audit trail is ref history; nobody administers a session database because none exists. + +*An outside contributor* has almost everything a member has: they author signed entities locally, run the project's real CI on their own executor, and submit through the inbox — attribution guaranteed by their own signature, acceptance recorded in the maintainer's. + +*An auditor* needs only a clone: every policy decision, membership change, CI verdict, and adoption is a signed commit whose verification requires no server's cooperation. + +The design's central property, restated as experience: there is one channel. +Human or machine, local or hosted, everything that changes state flows through the same verified, audited path — and that path lives in the repository you already have.

docs/faq.adoc @@ -1,0 +1,241 @@ += git-ents: FAQ and User Stories + +`abstractions.adoc` states the invariants and `design.adoc` states the shape; this document answers the questions a person actually has, organized as user stories. +Every interaction the system supports should appear here; an interaction with no story is either derived from one that does, or a gap in this document. + +''''' + +== What is this? + +*What is git-ents?* +A forge — issues, comments, code review, CI, membership — where every piece of forge state is a git object under `refs/meta/*` in the repository itself. +There is no forge database; a clone of the repository is a clone of the forge. + +*How does it work, in one paragraph?* +Entities (members, comments, effects, results) are Rust structs stored directly as git trees, one ref per independently-authored entity. +Every mutation is a commit signed by its author; a pure verification function — the gate — checks that each meta-ref tip is signed by a member authorized for that refname. +Side effects (CI and everything like it) are declarative subscriptions to commit sets, executed in sandboxes by workers who are themselves members, writing results back as signed commits. +The hosted server is only a store that runs the gate with teeth; locally, the same gate advises instead of blocks. + +*What do I need to run it?* +Locally: the `git-ents` binary and a repository. +There is no daemon, no account, and no server in the local story. + +*Is the hosted server required for anything?* +Only for a shared canonical store — a single place a team agrees is authoritative. +Identity, policy, discussion, CI, and audit all work from any clone. + +''''' + +== Getting started (solo developer) + +*I want a forge for my personal project, with no server.* +Run `git ents setup` in your repository. +You now have comments, effects, toolchains, and a local web UI, all storing state under `refs/meta/*` in the repo you already have. + +*I want to enroll myself as a member.* +`git ents members add` creates a member entity carrying your public key, written as a commit signed by that key. +On an empty member list, the first enrollment is self-admitting by design — see the bootstrap question under Hosted. + +*I want to browse my forge in a browser.* +`git ents serve` runs the web UI locally against your working repository. +Edits made in the UI are signed with your own member key and are indistinguishable from CLI-authored commits. + +*I want all of this to survive cloning my repo to a new machine.* +It does by construction: `git clone` plus fetching `refs/meta/*` moves the entire forge, including its audit history and the signatures needed to verify it. + +''''' + +== Comments and anchors + +*I want to leave a comment on specific lines of code.* +`git comment add --file src/lib.rs --lines 40-52 -m "why saturating_add?"`. +The comment is a typed tree on its own ref, anchored to the blob, line range, and commit you commented on. + +*I want my comment to still make sense after the code changes.* +Anchors project onto newer commits at read time using blame plus fuzzy matching; the stored anchor is never mutated. + +*I want comments to survive force-pushes, branch deletion, and gc.* +The anchor embeds the commented blob and its surrounding context as ordinary tree entries, reachable from `refs/meta/*`. +Content addressing makes this free, and no gc special-casing is needed. + +*I want to see all comments on the current commit.* +`git comment show` aggregates comment refs and projects their anchors onto your checkout. + +*I want to build something else on anchors — review threads, TODO overlays, blame annotations.* +Anchors are consumer-independent; comments are merely their first client. + +''''' + +== Effects (CI and everything like it) + +*I want tests to run when branches move.* +`git effect add ci --on 'refs/heads/*' --toolchain rust-1.88 -- cargo test`. +The effect is repository data; on a hosted store, matching pushes enqueue runs. + +*I want to run CI locally before pushing, and trust that it matches production.* +`git effect run ci` materializes the same toolchains and executes in the same sandbox path as the hosted worker; only the queue is skipped. +This identity is a stated correctness anchor of the project, not a convenience. + +*I want a staged pipeline — integration tests only after unit tests pass.* +Triggers are commit-set queries: `git effect add integ --on 'rev(refs/heads/main) & results(ci, pass)' ...`. +The stage is a set intersection; there is no orchestrator and no pipeline database — the results namespace is the pipeline state. + +*I want fan-in: run when several conditions all hold.* +Intersect the sets; the effect fires when the last prerequisite lands, regardless of the order refs moved. + +*I want to exclude WIP branches.* +Set difference: `rev(refs/heads/*) - rev(refs/heads/wip/*)`. + +*I want an effect that reacts to another effect's results.* +Name `results(...)` in your query. +Downstream-of-effects is syntactically visible, so effect recursion is always opted into and never reached by accident. + +*I want to see what CI said about a commit.* +`git effect show ci <commit>` reads the results ref for that commit; `git effect log ci` shows results history. +Results are one ref per tested commit, so concurrent runs never conflict. + +*I want CI runs on my own cloud account, from my laptop.* +`git effect run ci --executor sprite` with your own Fly token. +Storage and execution are orthogonal; everything a run needs is repo data, so any store can pair with any executor. + +*I want to know why my effect ran twice.* +The queue is at-least-once; the dedup key `(effect, oid)` plus content addressing makes reruns produce identical, idempotent outcomes. +The work set is `trigger − results(self, any)`, so a commit with a result is no longer an obligation. + +*I want an effect triggered by file contents, on a timer, or by an external webhook.* +Deliberately unsupported in the trigger language. +Content awareness belongs inside the effect's command, and time or external events belong to whatever writes a ref; triggers stay DAG membership plus results membership. + +*I want to run an effect without a sandbox.* +Only behind an explicit `--unsandboxed`, only locally, and an effect can never demand it as data. + +''''' + +== Toolchains + +*I want CI to use a pinned compiler, everywhere.* +`git toolchain import rustup:1.88-aarch64-apple-darwin` stores a ~1KB hash-pinned manifest under `refs/meta/toolchains/*`. +The repo carries its own execution environment with provenance; only the sandbox ever touches the bytes. + +*I want to inspect or audit a toolchain.* +`git toolchain view rust-1.88` and `git toolchain log rust-1.88` — the manifest is a typed tree and its history is a ref like any other. + +''''' + +== Teams, membership, and trust + +*I want to add a teammate.* +`git ents members add` — a signed commit by someone authorized for the member refs. +Enrollment is repository state; there is no user database beside the repo. + +*I want to remove someone's access without breaking history.* +`git ents members revoke` marks the member revoked; the key is explicitly rejected from then on. +Revocation is a state, never a deletion — deleting the entity would merely make their old signatures unverifiable, which is the opposite of what an audit needs. + +*I want to verify that nobody tampered with forge state.* +Every meta-ref tip must be signed by a member authorized for that refname, bound to the ref by an `Ents-Ref:` trailer, descending from the previous tip. +`git ents members check` verifies this from any clone, with no server's cooperation. + +*I want to know who did what, and when.* +Each ref's commit history is the audit trail; each commit is signed by its actor, human or worker. + +*I want machine actors (CI workers) in the same trust model as people.* +Workers are members: enrolled keys, scoped by refname rules to their results refs, revocable as state. +No privileged write path exists for machines. + +''''' + +== Contributing from outside + +*I want to comment on a project I can't push to.* +Author the comment locally — the local store accepts any write. +The gate tells you, offline, that the canonical store would reject it, and sync offers to push to your inbox ref instead. + +*I want a maintainer to accept my contribution with my name cryptographically on it.* +The maintainer adopts by merging your signed commit onto the canonical ref. +Their merge satisfies the tip invariant; your signature survives intact in ancestry. +Cherry-pick is forbidden as an adoption verb because it would destroy the signature it is meant to honor. + +*I want to prove my patch passes CI, without any access.* +Run the project's real effects on your own executor; results land in your namespace (`refs/meta/results/~you/*`), signed by you, shareable and verifiable. +A maintainer can adopt them, with the trust decision explicit in the adoption merge. + +*I want to know why this feels like the old email-patch workflow.* +Because it is that workflow rebuilt on native primitives: inbox ref as mailing list, adoption merge as the maintainer applying a signed patch, attribution guaranteed on both sides. + +''''' + +== Local vs hosted + +*I want to work fully offline.* +Everything writes locally; the gate never blocks a local write. +Policy is repository state, so the gate evaluates the actual canonical rules offline, staleness bounded by your last fetch. + +*I want to know before pushing whether my push will be accepted.* +The push pre-flight runs the same gate function the hosted store runs at CAS time; it is a prediction that can only be stale, never wrong about the rules. + +*I want to understand what the hosted server actually is.* +A store with a mandatory gate and a durable queue: custody of canonical refs, admission with teeth, and effect dispatch. +Policy does not live there; it merely has consequences there. + +*I want web edits on the hosted UI attributed correctly.* +Hosted web edits are commits signed by the server's key, which must itself be an enrolled member — a necessity because a browser cannot hold your signing key. +Locally, the UI signs as you; the server-key indirection is never imported where it isn't forced. + +*I want to self-host on one node without Postgres or Tigris.* +Deployment exists only in composition roots wiring four traits (`RefStore`, `ObjectStore`, `EventSink`, `Executor`). +A single-node root — SQLite, local filesystem, Docker — is the project's own honesty test for its seams. +*(Gap: this root does not exist yet.)* + +*I want to log into a hosted forge.* +`git ents login` links your key to an account; auth state is repository state, so there is no session database or token table. + +*I want to create a repository on a hosted server.* +Today, first push creates it — which means "first push owns the repo" is open per repository. +Closing it requires a server-level key→account registry, because bare repos are created before any signed push exists. +Known gap; direction chosen, not yet enforced. + +''''' + +== Power tools and edges + +*I want to inspect any entity, even one my binary doesn't know.* +`git store show refs/meta/<anything>` pretty-prints any typed meta-ref — the generic escape hatch. + +*I want to find every comment on a given object without enumerating refs.* +Fanout indexes under `refs/meta/index/*` map oids to the entities anchored to them, rebuilt by an effect. +A stale or absent index degrades to scanning ref tips — never to wrong answers. + +*I want to change an entity's schema.* +Changing the struct is a storage migration: rewrite the tree under the new struct and commit on the old tip, itself a signed commit. +History keeps the old encoding as archive; the tip of a meta-ref is always readable by the current binary. + +*I want to know what happens when someone force-pushes a branch my effect watches.* +Trigger sets are monotone and entry-only: a commit leaving the set retracts nothing, because results are immutable history — it simply stops being an obligation. + +*I want to push to a checked-out branch on my local server.* +Local serving sets `receive.denyCurrentBranch=updateInstead`, so accepted pushes also update the working tree. +Known edge: `updateInstead` fails on a dirty worktree, so branch pushes are not perfectly identical to hosted mode — metadata pushes are, because `refs/meta/*` never touches a worktree. + +*I want to script against the forge from my editor or CI.* +`git-ents-server` is a library first; the serve command and binaries are thin wrappers, and the primitives (`git store`, `git anchor`, `git comment`, `git effect`, `git toolchain`) are plumbing over the same library. + +''''' + +== The one-sentence answers + +*Why refs for everything?* +Because a ref is simultaneously the unit of storage, sync, authorization, and history — one abstraction doing four jobs. + +*Why signed commits instead of push certificates?* +Because a commit signature replicates with the repo and verifies in every clone forever, while a push cert evaporates at the transport layer. + +*Why can anyone run CI but nobody fake it?* +Because "official" is a refname rule on canonical results refs, not a runtime property of a blessed machine. + +*Why is there no workflow language?* +Because pipelines are set algebra over commits and results, and composition happens in the repository, not in a trigger DSL — stated in the abstractions as a bet, deliberately. + +*Why does deleting the server lose nothing but custody?* +Because every clone contains the forge; the server was only ever the place where the gate's verdict had teeth.