git ents: docs: rewrite development plan around token-over-HTTPS auth

commit d936458

docs: rewrite development plan around token-over-HTTPS auth

Authentication is a bearer token over HTTPS via git-http-backend, matching GitHub’s HTTPS model. Commit signing is demoted to optional provenance and a gh-style credential helper is an optional later phase.

Assisted-by: Claude:claude-opus-4-8

Joseph D. Carpinelli · 2 months ago

tree browse at HEAD · parents: 755e0b4 · comment on this commit

Reviews

No reviews of this commit yet — record a verdict below.

Start a review

PROMPT.adoc @@ -8,111 +8,83 @@ == Overview -The `git-ents` project is a distributed git hosting service backed by a content-addressed object store and a linearizable ref store. -The architecture separates immutable object replication (eventually consistent, trivially parallel) from mutable ref updates (serialized through a consensus layer), mirroring the metadata/data plane split found in production distributed storage systems. -The initial target is a small fixed cluster tolerating node and disk failure within a single failure domain. -Fly.io provides compute and persistent volumes during development; the architecture imposes no Fly-specific dependencies. +The `git-ents` project is a single-node git hosting service reached over HTTPS. +A thin gateway delegates all git protocol work to git's own `git http-backend` CGI, which implements the entire smart-HTTP protocol. +Authentication is a bearer token carried in the HTTP `Authorization` header, exactly as GitHub authenticates HTTPS git operations: the username is ignored and the password is the token. +TLS, the public IP, and the custom domain are provided by Fly.io's HTTP proxy, so the gateway holds no certificates and opens no privileged ports. == Design Principles -* Git objects are immutable and content-addressed, so they never require coordination; replication is anti-entropy. -* Refs are the only mutable shared state, and all correctness guarantees reduce to linearizable ref advancement. -* Objects reachable from a committed ref must be durable before the ref update commits; this ordering invariant is non-negotiable. -* User account state is git-backed (stored in `_meta/users.git`) and cache-served at auth time; consensus-driven cache invalidation eliminates the bootstrapping circularity. -* Extension seams for Raft, multi-node membership, and CI webhooks are defined from day one, even when backed by trivial single-node implementations. +* Authentication is a token over HTTPS, nothing more. Git sends it as HTTP Basic auth; the gateway compares it against a configured secret. No custom auth protocol, no key exchange, no signing in the hot path. +* The token gates writes; reads may be gated too or left open, as chosen. This is the same model `git push https://...` already speaks, so stock git works with no client plugin. +* Client ergonomics are separable from server auth. Stock git caches the token in the OS keychain; a credential helper (the `gh auth setup-git` pattern) is an optional later convenience, not part of the auth design. +* Commit signing is provenance, not authentication. If wanted, it is an optional "verified" check, never the gate that admits a push. +* Git objects are immutable and content-addressed; git serializes ref updates itself on a single node. No discovery, no consensus, no CI. == Repository Layout ---- git-ents/ - src/ - main.rs # entrypoint, config, startup - ssh.rs # russh server, command dispatch - namespace.rs # path resolution: (username, repo) -> /data/repos/... - ref_store.rs # CAS trait + single-node impl - object_store.rs # thin gix wrapper - auth.rs # key cache, reload on ref update - Dockerfile - fly.toml + crates/ + git-ents-server/ # the HTTPS gateway binary + src/ + main.rs # entrypoint, config, startup + http.rs # token gate + git-http-backend dispatch + Dockerfile + .config/ + fly.toml ---- -== Phase 1: Single-Node MVP +== Phase 1: HTTPS Git Delegation -Goal: both `git push` and `git clone` work over SSH against a persistent volume on Fly.io. - -=== Tasks - -. Scaffold a Rust project, pulling in `tokio`, `russh`, `gix`, and `tracing` as dependencies. -. Implement the `namespace.rs` module to resolve a `(username, repo)` pair to `/data/repos/{username}/{repo}.git`, rejecting unknown paths. -. Implement the `object_store.rs` module as a thin `gix` wrapper for initializing bare repos and accessing objects. -. Implement the `ref_store.rs` module: define the `RefStore` trait exposing `compare_and_swap(ref_name, old_oid, new_oid)`, with a single-node impl that delegates directly to git. -. Implement the `ssh.rs` module to accept connections via `russh`, authenticate by public key, and dispatch `git-upload-pack` and `git-receive-pack` as subprocesses, routing all ref updates through `ref_store`. -. Implement the `auth.rs` module to load public keys from the `_meta/users.git` repo at startup into an in-memory cache and expose a reload interface. -. Write a `Dockerfile` and `fly.toml`, mounting a persistent volume at `/data` and exposing port 22 (TCP). -. Deploy to Fly.io and verify that `git clone`, `git push`, and a re-clone round-trip all succeed. - -=== Acceptance Criteria - -* Running `git push ssh://git-ents.fly.dev/alice/repo.git main` succeeds. -* Objects persist across process restart (the volume survives redeploy). -* A push from an unknown key is rejected before any ref is updated. - -== Phase 2: User Accounts in Git - -Goal: user public keys are stored and versioned in `_meta/users.git`, and the auth cache invalidates on ref update. +Goal: `git clone` and `git push` work over HTTPS against a persistent volume on Fly.io, authenticated by a bearer token, with the gateway delegating all protocol work to `git http-backend`. === Tasks -. Initialize the `/data/repos/_meta/users.git` bare repo on first startup. -. Define the key blob layout: one file per user at `keys/{username}.pub` in the tree. -. On startup, read the HEAD of `_meta/users.git` and warm the auth cache. -. Wire the reference-transaction hook on `_meta/users.git` to call `auth::reload()` after each committed ref update. -. Provide an admin path (initially direct volume access or a privileged SSH command) to bootstrap the first user key. +. Add a minimal synchronous HTTP server dependency (e.g. `tiny_http`); no async runtime is required. +. Implement `main.rs` to read configuration from the environment (the auth token `ACCESS_TOKEN`), bind the HTTP listener, and start the server. +. Implement `http.rs` to, for every request: +.. derive the repository path from the URL and reject any path containing `..` or escaping `/data/repos`; +.. require the bearer token for writes (and for reads, if reads are private): parse the `Authorization` header as HTTP Basic and compare the password field against `ACCESS_TOKEN`, returning `401` with a `WWW-Authenticate: Basic` challenge on mismatch; +.. auto-initialize the bare repo (with `http.receivepack=true`) if absent on a write; +.. invoke `git http-backend` as a CGI child: set `GIT_PROJECT_ROOT=/data/repos`, `GIT_HTTP_EXPORT_ALL=1`, `PATH_INFO`, `REQUEST_METHOD`, `QUERY_STRING`, and `CONTENT_TYPE`; pipe the request body to its stdin; and stream its CGI stdout (status + headers + body) back to the client. +. Write a `Dockerfile` whose runtime image installs `git`, mounts the persistent volume at `/data`, and listens on the `internal_port` from `fly.toml`. +. Confirm `fly.toml` keeps the `[http_service]` with `force_https` and the volume mount; no dedicated IP and no TCP service are needed. +. Deploy to Fly.io and verify that `git clone`, `git push`, and a re-clone round-trip all succeed when given the token, and are rejected without it. === Acceptance Criteria -* Adding a key to `_meta/users.git` and pushing causes the server to accept connections from that key within one reload cycle. -* Removing a key causes subsequent auth attempts from that key to fail. -* A server restart re-derives auth state entirely from git; no separate key file is consulted. +* `git push https://x:<token>@<domain>/repo.git main` reaches `git http-backend` and updates the repo. +* A push or private read without a valid token is rejected with `401`. +* Objects persist across process restart; the volume survives redeploy. +* Cloning a repo created by a previous push returns the same objects. -== Phase 3: Multi-Node Consensus +== Phase 2: Custom Domain -Goal: a three-node cluster where ref updates require quorum and object replication precedes ref commit. +Goal: the service is reachable over HTTPS at a custom domain on Fly.io. === Tasks -. Replace the single-node `RefStore` impl with an `openraft`-backed impl whose state machine applies `compare_and_swap` entries from the log. -. Add node membership config to `fly.toml`: three Fly Machines in one region with stable private IPv6 addresses via 6PN. -. Provision one persistent volume per Machine. -. Implement object pre-flight: before proposing a ref CAS to Raft, verify all referenced objects are present on a quorum of nodes, replicating missing objects peer-to-peer via pack transfer. -. Forward ref update proposals from the SSH layer to the current Raft leader; non-leader nodes redirect the client. -. Implement cross-node cache invalidation in `auth.rs`: on Raft log apply for `_meta/users.git` head advancement, all nodes reload independently from their local object store. +. Add a Fly certificate for the hostname (`fly certs add <hostname>`). +. Create the DNS records Fly reports (`CNAME` to the app, or `A`/`AAAA` to the shared IPs) and the `_acme-challenge` record for validation. +. Verify `git clone https://<domain>/repo.git` resolves, validates TLS, and connects. === Acceptance Criteria -* Killing one of three nodes leaves pushes and clones functioning on the remaining two. -* Restarting the killed node causes it to catch up via Raft log replay without manual intervention. -* A push rejected by quorum (e.g. a non-fast-forward without force) is rejected consistently across all nodes. -* No dangling ref is ever committed: if object pre-flight fails, the push is rejected before Raft sees the proposal. +* Cloning and pushing over `https://<domain>/...` succeed end to end. +* Fly issues and renews the certificate automatically; the gateway manages no certificates. -== Phase 4: CI Integration +== Phase 3: Credential-Helper Ergonomics (Optional) -Goal: push events trigger CI via sprites.dev webhooks. +Goal: users authenticate once instead of pasting the token on every operation, following the `gh auth setup-git` pattern. === Tasks -. Add an internal-only HTTP server for webhook dispatch. -. On successful ref update (post-Raft commit on the leader), emit a push event to the configured sprites.dev endpoint. -. Include the ref name, old OID, new OID, and pusher identity in the payload. +. Add a `git-ents auth login` subcommand that accepts a token and stores it (OS keychain or `git credential store`). +. Register the binary as a git credential helper for the service host, so HTTPS git operations retrieve the token automatically. +. Document the one-time setup; confirm subsequent `git clone`/`push` need no interactive credential entry. === Acceptance Criteria -* A push to `main` triggers a sprites.dev pipeline run. -* A rejected push (failed CAS) does not emit a webhook. - -== Future Work - -* Web UI for read-only repository browsing, served from any node. -* Repo creation and deletion via SSH commands or an HTTP API. -* Distributed GC: safe reclamation of objects unreachable from any committed ref, requiring cluster-wide reachability agreement. -* Read replicas in additional Fly regions for object serving (reducing clone latency), with consensus remaining single-region. +* After a single `git-ents auth login`, `git clone` and `git push` over HTTPS succeed without prompting. +* Stock git (with the OS keychain) still works for users who skip the helper.