nova

Project

Roadmap

Phase 0 v3 — Specifications (complete)

Lock the protocol and contracts as documents before any production code is written. The original Phase 0 was completed, then revised in a v2 consistency pass after design audits identified contradictions and gaps (see docs/REVIEW_2026_05_09.md), then revised again in v3 after a second round of architectural review identified document drift and missing classification (see docs/REVIEW_2026_05_19.md).

Original Phase 0 deliverables (still complete, now updated to v2): - [x] Repository skeleton - [x] OpenAPI specification (docs/specs/openapi.yaml) - [x] Signed URL format (docs/specs/SIGNED_URL_FORMAT.md) — v2 structured revocation - [x] Data model (docs/specs/DATA_MODEL.sql) — v2 split keys, derivatives-as-blobs, 5-state liveness - [x] Encryption envelope (docs/specs/ENCRYPTION_ENVELOPE.md) — v2 master-key versioning, narrowed GDPR claim - [x] Federation protocol (docs/specs/FEDERATION_PROTOCOL.md) — v2 mTLS, donor-to-donor repair - [x] IPFS daemon hardening (docs/specs/KUBO_HARDENING.md) - [x] Product module interface (docs/specs/PRODUCT_MODULE_INTERFACE.md) — v2 split AnalyzeUpload + OnCommitted, format conversion - [x] Healing protocol (docs/specs/HEALING_PROTOCOL.md) — v2 5-state liveness, configurable R, source-enforcement - [x] Orchestrator resilience simulation (simulations/orchestrator_resilience.py) - [x] Threat model (docs/THREAT_MODEL.md) - [x] Privacy audit (docs/PRIVACY_AUDIT.md) - [x] Operator checklist (docs/legal/OPERATOR_CHECKLIST.md) — v2 narrower [REQUIRED] - [x] ToS template (docs/legal/TOS_TEMPLATE.md) - [x] Takedown procedure (docs/legal/DMCA_PROCEDURE.md) — v2 quarantine-first default - [x] Volunteer deployment guidance (docs/VOLUNTEER_DEPLOYMENT_GUIDANCE.md) - [x] nginx reference config (nginx/nova.conf.example) - [x] nginx walkthrough (docs/recipes/NGINX_REFERENCE.md) - [x] Phase 0 dependency-only docker-compose.yml - [x] Cloudflare recipe (docs/recipes/CLOUDFLARE.md) — v2 reframed as optional

v2 additions (new specs from the design audit): - [x] IPFS import rules (docs/specs/IPFS_IMPORT_RULES.md) — deterministic CIDs for proof-readiness - [x] Integrity audit (docs/specs/INTEGRITY_AUDIT.md) — Phase 1 local fixity - [x] Possession audit (docs/specs/POSSESSION_AUDIT.md) — Phase 2 donor spot-checks - [x] Severe content procedure (docs/legal/SEVERE_CONTENT_PROCEDURE.md) — Phase 4 implementation - [x] Review summary (docs/REVIEW_2026_05_09.md) — what changed and why

v3 additions (architectural classification and slow-attrition handling): - [x] Architecture decisions (docs/specs/ARCHITECTURE_DECISIONS.md) — three-tier classification (protocol-enforced / operator-tunable / operator freedom) - [x] THREAT_MODEL.md boundary ⑤ corrected (HTTPS-over-Nebula, not libp2p); explicit out-of-scope rationale for threshold cryptography, end-to-end encryption, PSI moderation, S3 API, and multi-master HA; new residual-risk entry for slow attrition - [x] HEALING_PROTOCOL.md slow-attrition detection (federation.shrinking webhook + capacity_runway_floor_days) - [x] Operator recipes (docs/recipes/AUTOMATED_ONBOARDING.md, KEY_ESCROW.md, COLD_STANDBY.md) — patterns operators build on top of the protocol - [x] Simulations: sybil_concentration.py, long_tail_churn.py, key_rotation_load.py - [x] Review summary (docs/REVIEW_2026_05_19.md) — what changed and why

Phase 1 — Single-node MVP (complete at v0.1.0-rc1)

Standalone coordinator with embedded hardened IPFS daemon, Postgres, nginx + certbot, signed-URL HMAC, per-blob encryption on by default, on-the-fly image transforms, drag-and-drop upload widget. Exports pkg/coordinator as a public Go library package (with the pkg/coordinator/product subpackage at v0.x.y until external adapter authors are real consumers in Phase 4).

v2 promotions into Phase 1: - Master-key rotation tooling (novactl keys rotate-master), was previously deferred to Phase 5. - Local integrity audits running in the background; admin UI surfaces recent failures. - Deterministic IPFS import per IPFS_IMPORT_RULES.md; blob manifests + blob_blocks recorded for every upload. - Quarantine-first DMCA flow with scheduled tombstone job. - Manual operator path for severe-content quarantine + legal-hold via novactl moderation quarantine ... --legal-hold.

v3.1 amendment: - pkg/node is not exported in Phase 1. The donor binary ships in Phase 2; freezing a public Go interface before any caller exercises it would produce immediate semver churn at Phase 2. Phase 1 keeps node-side types under internal/node/ and promotes to pkg/node in Phase 2 alongside cmd/node. See docs/REVIEW_2026_05_25.md. - The envelope codec ships v1 (single-shot XChaCha20-Poly1305), but the implementation uses a versioned Codec interface so v2 (streaming-AEAD, see Phase 2) drops in without disturbing v1 paths. Blob metadata exposes envelope_version so consumers can branch.

Phase 1 — Progress (complete)

Each milestone is a tagged annotated commit; the canonical walking-skeleton breakdown lives in docs/superpowers/specs/phase1/2026-05-25-phase1-single-node-mvp-design.md § "Walking-skeleton milestone breakdown".

  • [x] M1 foundation (m1-foundation) — repo bones, schema + migrations (DATA_MODEL.sql v2 → internal/db/migrations/0001..0005), embedded Kubo skeleton, config loader, Makefile + CI.
  • [x] M2 envelope + IPFS (m2-envelope-ipfs) — XChaCha20-Poly1305 v1 codec, deterministic IPFS import per IPFS_IMPORT_RULES.md, master-key wrap/unwrap, job queue.
  • [x] M3 storage + read API (m3-storage-read-api) — Resolve
  • OpenBytes, /blob/{cid} GET/HEAD + .json, in-process rate-limit middleware, /health.
  • [x] M4 upload pipeline (m4-upload-pipeline) — tus + multipart, the AnalyzeUpload/OnCommitted product seam, encryption-at-rest path, data_encryption_keys lifecycle, T1.20 public-uploads floor.
  • [x] M5 image transforms (m5-image-transforms) — nova-image Product impl, govips wrapper with megapixel + concurrency bounds, /i/* single-flight serve, derivative pre-warm, PDQ pass-through scanner.
  • [x] M6 auth (m6-auth) — argon2id passwords + timing equalizer, EdDSA local issuer + JWKS, rotating refresh tokens with reuse detection, external-OIDC verify-only adapter with resilient discovery, bearer middleware, per-IP login limiter, T1.19 + signing-key floors, novactl auth login|whoami|logout.
  • [x] M6.1 keystore hardening (m6.1-keystore-hardening) — env → _FILE/run/secrets/master-key-<label> resolver chain with ACTIVE/FILE pseudo-label filtering; THREAT_MODEL.md boundary ③ amended.
  • [x] M6.2 audit remediation (m6.2-audit-remediation) — spec-drift reconciliation across persistent docs; verified security hardening (rate-limiter LRU + sweep, trusted-proxy XFF enforcement, login-failure log unification, refresh-family revocation correctness, master-key source logging, ctx-aware Unwrap, multipart LimitReader); refresh-token GC partial-index alignment; /readyz with DB + Kubo + OIDC checks; structured coordinator startup log. See docs/REVIEW_2026_05_31.md.

Phase 1 — Deferred / Future-milestone slots

These deliverables remain from the Phase 1 v3.1 commitment and are assigned to the slots already specified in docs/superpowers/specs/phase1/2026-05-25-phase1-single-node-mvp-design.md § "Walking-skeleton milestone breakdown". Implementation lands in the named milestone; no work is in scope for M6.2 beyond naming the slots here.

Slot Deliverable
M7 Signed-URL HMAC verifier (internal/auth/signedurl) gating /blob + /i/*; signing_keys rotation via /api/v1/admin/keys/rotate-signing with grace window; structured (kind, value) revocation via /api/v1/admin/signed-urls/revoke; server-side minting via /api/v1/admin/signed-urls/sign + novactl signed-url sign. Implemented (tag m7-signed-urls).
M8 In-process integrity-audit scheduler (internal/audit/integrity) running the seven audit kinds on per-kind cadences (no jobs.Queue; resumes from natural cadence on restart); /api/v1/admin/audits/integrity paginated listing; failure surfacing via warn logs + integrity_audits rows + a FailureSink seam (nova_integrity_audit_failures_total metric and integrity.audit_failed webhook deferred); monthly-partition create-ahead + retention pruning. Implemented (tag m8-integrity-audit). Design: docs/superpowers/specs/phase1/2026-06-02-phase1-m8-integrity-audit-scheduler-design.md.
M9 DMCA quarantine + ≈1-minute in-process tombstone sweep + counter-notice; severe-content manual quarantine with --legal-hold + operator-only clear-legal-hold (enforced by no_shred_under_legal_hold CHECK); novactl moderation quarantine/takedown/restore/clear-legal-hold/list; operator-curated CID blocklist; /api/v1/admin/moderation/* + /api/v1/admin/audit-log; public POST /legal/dmca intake; M7 audit backfill; audit_log partition create-ahead. Implemented (tag m9-moderation). Design: docs/superpowers/specs/phase1/2026-06-02-phase1-m9-moderation-design.md. Plan: docs/superpowers/plans/phase1/2026-06-02-phase1-m9-moderation.md. Deferrals: perceptual/visual blocklist → Phase 3; NCMEC CyberTipline + legal-hold-clear admin SPA → Phase 4; repeat-infringer auto-suspension → later (no account-state column); Kubo-pinset/DB orphan reconciliation → Phase-5 hardening.
M10 Master-key rotation (novactl keys rotate-master --from v1 --to v2, GET /api/v1/admin/keys/rotation-status); parallel re-wrap worker (default 4 goroutines, 256-row batches, 50 ms pace); one atomic version-guarded UPDATE per DEK (no per-row rotating state; rotating marks the source master_key_versions row); signing keys re-wrapped (state IN ('active','retired')); stalled-rotation /readyz degradation + novactl keys status; ResumeIfRotating crash recovery; audit master_key.rotation_started/completed/resumed. Implemented (tag m10-master-key-rotation). Design: docs/superpowers/specs/phase1/2026-06-03-phase1-m10-master-key-rotation-design.md. Plan: docs/superpowers/plans/phase1/2026-06-03-phase1-m10-master-key-rotation.md. Deferrals: runtime/no-restart activation → not planned; master-key generator helper (novactl keys gen-master) → later; cross-node rotation propagation → Phase 2; novactl keys rotate-signing wrapper → optional.
M11 Admin SPA (web/admin/): hermetic React + Vite (self-hosted IBM Plex latin, no CDN; CI hermetic-spa gate on the bundle); two auth drivers behind one provider — local-issuer password→token with silent refresh, and external-OIDC authorization-code + PKCE (issuer added to the SPA CSP connect-src); operator screens for blob list/view/soft-delete, moderation queue + DMCA + blocklist, integrity-audit failures, key rotation (master + signing), read-only jobs view, audit log. Backend slice: a neutral internal/lifecycle.TombstoneTree primitive (extracted from M9 — crypto-shred lives in one place) + owner soft-delete + in-process grace sweep (blob.soft_deleted/blob.tombstoned audit, distinct from dmca.*); mounted GET/DELETE /api/v1/blobs/{cid} (the M6-deferred owner routes); GET /api/v1/admin/blobs + read-only GET /api/v1/admin/jobs; coordinator-served /admin/* static (strict CSP + SPA fallback) gated by NOVA_ADMIN_DIST_DIR; migration 0009 (blobs.soft_deleted_at). Implemented (tag m11-admin-spa). Design: docs/superpowers/specs/phase1/2026-06-04-phase1-m11-admin-spa-design.md. Plan: docs/superpowers/plans/phase1/2026-06-04-phase1-m11-admin-spa.md. Deferrals: jobs retry → fast-follow; blob PATCH / /api/v1/images/{cid} / collections / perceptual search → later; clear-legal-hold UI → Phase 4; upload widget → M12; production nginx two-vhost split + Docker → M13.
M12 Drag-and-drop upload widget (web/widget/): hermetic Vite library-mode IIFE bundle exposing the global NovaUploadWidget (single-<script> embed, stable entry filename, CSS injected at runtime); @uppy/core+drag-drop+tus+status-bar (3.x; the maintained @uppy/status-bar, not the deprecated @uppy/progress-bar); the Nova-aware finalize orchestrator (tus upload-success is transport-only → POST .../finalizeUploadResult); getToken() resolved per request (survives the M6 15-min access TTL; null ⇒ public-uploads floor); mount/mountAll + a data-nova-upload-widget auto-bootstrap with a WeakMap double-mount guard. Backend slice: a feature-gated coordinator /widget/* static seam (internal/api/handlers/widget_static.go, strict CSP, no SPA fallback) gated by NOVA_WIDGET_DIST_DIR; web/widget re-added to the root workspaces; a hermetic-widget CI gate that greps both the HTML/CSS and the inlined JS bundle for external-origin patterns. Implemented (tag m12-upload-widget). Design: docs/superpowers/specs/phase1/2026-06-07-phase1-m12-upload-widget-design.md. Plan: docs/superpowers/plans/phase1/2026-06-07-phase1-m12-upload-widget.md. Deferrals: cross-origin embedding + first-class CORS → operator nginx / later milestone; production nginx two-vhost split + Docker → M13; rich Uppy Dashboard / hosted upload app → later; tus-result preset URLs → later backend change.
M13 First-run setup wizard + Docker production. Shared UI-agnostic core (internal/setup/: answers + per-step validation reusing the config.validate floors, CSPRNG key material, operator.yaml/nova.conf render, per-mode TLS, atomic sentinel-last commit) drives both a hermetic React+Vite web wizard (web/setup/; hermetic-spa gate) and a headless novactl setup --interactive | --config-file. Setup mode is folded into the coordinator boot path (coordinator.RunSetupServer, sentinel-gated in cmd/coordinator) — a reduced boot mounting only the loopback-bound /setup/* seam (internal/api/handlers/setup.go) until .bootstrap-complete is written; cmd/setup-wizard is a thin alias. operator.yaml is now wired into cmd/coordinator as the canonical non-secret config source, with the existing NOVA_* env reads preserved as overrides. The two-vhost split is nginx-only (templated nova.conf from internal/setup/templates/nova.conf.tmpl: public_host serves /blob·/i·/legal·/health·/api/v1/uploads\|blobs\|images·/widget·ACL'd /metrics, /fed→404, default→404; admin_host serves /admin·/api/v1/admin·/api/v1/auth·/api/v1/users/me·/health, /fed→404, default→404); the coordinator keeps its single mux. TLS modes: dev-self-signed (auto CA+leaf), static (operator PEM), http-01 (certbot, prod profile, best-effort renewal scaffold — initial issuance is operator-handoff); dns-01/onion render config + print operator-handoff instructions. Docker: multi-stage Debian-slim/glibc image (non-root via gosu drop in docker/init/entrypoint.sh), docker/docker-compose.yml with setup + prod profiles; published ports 8442:80, 8443:443, 127.0.0.1:8445:8445, wizard on 127.0.0.1:8444; secrets (master-key-v1, swarm.key, oidc-signing-key) generated by the wizard into the nova-secrets volume. The web wizard configures the local issuer (default); external-OIDC is configured via the headless novactl setup --config-file / manual operator.yaml path (auth_mode: external + issuer_url/client_id), not the web stepper. Integration test proves the two-vhost split + the setup→normal sentinel flip. Implemented (tag m13-setup-wizard). Design: docs/superpowers/specs/phase1/2026-06-08-phase1-m13-setup-wizard-design.md. Plan: docs/superpowers/plans/phase1/2026-06-08-phase1-m13-setup-wizard.md. Deferrals: exhaustive container hardening + release signing + CI e2e smoke + screenshot quickstart → M14 / Phase 5; full dns-01/onion automation → later; certbot full deploy-hook/reload + initial ACME issuance → M14; operator.yaml decode of the M7–M12 tuning knobs → later (those stay env-only); in-process uid-0 floor → later (non-root is enforced via the container today); web-wizard external-OIDC → the headless/manual path.
M14 Polish, security housecleaning, CI e2e smoke, release candidate. CI repairs: golangci-lint migrated to v2 (v2 config + golangci-lint-action@v8 + Go version derived from go.mod via go-version-file); the dead schema-drift diff replaced by a migration-immutability check (internal/db/migrations/MANIFEST.sha256 + scripts/check-migrations-frozen.sh, blocking CI job migrations-frozen; the 0001_init.sql header corrected — DATA_MODEL.sql is the annotated living reference, the migrations are authoritative). Dependabot triage: all 25 alerts / 10 advisories assessed, none enabling compromise of a production deployment — the single runtime-reachable item (quic-go) is a memory-exhaustion DoS (full triage table in the design doc); the two runtime-reachable patches landed — quic-go v0.59.1 (CVE-2026-40898, DoS via the embedded Kubo QUIC stack) and otlptracehttp v1.43.0 (CVE-2026-39882) — and every npm advisory (all dev-toolchain-only) cleared by the toolchain jump: Vite 8.0.16 + Vitest 4.1.8 + plugin-react 6 + jsdom 29 across all three SPAs (the Node-16-era pins are gone), Node 22 in .nvmrc/engines/CI/docker/Dockerfile, a root npm overrides pinning @uppy/core's transitive nanoid ≥5.1.6. Ongoing currency: .github/dependabot.yml (gomod/npm/github-actions, weekly, grouped) + the CONTRIBUTING.md "Toolchain currency" policy. Full-stack e2e smoke (scripts/smoke.sh, wired as a blocking CI smoke job): image build → headless novactl setup --config-file → prod profile boot → anonymous upload → byte-identical /blob read → /i/…/w320.png transform → operator login + DELETE → 404/410. The M13 certbot deferral closed: http-01 initial issuance is automated (docker/certbot/certbot-loop.sh issues on first boot and deploys key-first/cert-atomic into /etc/nova/tls; a self-signed placeholder breaks the nginx⇄certbot bootstrap deadlock) and renewals hot-reload nginx (docker/nginx/cert-watch.sh watches the cert hash and SIGHUPs nginx); the new nova-letsencrypt volume persists the ACME account/lineage; dns-01/onion stay operator-handoff. Container hardening floors: healthchecks on all five compose services; read-only rootfs + tmpfs on coordinator/nginx/nginx-setup (postgres pre-existing; certbot exempted with comment); no-new-privileges + cap_drop: [ALL] + minimal commented cap_adds everywhere. docs/quickstart.md operator quickstart (screenshot capture is a pending human action — file list in docs/images/quickstart/README.md). Implemented (tags m14-polish-release + v0.1.0-rc1Phase 1 complete at release candidate). Design: docs/superpowers/specs/phase1/2026-06-09-phase1-m14-polish-release-design.md. Plan: docs/superpowers/plans/phase1/2026-06-09-phase1-m14-polish-release.md. Deferrals: release signing (sigstore/cosign + release.yml) → Phase 5 (the master plan's original position; the M13-spec line assigning it to M14 was in error and is corrected); seccomp/AppArmor profiles + dropping nginx's DAC_READ_SEARCH via entrypoint group-perm rework → Phase 5; per-service log shipping + chaos testing → Phase 5.

Phase 2 — Federation + streaming-AEAD envelope

Split coordinator from pinning-node binary. Mesh-VPN-authenticated federation, replication-factor enforcement, donor-operated nodes. Streaming-AEAD envelope (v2 wire format) so encrypted blobs support HTTP Range requests, CDN partial-object caching, and modern web media playback expectations.

Storage/read architecture redirect (P2-M2.1, 2026-06-20). Phase 2's storage model was redirected: the operator is not required to retain the full corpus. Donor-backed reads, origin pruning, and reputation-based best-link selection (VPS-primary / residential-fallback) are the Phase-2 target — the donor replica set is the durable substrate and the operator keeps a bounded cache. Storage and read fan-out are independent axes: donors are donor-blind, so the operator stays the decrypt/serve point and fan-out to many DAU scales via the operator hot cache and/or optional CDN, not via donors serving users. This amends the federation design's earlier "replicate, don't migrate" stance; binding M3/M4/M5 constraints live in that design's "Storage/read architecture (P2-M2.1 amendment)" section.

Phase 2 — P2-M0.x remediation track (operator-UX / privacy / pitfall fixes before additive Phase 2 work)

Slot Deliverable
M0.1 Correctness fixes (admin SPA nav double-prefix, compose name: pin, plumb NOVA_PUBLIC_UPLOADS/NOVA_TOS_URL). Implemented (tag p2-m0.1-correctness-fixes).
M0.2 paranoid reframed as a default-off warn-not-force preset over individually addressable constituents (record_source_ip, source_ip_retention_days, public_ipfs_dht, webhooks); startup warning replaces forced override when a protective default is relaxed. Implemented (tag p2-m0.2-privacy-posture). Design: docs/superpowers/specs/phase2/2026-06-13-m0.2-privacy-posture-model-design.md.
M0.3 CORS + upload-credential hardening: scoped revocable upload tokens (nova_ut_…), per-session concurrency/file-count limits, CORS allowlist on upload routes. Implemented (tag p2-m0.3-offorigin-widget). Design: docs/superpowers/specs/phase2/2026-06-13-m0.3-offorigin-widget-design.md.
M0.4 Runtime config backend: operator.yaml read/update admin API (GET/PATCH/PUT /api/v1/admin/config), live hot-reload for live-class fields, novactl config get/set/apply. Implemented (tag p2-m0.4-config-backend). Design: docs/superpowers/specs/phase2/2026-06-14-m0.4-config-backend-design.md. Plan: docs/superpowers/plans/phase2/2026-06-14-m0.4-config-backend.md.
M0.5 Setup-wizard redesign: consequence copy + learn-this/abstract-away jargon info-buttons + tri-state paranoid delineation + additive Answers constituents. Implemented (tag p2-m0.5-wizard-redesign). Design: docs/superpowers/specs/phase2/2026-06-14-m0.5-setup-wizard-redesign-design.md. Plan: docs/superpowers/plans/phase2/2026-06-14-m0.5-setup-wizard-redesign.md.
M0.6 Admin Settings screen (web/admin): operator-only /settings route driving the M0.4 config API — curated, explained controls over a typed draft (tri-state webhook-aware ParanoidSection; CORS enable + origin add/remove with new URL().origin normalization + enabled-with-empty-list guard; live upload limits; public-uploads/ToS with a T1.20 local guard), minimal JSON-Merge-Patch save with If-Match optimistic concurrency (200 reseed + restart_required banner / 409 conflict-reload / 422 inline), plus a collapsible read-only full-surface effective-config viewer with live/restart/env badges driven by the GET fields metadata. auth.paranoid is derived as the AND of the children and webhooks-empty, so no save trips an ApplyPrivacyPreset startup WARN (the runtime webhook case the M0.5 first-run never hit); the screen surfaces privacy_warnings and resolves editable-constituent drift on save. Ported M0.5's InfoTerm/ConsequenceNote/ParanoidSection/glossary into web/admin (no shared package); badges prefer live fields metadata with a registry fallback + no-drift test. No backend change. Implemented (tag p2-m0.6-settings-screen). Design: docs/superpowers/specs/phase2/2026-06-15-m0.6-settings-screen-design.md. Plan: docs/superpowers/plans/phase2/2026-06-15-m0.6-settings-screen.md. All P2-M0.x remediation items complete.

Phase 2 — Progress (additive federation track)

The main-track federation milestones (P2-M1 … P2-M10) are detailed in the master design's milestone breakdown (docs/superpowers/specs/phase2/2026-06-11-phase2-federation-design.md). P2-M0 (spec reconciliation) is merged (tag p2-m0-spec-reconciliation).

Slot Deliverable
P2-M1 Build / repo separation: a nova-node donor binary (cmd/node) whose dependency graph provably excludes operator-only code. Extracted stdlib-only internal/secret leaf (coordinator re-pointed, behavior-preserving); shared internal/federation/wire (fed/v1 messages + fail-closed capability negotiation + canonical Ed25519 repair-token claim/Verifyno mint, no replay, those are M4); donor-only internal/node/{config,bandwidth,state,agent,transfer,audit} — in M1 only the authoritative daily bandwidth token-bucket (D11) has real logic, the rest are interface seams (transport→M2, sync/state→M3, transfer/mint→M4, audit→M6); nova-node --config/--validate/--healthcheck with a fail-fast loopback health server. Load-bearing gate: scripts/check_node_deps.sh (go list -deps ./cmd/node, deny-by-default over all non-stdlib deps), wired as blocking CI donor-deps-boundary and demonstrated red against an injected operator import. Split Dockerfiles (docker/Dockerfilecoordinator.Dockerfile + a minimal 8.97 MB distroless-static CGO-off docker/node.Dockerfile; scripts/check_node_image.sh forbidden-inventory scan); deploy/donor/{compose.yaml,node.yaml.example} (Nebula sidecar, no published ports, read_only/cap_drop: ALL/no-new-privileges); CI SBOM (donor-build) + cosign keyless signing + provenance attested to the pushed digest (donor-sbom-sign, trusted-ref-gated, never on PRs); docs/quickstart/donor.md release-trust stub. node.yaml references all secret material by *_path (shallow validation — no PEM parse). No live federation; no schema/migration (migrations-frozen stays green). Implemented (tag p2-m1-build-repo-separation). Design: docs/superpowers/specs/phase2/2026-06-15-phase2-m1-build-repo-separation-design.md. Plan: docs/superpowers/plans/phase2/2026-06-15-phase2-m1-build-repo-separation.md. Deferrals: live mTLS-over-Nebula transport + registration + capability handshake → M2; pin_changes log + assignment sync + snapshot recovery → M3; coordinator-as-source + streaming transfer + deterministic re-import + Ed25519 token mint + donor↔donor repair → M4; 5-state liveness + healing + blob_replication_state → M5; possession audits + reputation → M6; volunteer digest-pin/cosign verify walkthrough + revocation/provider-loss drills → M7.
P2-M2 Identity, registration, capability negotiation — the first live federation milestone (the M1 seams become real). A standalone internal/federation/coordinator mTLS server runs as a second listener (Nebula-interface-bound, RequireAndVerifyClientCert against the operator federation CA; never on the public/admin mux — verified /fed/v1/*→404 there) serving POST /fed/v1/register + POST /fed/v1/heartbeat. Identity derives only from the verified leaf cert — DER fingerprint (sha256:<hex>, not SPKI) + a stable node_id UUID from the nova://node/<uuid> URI SAN; self-asserted JSON is never trusted as identity (the request federation_cert_fingerprint is cross-checked, nebula_cert_fingerprint is non-identity metadata). Fail-closed protocol + capability negotiation (no overlap → 400 incompatible_protocol/missing_capability, never a refused 201; M2 advertises/requires an honest empty capability set — the machinery ships, the future cap ids stay unadvertised until M3/M4/M6). Handler-level authorization: revoked → 403 node_revoked, presented-fp ≠ stored-fp → 403 fingerprint_mismatch (rotation cutover / stale cert), heartbeat from an unknown node → 403 registration_required. New rows land trust_state='probationary', status='active'; heartbeat records last_seen_at/last_free_bytes/last_stored_bytes and returns config_updates timers + current_epoch:0 + empty repair_token_public_key (the channel exists; the signer is M4). Shared stdlib-only internal/federation/transport (mTLS tls.Config builders + cert→identity); pure-Go Ed25519 internal/federation/ca (federation X.509 CA + coordinator server cert + donor client certs). novactl node ca-init/issue/nebula-template (local file ops; explicit two-trust-root naming federation-* vs nebula-*, no nebula-cert shell-out, no Nebula Go dep) + revoke/rotate-cert/list (DB-direct via DATABASE_URL, like novactl setup). Donor: the no-op agent becomes a real register→heartbeat loop (internal/node/agent) over an mTLS HTTPClient, with an atomic-JSON RegistrationStore (internal/node/state; temp→fsync→rename→dir-fsync, 0600). cmd/coordinator runs both listeners with a runBoth loop that binds the federation listener before declaring startup success and tears down as a unit on any exit. Operator federation config block (listen_addr/nebula_interface boot guard/federation_{ca,cert,key}_path + timers). nodes-scoped migration 0011 (trust_state text+CHECK, selected_protocol, advertised/required_capabilities, client_version, cert_revoked_at/cert_rotation_started_at/cert_rotated_at, last_free_bytes/last_stored_bytes). Donor dependency boundary extended by exactly one reviewed leaf (internal/federation/transport; gate stays green, demonstrated red against an injected internal/db). e2e loopback-mTLS integration test (register→heartbeat→probationary/active→revoke-blocks), public-mux 404 test, and rotation-cutover test (old→403/new→200/node_id unchanged). Implemented (tag p2-m2-identity-registration). Design: docs/superpowers/specs/phase2/2026-06-16-phase2-m2-identity-registration-design.md. Plan: docs/superpowers/plans/phase2/2026-06-16-phase2-m2-identity-registration.md. Decisions: nodes-scoped migration amends the P2-M0/M1 "all Phase-2 DDL in M3" note (schema lands when a milestone first needs durable truth); repair-token signer + current_epoch semantics → M4; downtime cert cutover (no zero-downtime overlap). Deferrals: pin_changes log + assignment/snapshot sync + node-local cursor → M3; coordinator-as-source + streaming transfer + deterministic re-import + Ed25519 token mint + donor↔donor repair → M4; 5-state liveness sweeper + healing + blob_replication_state + D8 failure-domain/placement_weight + federation.node_revoked webhook (no dispatcher exists yet) → M5; possession audits + reputation + trust graduation → M6; volunteer release docs/digest-pin drills → M7.
P2-M2.1 Reconciliation, supply-chain hardening, and storage/read architecture redirect — a between-milestone increment, no normative protocol code or migration (migrations-frozen stays green). Docs reconciliation: README (the What is not yet wired block listed shipped M7–M14 as deferred; novactl user create → setup-wizard account creation; status → Phase 2 in progress), CONTRIBUTING (Phase 0 → active development), SECURITY (Phase 0 / no releases / threat-model-planned → current posture + threat model exists) — aspirational "durable federated storage" language deliberately kept. Versioning: docs/VERSIONING.md semver-per-milestone policy (no two builds share a version); git describe --tags --always --dirty build stamping wired via make (VERSION/GO_LDFLAGS) into a main.buildVersion fallback, NOVA_VERSION env still overrides. Supply-chain CI: govulncheck (reachability, complements the manual Dependabot triage), CodeQL (Go + JS/TS), OpenSSF Scorecard, and coordinator-image SBOM + cosign keyless signing + provenance (parity with the donor image). Durability default: important replication factor default R=3 → R=5 (warn-not-force, tunable down to 3; lower R ⇒ higher permanent-loss risk, higher R ⇒ higher donor storage burden) — loader default applied before validation via shared DefaultReplication* constants so wizard-render and loader can't drift, the HEALING_PROTOCOL R=3/R=5 contradiction resolved, warn-not-force emission specified for P2-M5 where the orchestrator consumes R. Storage/read architecture redirect (the substantive change): the operator is no longer required to retain the full corpus — donor-backed reads, origin pruning, and reputation-based best-link selection (VPS-primary / residential-fallback) become the Phase-2 storage target; the donor replica set is the durable substrate, the operator keeps a bounded cache, and storage vs. read-fan-out are independent axes (donor-blind ⇒ operator stays the decrypt/serve point; fan-out via cache/optional CDN). Amends the federation design's "replicate, don't migrate" stance and moves the former out-of-scope bullet into scope; binding M3/M4/M5 constraints + a target config surface are recorded in the federation design's new "Storage/read architecture (P2-M2.1 amendment)" section (normative-spec edits deferred to each owning milestone, P2-M0 style). Widget→public-URL loop (step 3): closed the gap where anonymous/default uploads landed in no collection → private → the widget's returned /blob/{cid} 401'd anonymously, with no collection-create path outside raw SQL. Adds novactl collection create (DB-direct; owner = sole operator or --owner) + a live-reloadable uploads.default_collection_id so uploads with no explicit/token-bound collection auto-join a configured collection — point it at a public one to make anonymous widget uploads publicly viewable without per-upload wiring; the quickstart's raw-SQL seed is replaced. TDD: handler default-apply (tus+multipart, explicit-wins), config UUID validation, and DB-direct create + owner-resolution. Implemented (tag p2-m2.1-reconciliation-storage-redirect). Design: docs/superpowers/specs/phase2/2026-06-11-phase2-federation-design.md § "Storage/read architecture (P2-M2.1 amendment)".
P2-M3 Assignment synchronization — the donor's durable, recoverable view of what it should hold, with no byte transfer and no donor ack (those are M4). Migration 0012 versions pin_assignments (immutable assignment_id handle + generation; (cid,node_id) PK kept) and adds the durable pin_changes change-log (sequence bigserial, (node_id,sequence) index, byte_size) + a singleton federation_change_log_state retention watermark. Coordinator (operator-side internal/federation/coordinator, the M2 second mTLS listener) serves GET /fed/v1/pins/changes (incremental; next_seq + monotonic current_epoch; machine-readable snapshot_required when since_seq predates the prune watermark), GET /fed/v1/pins/snapshot (cid-cursor pagination with per-node epoch consistency409 when this node's set changed past the captured epoch), and POST /fed/v1/pins/{cid}/{ack,fail} (generation-keyed conditional state machine: 204 apply / 204 idempotent replay / 409 stale_assignment / 404 unknown; method guards + body↔path cid_mismatch + fail-reason validation). The advisory-locked, single-transaction AssignPin/UnpinPin seam is the only writer of assignment state — committed bigserial sequences are commit-order-safe (a donor never advances its cursor past a row that can still commit) — and is reused by novactl pin assign|unpin|list (DB-direct operator/test seam; list prints desired assignments vs verified holders separately, the latter empty in M3) and, later, the M5 scheduler. A coordinator retention goroutine prunes pin_changes older than federation.change_log_retention (default 168h) and advances the watermark; the change-log head is GREATEST(max(sequence), pruned_through_seq) so a fully-pruned log never regresses the head into a snapshot-recovery loop. Heartbeat now returns the real current_epoch. Donor: durable atomic-JSON local state — FileStore (cursor) + FileAssignmentStore (the desired-assignment set, idempotent by (assignment_id,generation), persisted set-first/cursor-second) — and a register→immediate sync→heartbeat+pins-poll control loop over an extended Client (GetChanges/GetSnapshot; no Ack/Fail on the interface — the donor cannot ack in M3); an unknown change kind fails closed into snapshot resync; recoverSnapshot returns the old cursor on any failure (never skips unpersisted state). Donor advertises pin-change-log/v1 + snapshot/v1. Observability is a structured-slog signal set (fed.changes.*/fed.snapshot.*/fed.ack.*/fed.changelog.pruned/fed.assign.txn/node.sync.*), USE/RED-named for a P2-M7 Prometheus promotion. Tests: full coordinator handler matrix + advisory-lock commit-ordering + retention/snapshot_required + head-monotonicity, e2e loopback-mTLS changes-convergence + unpin + deterministic snapshot recovery (all asserting zero acked rows), and donor idempotent-apply / recovery / fail-closed / crash-before-cursor / 409-restart. donor-deps-boundary + migrations-frozen stay green. Implemented (tag p2-m3-assignment-sync). Design: docs/superpowers/specs/phase2/2026-06-22-phase2-m3-assignment-sync-design.md. Plan: docs/superpowers/plans/phase2/2026-06-22-phase2-m3-assignment-sync.md. Deferrals: coordinator-as-source + streaming transfer + deterministic re-import + Ed25519 token mint + donor↔donor repair + production donor fetch→verify→ack → M4; 5-state liveness sweeper + healing + blob_replication_state + placement → M5; possession audits + reputation → M6; Prometheus /metrics → M7.
P2-M4 v1 opaque replication vertical slice — the first data plane, closing the evidence loop M3 left open over the existing v1 envelope: assignment → coordinator-as-source signed grant → donor fetches bounded ciphertext → deterministic re-import + root-CID verify → local pin → persist verified state → production ack/fail. Ed25519 repair-token mint in the coordinator-only internal/federation/tokens (private key via secret.ResolveSecretfederation.repair_signing_key_path; public key delivered to donors as base64url on heartbeat; verify is the shared wire.Verify). Coordinator-as-source GET /fed/v1/blob/{cid} on the M2 mTLS listener — token verify + source_node_id/dest_node_id/cid binding (dest from the verified cert, never self-asserted) + restart-safe replay defense (reject not_before < source_boot_time + in-memory single-use jti TTL cache) + preflight size over GetBlobByteSize (state='active' only — quarantined/tombstoned/soft-deleted are not sourceable → clean 404 blob_unavailable) rejected before any body byte + io.LimitReader. /pins/changes mints a fresh per-serve Source token for each pending assign (never persisted in pin_changes; not_before clamped ≥ source_boot_time; skipped on non-positive TTL). Donor: a hardened Kubo sidecar over the loopback HTTP API (internal/node/ipfsclient) whose AddDeterministic branches exactly like EmbeddedBackend (raw block/put ≤1 MiB vs dag-pb add above, shared internal/ipfs/importspec params so root CIDs match bit-for-bit) and whose Has checks the recursive pinset; transfer fetch→re-import→canonical CID-string verify (the donor is go-cid-free — boundary decision 2026-06-23) with maxBytes+1 over-grant refusal (no truncated pin) and classified Fail reasons; durable verify/ack progress (FileProgressStore, atomic-JSON) with crash-safe persist-before-ack and a startup Has-recheck reconcile; generation-aware skip (a reassign at a new generation is never skipped by a stale acked-delivered) and unpin handling (clears progress + sidecar Unpin); storage_max_bytes (0 = uncapped) + kubo_api_addr config; the agent advertises blob-transfer/v1. Coordinator wiring is graceful: without a repair-signing key the control plane still runs (source endpoint 503s, no Source minted) until the operator provisions one. No migration (the production ack drives M3's 0012 state machine; key material is a secret path, the coordinator source identity is the reserved constant wire.CoordinatorSourceID); donor-deps-boundary (extended by exactly internal/ipfs/importspec) + migrations-frozen stay green. Tests: coordinator source-endpoint matrix (token/binding/replay/pre-boot/preflight/oversize), per-serve Source mint + boot-time + pubkey + require-cap, donor ipfsclient raw/dag-pb + pin semantics, transfer classification + oversize-not-imported, durable progress, client Ack/Fail/Fetch, agent replicate/reconcile/unpin/generation, and e2e loopback-mTLS replication → verified-holder + crash-before-ack recovery + cid_mismatch fail. Implemented (tag p2-m4-replication-slice). Design: docs/superpowers/specs/phase2/2026-06-23-phase2-m4-replication-slice-design.md. Plan: docs/superpowers/plans/phase2/2026-06-23-phase2-m4-replication-slice.md. Deferrals: donor-backed reads + require_replication_quorum_before_commit + origin/staging pruning + prune_safety_floor + coordinator_storage_mode/bounded cache + transform re-fetch → P2-M4.1; donor-as-source inbound /fed/v1/blob server + donor↔donor repair + the D11 egress budget's first debit → M5; placement/scheduler + failure-domain anti-affinity + blob_replication_state → M5; possession audits + reputation → M6; Prometheus /metrics → M7; real-Kubo block/put round-trip + CAR/streaming transfer → M7/M8.
P2-M4.1 Storage/read redirect — the milestone that makes the verified donor replica set the durable substrate, carrying the P2-M2.1 amendment M4 deferred. The coordinator's local Kubo becomes a bounded prunable cache, not the canonical origin; on a cache miss the coordinator sources ciphertext from a donor, verifies before decrypt, decrypts, and serves — staying the sole decrypt/serve point (T1.26, donor-blind). Built (Tasks 1–14, all reviewed): envelope-size propagation through every grant/preflight/snapshot path (blob_manifests.envelope_size; M4 grants reasoned over plaintext size while serving the envelope — latent defect closed); a coordinator federation client identity (nova://coordinator/<uuid>, role-aware transport.IdentityFromCert/CoordinatorClientTLS); a donor-safe replay helper (internal/federation/replay, the lone new boundary allowlist) + reversed-direction read-grant mint (source=donor, dest=reserved wire.CoordinatorSourceID, cid/generation/max_bytes/jti-bound); migration 0013 — the durable blob_storage_state projection (commit_state/durability_class/local_role/cache_segment/local_present/local_bytes/prune_eligible_at; acked holder counts a rebuildable cache, never authority) + nodes.source_nebula_addr + backfill; new capability read-source/v1 advertised with the donor's source_nebula_addr; a coordinator-only donor read-source server (internal/node/source, mTLS GET /fed/v1/blob/{cid}, full verify chain — peer-cert coordinator role + wire.Verify + source/dest/cid/generation binding + acked-progress + boot-floor + single-use jti + pin check + envelope-size preflight + the D11 egress budget's first debit (budget_exceeded on insufficient) — refused before any body byte); the coordinator donor-fetch tier behind OpenBytes (reputation-ordered sourceable-holder select → mTLS fetch under a coordinator-minted grant → io.LimitReader bound → deterministic re-import root-CID match BEFORE decrypt → bounded re-cache; never serves unverified bytes); read-path containment (per-fetch timeout, per-CID single-flight, coordinator bulkhead + per-donor concurrency, per-donor circuit breaker, bounded fallback) with ErrStagingNotVisible→404 / ErrNoSourceableHolder→503 semantics (a committed-but-momentarily-unsourceable blob is 503, not 404); coordinator_storage_mode (origin_copy default = never prune / bounded_cache / transient = unpin-on-read) + a size-aware SLRU/2Q bounded cache (probationary→protected on the second access; evict probationary oldest-first; bounded_cache_protected_ratio cap; bounded_cache_max_object_bytes admission refusal — scan/crawler pollution of large immutable blobs defeated); a minimal admission assigner over the M3 AssignPin seam (R(class) source-capable donors by reputation/liveness, one tx, best-effort gate-off); the async commit gate require_replication_quorum_before_commit (default-off) — gate-on uploads return 202/durability_state:"staging", are not read-visible, and defer the product OnCommitted to a crash-safe durability reconciler that flips staging→committed on a live acked sourceable quorum (then fires OnCommitted exactly once, MarkCommitted rows==1 guard) or staging→failed on age; the origin pruner + prune_safety_floor (unpin the local copy only at/above N live acked donor holders — CountSourceableHolders counts donors only, so the coordinator's own cache never inflates the floor; never prune below; crash-window backend.Has↔projection reconcile both directions); transform re-fetch of pruned parents through OpenBytes + a staging backdoor guard (the image surface refuses a non-committed parent with 404; prewarm fires only post-OnCommitted); config validation (transient⇒gate; 1 ≤ commit_quorum ≤ replication.factor; prune_safety_floor ≥ commit_quorum; bounded-cache/ratio bounds — unsafe combos refused, dubious ones warned), the four first-class /settings knobs (coordinator_storage_mode, bounded_cache_max_bytes, require_replication_quorum_before_commit, prune_safety_floor; rest advanced; restart-effect), evidence-shaped slog observability (storage.read.{cache_hit,cache_miss,donor_fetch,donor_fetch_failed}, storage.commit.{committed,failed}, storage.prune.{applied,skipped_floor,below_floor_alert}, node.source.{served,refused}), and a 2-donor loopback-mTLS E2E asserting the full chain: gate-on staging upload → 404 until a sourceable quorum acks → reconciler commits (OnCommitted fires) → pruner unpins the origin at/above the floor → a cold read selects a sourceable holder, the donor serves the full envelope over a coordinator-minted grant (egress debited), the coordinator verifies before decrypt, serves, and re-caches → a second read is a cache hit → a committed blob with no reachable sourceable holder returns 503. No migration beyond 0013 (forward-only, appended to MANIFEST.sha256); donor-deps-boundary (donor go-cid-free; new boundary = internal/federation/replay only) + migrations-frozen stay green. Implemented (tag p2-m4.1-storage-read-redirect). Design: docs/superpowers/specs/phase2/2026-06-23-phase2-m4.1-storage-read-redirect-design.md. Plan: docs/superpowers/plans/phase2/2026-06-23-phase2-m4.1-storage-read-redirect.md. P2-M5 may now assume the coordinator origin copy is not guaranteed. Deferrals → P2-M5: donor↔donor repair + the reserved repair-stream/v1 capability + the healing scheduler + 5-state-driven placement + failure-domain anti-affinity + blob_replication_state + client-direct donor reads; possession audits + reputation graduation → M6; Prometheus /metricsM7; real-Kubo block/put round-trip + CAR/streaming-AEAD v2 → M7/M8+.
P2-M5 Liveness & healing — where Phase 2's durability commitments become a functioning system. The first internal/orchestrator: a single-leader tick loop (liveness sweep → bounded reconcile drain → strict-Tier-1 healing → concentration/attrition signals). Migration 0014 — the rebuildable blob_replication_state projection (acked-on-countable counts: status IN (active,suspect) ∧ assignment_sync_state='current'; pending/cache/origin never count toward R; safety_tier ∈ {donor_lost,tier1,tier2,healthy} + local_recoverable) + a durable reconcile queue + the webhook_suppression table + D8/D9 placement columns (failure_domain_id/donor_principal_id/provider/asn/region/operator_verified_at/placement_weight) + nodes.assignment_sync_state + revoked_signaled_at + last_egress_* telemetry + pin_assignments.source_node_id (nullable FK) / source_attempts / source_next_attempt_at + pin_changes.source_node_id; backfill anchored on blob_storage_state. 5-state liveness sweeper (active→suspect→unreachable→evicted, strict-advancement; heartbeat is the canonical recovery path — suspect/unreachable→active re-enters reconciling and is not counted until it resyncs to current; evicted deletes its assignments after enqueue, revoked retains rows non-counting + emits federation.node_revoked exactly once via revoked_signaled_at; endpoint × status matrix). Single placement engine over pkg/coordinator/admission (lexicographic failure-domain anti-affinity — preference never veto; unverified dims collapse to one unknown bucket; trust/probation caps; bandwidth-decoupled ~√free × trust × placement_weight weight; reputation floor) + class-aware R validation (important R<2 refused; normal/cache warn-not-force). Donor↔donor repair: the M4.1 source server generalized to serve RoleNode callers (dest bound to the requester's verified node id; the same donor-authoritative egress bucket debited; io.CopyN exactly-size oversize hardening); additive Claims.Dest{AssignmentID,Generation} (source claim names the source's acked assignment, Dest* bind the destination's pending one); AssignPinWithSource (NULL ⇒ coordinator-as-source, never the synthetic id) + /pins/changes & /pins/snapshot late-mint (a stored source late-bound to its current address, requeued-with-backoff when no longer repair-sourceable; SnapshotItem.source so a snapshot-recovering donor still learns its source); the donor verifies the grant's dest binding before fetching, advertises repair-stream/v1. Healing scheduler: strict Tier-1 then Tier-2, RecomputeCID-before-reserve, asymmetric repair-source select (max remaining×reputation over repair-sourceable holders), telemetry-hinted pacing (heartbeat egress telemetry is a best-effort step_capacity hint; the donor bucket stays authoritative), restart-safe (re-derives tiers from the projection). First webhook dispatcher (internal/notify): best-effort, HMAC-SHA256 signed-when-configured (v1=hex(HMAC(secret, ts.body))), bounded worker pool (excess dropped), paranoid-gated, durable scoped suppression (event_type+destination+scope_key, once-per-window). Concentration (per-node Gini + per-dimension largest/top-k share + normalized entropy, unknown collapsed before grouping) + mass-casualty (federation.degraded, no budget override) + corrected slow-attrition (repair_time_days = desired ÷ surviving daily egress, storage_headroom; the dimensionally-wrong runway_days removed). Config: reputation_floor knob, R-change reload hook → RecomputeTargets, prune_stale_seconds deprecated (liveness status is now the freshness authority), four first-class /settings healing knobs, novactl node set-domain (operator-verified D8 dims). No migration beyond 0014 (forward-only, appended to MANIFEST.sha256); donor-deps-boundary (repair reuses existing donor-graph packages — no new allowlist entries) + migrations-frozen stay green. Implemented (tag p2-m5-liveness-healing). Design: docs/superpowers/specs/phase2/2026-06-28-phase2-m5-liveness-healing-design.md. Plan: docs/superpowers/plans/phase2/2026-06-28-phase2-m5-liveness-healing.md. Deferrals: possession audits + reputation graduation + pin_auditsM6; corpus-scale benchmarks + Prometheus /metricsM6/M7; multi-coordinator fencing → Phase 6; streaming-AEAD v2 → M8+.
P2-M6 Possession audits & reputation — evidence-driven trust. Closes the acked → challenged → verified → reputation moved → trust graduated → selection biased loop. Migration 0015 (forward-only ALTER; migrations-frozen stays green): pin_audits.received_at (coordinator receive-time; NULL on timeout; D10) / decided_at (always-set; indexing column) / transcript_hash (domain-separated "NOVA-POSSESSION-AUDIT-v1" digest; D-M6-3a) + nodes.trust_epoch_started_at / trust_review_required_at / trust_review_reason; joined_at-backfill so existing donors keep their tenure; EXPLAIN-gated indexes (pin_assignments_acked_at_idx new-ack fast lane, pin_audits_recent_pass_node_blob_idx, pin_audits_recent_fail_node_idx). Donor: BlockGetLocal (/api/v0/block/get, offline=true — no Bitswap fetch → 404 if absent) + POST /fed/v1/audit/challenge handler (coordinator-role-gated, RoleNode refused, no repair token, returns raw block bytes, separate audit egress governor) + capability audit-block-hash/v1. Coordinator dispatcher (internal/audit/possession): mTLS challenge over stored CID prefix reconstruction (stored.Prefix().Sum(bytes).Equals(stored); no coordinator local copy required) + transactional outcome (reputation move + pin invalidation on hard fail + trust state machine, SELECT … FOR UPDATE lost-update-safe) + post-commit federation.node_suspect webhook (24h node-scoped suppression; alias node.suspect). Two-stage weighted scheduler: due-node selection by cadence modulation + quota-bounded new-ack fast lane (within 15 min of ack); startup reconcile seeds lastRun from MAX(decided_at) per node. Trust state machine (D-M6-8, T1.32): auto probationary→trusted on age/audits/transfers/reputation/no-review-marker; auto trusted→probationary below reputation_floor; suspended operator-only (novactl node trust suspend|unsuspend); hash-mismatch resets epoch + sets review marker (novactl node trust clear-review to clear). Config: possession_audit block + two first-class /settings knobs (interval, deadline). Loopback-mTLS E2E: no-coordinator-origin verify + lying-donor hard-fail. donor-deps-boundary (no new allowlist entry) + migrations-frozen (only 0015 added) stay green. Implemented (tag p2-m6-possession-audits). Design: docs/superpowers/specs/phase2/2026-06-29-phase2-m6-possession-audits-design.md. Plan: docs/superpowers/plans/phase2/2026-06-29-phase2-m6-possession-audits.md. Deferrals: envelope_round_trip (two-call audit) + corpus-scale benchmark gate + Prometheus /metrics (D-M6-11 slog set is the blueprint) + below-floor BULK re-replication → M7.
P2-M7 Production hardening & donor release — the volunteer-ready federation release. No new replication policy; the milestone proves the M1–M6 stack operationally, observably, and compatibly. Migration 0016 (forward-only; migrations-frozen stays green): nodes.draining_at (the AUTHORITATIVE voluntary-departure marker, distinct from placement_weight=0; never touched by register/heartbeat — proven by the migration test) + partial covering index nodes_draining_idx. Drain lifecycle (D-M7-6): novactl node drain|undrain — one-shot, transactional (mark + fail pendings + enqueue reason node_draining), idempotent, refuses non-live nodes and warns on non-current sync (--force); query classification split rigidly into safety counts (healthy_acked/sourceable_acked/CountSourceableHolders EXCLUDE draining), placement (ListPlacementCandidates excludes), and selection (ListSourceableHolders/ListRepairSourceHolders keep draining, deprioritized by a prepended sort key — repair source of last resort); drain debt (CountDrainPendingCIDs, pending never reduces it) + eligible-destination in-flight; the healing tick now walks donor_lost before tier1 so a drained SOLE holder re-replicates from itself instead of stranding; two-step decommission UX gated on zero debt. Coordinator-only Prometheus /metrics (D-M7-1, T1.33): dedicated listener (metrics_listen_addr tri-state, loopback default, empty disables, bind failure startup-fatal, env NOVA_METRICS_LISTEN_ADDR wins), DB-derived scrape-time families (replication tiers, reconcile queue depth/age, node states, below-floor replica debt, drain debt, durable audit results) + process-local hook families (register failures, trust transitions, reputation moves, audit latency, donor fetch outcomes, egress refusals, selection failures) via nil-safe observer seams — consuming packages stay prometheus-free; label discipline test rejects per-CID/blob/path labels; donor graph github.com/prometheus/* HARD-DENIED in check_node_deps.sh. Corpus bench gate (D-M7-2): internal/benchcorpus — scratch-DSN-guarded, Zipf-skewed seeder (CopyFrom), make bench-corpus (release, ~9.8M blob_blocks, BENCH_CALIBRATE=1 threshold lifecycle) / bench-corpus-ci / bench-corpus-explain (deterministic index-availability EXPLAIN gate incl. nodes_draining_idx); bench-regression CI job; artifacts under reports/benchmarks/. Compat (D-M7-3): capability matrix tests (configured-required {pin-change-log/v1,snapshot/v1} fails closed; blob-transfer/read-source/repair-stream/audit-block-hash are route-gated — register fine, selection skips) + scripts/crossversion_e2e.sh (make crossversion-e2e): real-binary N−1(p2-m6-possession-audits)×HEAD matrix, per-side schema ceilings, join → replicate → audit → hash-verified DONOR-BACKED serve proof (coordinator Kubo repo wiped; bytes must come from the donor) → drain/undrain (HEAD coordinator only). The cross-version drill found and fixed two shipped production defects: the M6 audit dispatcher built scheme-less challenge URLs and the scheduler recorded discarded-error zero-value results as PASSES (audits fabricated passes without reaching donors — fixed: scheme prepended, OutcomeUnknown is the zero value, dispatch errors surfaced); the M4.1 coordinator→donor client used hostname/ServerAuth TLS verification no real donor serving cert can satisfy (fixed: federation-CA chain + nova:// URI SAN identity verification, tested both directions). Drills (D-M7-4): revocation-heals, provider-loss-heals (verified failure domains → surviving-domain placement via emergency coordinator source), disk-full (ENOSPC → out_of_space classifier — the one donor-side diff), corrupt-state fail-safe (registration fail-fast; cursor/progress safe-to-delete), and the drain e2e capstone (repair FROM a draining source over real mTLS; revoke after zero debt; healthy count unchanged). Docs (D-M7-7): docs/quickstart/donor.md volunteer walkthrough (workflow-extracted cosign identity/issuer policy, SBOM/provenance verification, digest pinning, first-boot cadence realities, graceful-leave), docs/runbooks/donor-lifecycle.md (revoke-vs-suspend-vs-drain, safe-to-revoke 3-condition gate, below-floor debt) + docs/runbooks/failure-drills.md; README/VERSIONING/VOLUNTEER drift fixes. Implemented (tag p2-m7-production-hardening-release). Design: docs/superpowers/specs/phase2/2026-07-01-phase2-m7-production-hardening-release-design.md. Plan: docs/superpowers/plans/phase2/2026-07-01-phase2-m7-production-hardening-release.md. Deferrals: below-floor BULK re-replication queue (hysteresis/rate-limit/untrusted-replacement) → P2-M7.1; envelope_round_trip + two-call /fed/v1/audit/responseP2-M8+; donor-local metrics → later opt-in; donor↔donor repair-client TLS identity verification (same class as the fixed coordinator→donor path; donor-side change) → P2-M7.1/M8; multi-coordinator fencing → Phase 6.
P2-M7.1 Beta readiness — the pre-first-private-beta hardening pass. No new product surface; it absorbs every deferred Phase-2 finding without a home, lands the below-floor remedy the M6/M7 runbooks promised, and runs a doc/security/logic/performance review over the highest-risk modules. Partition provisioning: migration 0017 + the retention Maintainer now create-ahead the jobs/audit_log/integrity_audits monthly partitions (fixes the production-down partition-exhaustion bug — inserts stopped once the install-time partitions aged out; the Maintainer runs whenever pool+backend+keystore are present, NOT behind IntegrityAudit.Enabled, so a disabled-audits operator still gets partitions); dbtest fails fast on a leaked-tx pool close. Below-floor replacement (D-M7.1-3): migration 0018 nodes.below_floor_since sustained marker + partial index; query classification mirrors drain (safety counts healthy_acked/sourceable_acked/CountSourceableHolders EXCLUDE a SUSTAINED-below-floor node, placement excludes, selection keeps-but-deprioritizes behind draining — composite preference healthy > draining > below-floor, the TRUE last resort); the drain-debt gate (CountDrainPendingCIDs/CountDrainInflightCIDs) also excludes sustained holders/destinations. The sweep (single-leader tick, below_floor.go): MaintainBelowFloorMarkers (hysteretic — stamp at reputation < floor, clear only at ≥ floor + hysteresis_margin, default 0.05; always runs for observability) before the healing tick; ReplaceBelowFloor (gated on enabled) AFTER the tick's donor_lost/tier1 passes — bounded requeue (EnqueueBelowFloorReconcile, reason below_floor, capped requeue_batch default 500) then replace-then-demote (DemoteBelowFloorReplicas fails the untrusted assignment ONLY once the trusted-holder count computed from AUTHORITY (not the possibly stale-high projection — the dip-avoidance property) is at target; a SOLE holder never demotes, staying repair source of last resort). below_floor_replacement config block (enabled/hysteresis_margin/grace/requeue_batch) + two /settings knobs; nova_below_floor_nodes{state} gauge + nova_below_floor_requeued_total counter; HEALING_PROTOCOL.md + donor-lifecycle.md flipped from "deferred/manual" to "automated". Donor↔donor repair TLS (D-M7.1-4): DonorRepairClientTLS binds the repair fetch to the instruction's named source identity (federation-CA chain + EXACT nova://node/<id> URI-SAN; wrong-identity refusal proven at transport + Fetch — an enrolled-but-compromised donor cannot impersonate the scheduled source). Supply chain: every GitHub-Actions uses: SHA-pinned (majors #1–#5 absorbed) with least-privilege permissions; Dockerfile FROMs + docker-compose.yml runtime images digest-pinned (certbot/certbot was implicit-:latest), refresh-docker-digests.sh + Dependabot docker ecosystem cover both. Dependencies: all Dependabot vuln alerts closed (Go ×2, npm ×7 dev/test-only — npm audit 0 both modes; x/crypto→v0.52.0 for 5 reachable SSH vulns; govulncheck residue = GO-2024-3218 only, no fixed release exists, accepted); TS 6 + react-router 7 majors taken, react-query 5 + Uppy 5 + the kubo/boxo/kad-dht group (needs go 1.26.4) deferred as post-beta currency. Review (D-M7.1-6): security — compose pins + a TLS session-resumption invariant guard (InsecureSkipVerify+VerifyPeerCertificate clients must keep ClientSessionCache nil or a resumed session skips the custom check); logic — pin_db_test false-PASS + scanner.Err() surfacing; performance — a TestExplainPlans case pinning the sweep's victim selection to nodes_below_floor_idx; envelope AEAD / argon2id / constant-time compares / SQL-injection surface reviewed sound. Docs: doc-drift sweep (envelope-audit deferral target corrected P2-M7→P2-M8+; DATA_MODEL.sql mirrors 0018), a from-scratch README rebuild with brand hero + screenshots + progress table (dev walkthrough moved to docs/development.md), and docs/REVIEW_2026_07_04.md (findings + deferral ledger). make test/make web/codegen-check/migrations-frozen/node-deps-check/bench-corpus-explain green. Implemented (tag p2-m7.1-beta-readiness). Design: docs/superpowers/specs/phase2/2026-07-04-phase2-m7.1-beta-readiness-design.md. Plan: docs/superpowers/plans/phase2/2026-07-04-phase2-m7.1-beta-readiness.md. Review: docs/REVIEW_2026_07_04.md. Deferrals: react-query 5 + Uppy 5 majors + the kubo/boxo/kad-dht Go group → post-beta currency; envelope_round_trip + two-call audit → P2-M8+; below-floor grace threading into RecomputeCID/CountSourceableHolders (accepted safe divergence) + partition-provisioning decoupling from the audit subsystem → follow-ups; multi-coordinator fencing → Phase 6. Next: the P2-M7.x field-findings remediation track (below), then streaming-AEAD envelope (P2-M8+).

Phase 2 — P2-M7.x field-findings remediation track (first-deployment feedback)

Source: the operator's first-deployment field findings (kept out of the repo) — 23 items collected in practice, by following the shipped documentation, while standing up a public operator node on a fresh Debian 13 VPS (2026-07-27/28), mirroring real content (2026-07-30), and onboarding the first external donor (2026-08-08). Not a code-reading audit; every item states its evidence. §1 (widget collection_id as the string "undefined") is already fixed in c987b48. The findings' architecture analysis is not milestone work here — its ordering rationale and durability claims discipline are folded into Phases 6/7/8 below, and its implementable "cheaper wins" land in P2-M11.

The track's thesis, in the findings' own words: the next marginal federation protocol feature is less valuable than making the existing path reproducible for a second human being.

Three cross-cutting constraints bind every milestone in this track.

  1. Upgrade-path continuity is a release gate, not an aspiration. An existing federation must cross every milestone using only docs/UPGRADING.md. Out-of-band manual remediation is a last resort; where a milestone would require one, the milestone instead ships the automation that removes it (P2-M7.2 adoptive federation init, P2-M7.5 tombstone-eviction upgrade sweep). docs/UPGRADING.md is created in P2-M7.2 — every milestone adds its own entry as a deliverable — and becomes normative/generated/tooled in P2-M7.3. New blocking CI gate live-upgrade-e2e: stand up the previous tag with a coordinator and a registered donor, upgrade to HEAD following only the doc, assert the federation is healthy and the donor still sourceable, with zero commands not in the doc.
  2. Quickstarts get you operational; reference docs explain the knobs. A quickstart carries the shortest working path and nothing else — no caveat dumps, no architectural asides. Every knob moves to docs/reference/{operator,donor}-configuration.md, with docs/VOLUNTEER_DEPLOYMENT_GUIDANCE.md retained as the judgment companion (host choice, provider diversity, backups, troubleshooting) and its duplicate setup walkthrough deleted. Exception, deliberate: the donor's consent decisions — storage_max_bytes, bandwidth_budget_bytes_per_day, storage_dir — stay in the donor quickstart, because a volunteer must decide how much disk and traffic they are lending before they run anything. Everything else in node.yaml is generated by node invite and never hand-edited.
  3. One canonical path per audience, enforced by CI. Docs rot is what produced §22 — six independently maintained sources of truth that had already diverged on ports, topology, paths, secrets and milestone-era comments. The anti-rot mechanism is the P2-M7.2 drift-test suite, not review discipline.
Slot Deliverable
P2-M7.2 Federation productization — turning a working protocol into a deployable product. Closes §22. Ships deploy/operator/compose.federation.yaml (Nebula sidecar via network_mode: "service:coordinator" so nebula1 is visible to the coordinator without destroying its Docker network identity; NET_ADMIN, /dev/net/tun, UDP/4242 publication, runtime federation mounts — the non-federated base Compose stays unchanged), and a packaged nova-admin service/image carrying novactl plus a pinned nebula-cert with narrow mounts (config, an admin-only PKI volume, invite output) — the Nova federation CA key and Nebula CA key live only in that volume and are never mounted into the long-running coordinator or Nebula containers. novactl federation init is atomic, idempotent and refuse-to-clobber, running the ten-step bootstrap (preflight /dev/net/tun + route conflicts → validate overlay CIDR → Nova federation CA → coordinator server identity with correct IP/DNS SANs → coordinator federation client identity → repair-token signing seed → Nebula CA + lighthouse identity/config → runtime secrets with explicit permissions → non-secret address/fingerprint manifest → operator.yaml federation: block last and atomically) and is adoptive — an operator with a hand-built CA, hand-edited operator.yaml and a donor already mid-enrollment is inventoried, validated and imported rather than clobbered or told to start over (constraint 1; without this the flagship milestone would itself demand the manual remediation the track forbids). novactl federation doctor proves readiness before any donor identity is minted (nebula1 at the expected address; cert/key pairs and SANs match; repair signing key parses; coordinator client identity exists; the federation listener is overlay-only and listening on 9443; live mTLS probe; Kubo private-swarm prerequisites; CA private keys absent from runtime containers; file permissions sane) with --json for support automation. novactl node invite becomes the normal path — one complete runnable bundle (canonical Compose, complete node.yaml, federation identity, Nebula identity, private Kubo swarm key, Kubo hardening init, invite-manifest.json), with --nebula-public-key so the operator signs a donor-generated public key and never receives the overlay private key, and a generator assertion that the bundle contains none of the operator's CA private keys, coordinator identities, repair-signing key, master key or OIDC signing key. node issue and the template commands survive as expert primitives. Deployment vocabulary standardized4242/udp Nebula lighthouse, 9443/tcp coordinator federation mTLS (overlay-only), 9555/tcp donor read-source (overlay-only); the 8443 reuse (already the public nginx HTTPS port) is eliminated everywhere. One canonical donor deployment definitiondeploy/donor/compose.yaml and the generated Compose derive from a single embedded source, proven by a drift test; canonical topology is Nebula + hardened Kubo + nova-node, /etc/nova for public/config material and /run/secrets for private keys. Kubo hardening is part of the artifact (public bootstrap removed, public DHT/routing + mDNS disabled per the private-swarm model, API/Gateway bound appropriately, NAT/public-exposure behavior off, swarm key mounted automatically) — a donor created by the official path is private-by-construction. No mutable production tags: node invite requires or resolves an explicit Nova digest and records it in the manifest; reviewed Nebula/Kubo versions are pinned in the canonical deployment. Registration fix, not a documented workaround (internal/node/agent): successful initial registration updates in-memory registration state, starts read-source idempotently in-process, and runs the first heartbeat and first desired-pin synchronization immediately — normal timer intervals begin after that, and a process restart returns to being a recovery action rather than a first-boot protocol step. Docs: new docs/quickstart/federation-operator.md (prerequisites → init → doctor → invite → handoff → node list → drain/revoke/rotation → CA backup); docs/quickstart/donor.md rewritten donor-focused around "your operator sends you a folder, you run two commands" plus the three consent knobs; new docs/reference/{operator,donor}-configuration.md and docs/platforms/wsl2-donor.md + platform support matrix (WSL2-not-WSL1, systemd, /dev/net/tun, persistent Docker, Linux-filesystem storage not /mnt/c, sleep/hibernate ⇒ offline); deploy/operator/README.md reduced to artifact notes plus a link; historical "lands in M2" comments removed from operational files; Nebula PKI vs. Nova federation mTLS explained as separate trust roots with an explicit statement of which secrets may leave operator custody; docs/UPGRADING.md created with this milestone's entry. Gates: drift tests failing when operator docs contain a CLI command the current binary cannot parse, when generated mount sources are absent, when the donor bundle omits Kubo, when node.yaml paths do not correspond to mounts, when operator-only secrets appear in an invite, when 4242/9443/9555 disagree between artifacts, or when an unapproved mutable latest appears in an operational template; the clean-room deployment E2E (boot normal operator Compose → packaged bootstrap → Nebula sidecar → doctor passes → generate one invite → generated invite passes docker compose config → start that exact topology → observe registration + immediate heartbeat/sync → read-source starts without restart → coordinator authenticates to read-source → drain/revoke cleanly) plus live-upgrade-e2e, both on a privileged/self-hosted release gate, with cheap render/config/secret-leak checks on every PR. Risks to settle in the design: hosted CI cannot reliably provide TUN/privileged networking (hence the split gate); network_mode: "service:coordinator" must be reconciled with the M14 read-only-rootfs / cap_drop: [ALL] / no-new-privileges hardening; pinning nebula-cert introduces the first non-Go binary into an image that has none. Acceptance criterion: operator enables federation → doctor passes → operator emits complete invite → donor runs it unchanged → operator sees a healthy, sourceable donor is one tested product path. Implemented (branch p2-m7.2-federation-productization). Design: docs/superpowers/specs/phase2/2026-08-08-phase2-m7.2-federation-productization-design.md (ratified after review round 1; six required changes incorporated). Plan: docs/superpowers/plans/phase2/2026-08-08-phase2-m7.2-federation-productization.md. Delivered beyond the original design: the review exposed a hard startup deadlockcheckListenOnInterface failed config load while network_mode: service:coordinator needs a running coordinator — closed by amending the M2 contract to bind-or-degrade with a bounded wait (interface_wait_seconds, default 120s) plus nova_federation_listener_ready and /readyz on the operator-only metrics listener; doctor split into three planes so no check needs the Docker socket; init made failure-atomic-with-respect-to-activation with crash-injection proof and an explicit preservation inventory; and a runtime custody split (D-M7.2-2 step 8) that the overlay work surfaced as missing — runtime identities are installed into the config/secrets volumes while both CA keys stay in the admin-only volume. Findings beyond §22: node revoke/node rotate-cert were also documented with a non-existent --node-id; the keys dispatch sliced os.Args[2:] inconsistently; and Agent.Run already did an immediate first sync, so only the heartbeat and read-source start were genuinely absent. Gates: deployment-artifacts CI job (gen-deploy-clean, deploy-gates, docs-cli-parse over 40 invocations, port-vocabulary, no-mutable-tags, compose-custody, live-upgrade-e2e) on every PR; federation-deploy-e2e on the privileged gate. No migration; migrations-frozen, node-deps-check and codegen-check stay green. Next: P2-M7.3 (upgrade & release lifecycle).
P2-M7.3 🟡 Upgrade & release lifecycle — the day-2 contract. Closes §23. COMPLETION STATE 1 of 4: code complete. States 2 (RC verified), 3 (release published) and 4 (baseline→release transition passed on the real deployment) are NOT reached, each needs its own recorded write, and this row becomes ✅ only at state 4. No product tag exists: tag creation belongs exclusively to release.yml, which has never run — though its state machine is now complete end to end. novarel evidence|lock|bundle produce the three documents the package could previously only consume, and the whole path was driven locally: the bundle assembles and the lock REFUSES, naming a claim no executed gate supports. That refusal is the correct state and the reason there is no release. Intent vs lock (D-M7.3-2b/2c/2d). The circularity — stamping version and labels changes the image config and therefore the manifest digest — is broken by two documents: a reviewed, checked-in releases/intent/vX.Y.Z.json that declares claims, target schema, platforms, supported predecessors and capability profiles and carries NO digests, and a post-build signed lock.json that binds each claim to evidence from the exact candidate digests. A generated catalog_gen.go compiles the intent into every binary so upgrade status works offline. The bundle authentication graph is ACYCLIC and proven so: lock.sigstore.json authenticates the lock, the lock hashes every payload member, and it can never hash itself, its Sigstore bundle or its certificate; hashes.txt is derived, never an authority. The donor lock is a deterministic PROJECTION whose digest the release lock records — so it joins to the INTENT digest, since two documents cannot each be an input to the other. Out-of-band trust (D-M7.3-2a). scripts/nova-release: a fixed documented cosign verify-blob with issuer and identity hard-coded OUTSIDE the bundle, then a self-hash check, then every member against the lock in both directions, then the bundle's policy copy audited against the script's own — the copy is auditable, never the authority. Refusal tests assert the target admin was NEVER invoked, and the policy test uses a RESEALED bundle, because the interesting case is one that is internally perfect and still must not be trusted. Migration obligations (D-M7.3-8). Six independent dimensions composing across a range (conjunction of permissive, disjunction of restrictive, union of procedures), with OldBinaryCompatible carrying a NAMED predecessor. Not a scalar severity: 0003 drops two tables while the tables it recreates may still serve an older binary, and one ordinal loses whichever fact the operator needed. Apply and journal (D-M7.3-9a/9b). migrate apply --to <n> — advisory lock, re-read under it, refuse an unaccounted-for set, stop at target, acknowledge procedures by id. migrate down is GONE. migrate auto replaces the entrypoint's unattended up and applies only a range that is safe on every dimension. A local journal is written and fsynced before the apply, because 0019 creates the very table that records upgrade runs; the backfill is idempotent on (run_id, sequence). The coordinator refuses to start against a stale schema. Runtime contract and census (D-M7.3-6a/7c/13/21a). Donors declare version, digests, capabilities and protocols on every heartbeat under an explicit contract version; effective_capabilities becomes the single operational source across placement, repair, possession and storage state. Identity claims are census-only; capability advertisements gate ROUTING, not authorization. runtime_contract_observed_at separates legacy omission from rollback, which emit byte-identical heartbeats. The census is orthogonal AXES, never one enum, classified in Go because SQL cannot compare SemVer. Nonconformance ADVISES: a deprecation message and nothing withheld. Day-2 CLI. novactl upgrade status|check|verify, all from the TARGET admin, all fetching nothing. Preflight separates REQUIREMENT from OUTCOME so "could not be checked" cannot read as "is fine"; acknowledgement waives a skip and never a measurement; the rollback boundary prints first and names its predecessor. Verification is per-plane under one run id, written atomically to a mounted directory. node rollout authorize and node convert-bundle take the lock as a DIGEST, because a path is not an identity. Deployment. The base Compose names released artifacts by digest and carries no build:; a dev overlay adds it back; a release env separate from docker/.env is passed LAST and installed atomically. Donor bundle v2 by CONVERSION — never reissue, which would mint a new UUID and certificates — with per-component conditional rollback and a volunteer script holding no coordinator authority and removing no volume. Evidence. Three distinct gates plus a backup drill, in three tiers (pr-static · rc-docker · release-tun), with promotion requiring the evidence tier. EXECUTED 2026-08-11/12: upgrade-schema-e2e (the baseline coordinator runs AND serves against schema 19 — the only evidence a rollback-safe claim can have, and nothing tested it before), upgrade-wire-e2e (a 143c459 donor registers, acks, is audited and serves a hash-verified donor-backed read against the candidate), crossversion-e2e repinned to the intent-declared predecessor and rerun in all three pairings, and backup_restore_e2e (all three volumes destroyed and restored; the restored CA still vouches for a certificate it issued before). Also fixed, found by the gates themselves: the donor healthcheck override named a binary path that does not exist, so every generated donor reported unhealthy while working; admin.Dockerfile copied nebula-cert from a path absent in the base image; the node image shipped a root-owned data volume; ObligationID collided on two real procedures. Also closed by the post-merge amendment sweep: signed evidence statements, the lock builder and the bundle assembler (nothing produced any of the three before); the configuration fingerprint, over the effective config, preserving unknown keys so a rollback cannot report "unchanged"; the verify phase, so upgrade_runs no longer stops at apply; the missing DOCTOR plane; and the P2-M7.3 entry in UPGRADING.md. mixed-fleet-e2e is now WRITTEN AND PASSING (2026-08-12, sixteen assertions): six donors register simultaneously against the PREDECESSOR coordinator at schema 18, the schema advances to 19, and the candidate takes over — nobody evicted, nobody drained, no assignment failed, every version label gating nothing, the capability-missing donor losing exactly its role, and an evicted donor recovering with no re-enrollment. It moved from the release-tun tier DOWN to rc-docker: the overlay is transport, the claim is about coordinator policy, and a gate that runs on every main push beats one gated on a self-hosted runner; what loopback costs is recorded in its coverage entry. It found a production bug no single-donor test could: donors sent no nebula_cert_fingerprint, the column is UNIQUE NOT NULL, and the SECOND donor ever to register got an opaque 500 — a federation with two volunteers could not form. Donors now send their certificate's fingerprint, the coordinator synthesizes unknown:<node-id> for the already-deployed binaries that cannot (a nullable column would have broken the predecessor's scan, trading a registration bug for a rollback bug), and a genuine duplicate is a legible 409. The transition claim is SPLIT, which is what makes a first release possible at all. baseline-deployment-transition could never be proven at lock time — the lock is signed before publication, so a claim about crossing to a published release cannot be true until one exists, and no release could ever be cut. It is replaced by candidate-baseline-transition, proven PRE-publication by upgrade-candidate-e2e against the exact candidate artifacts (executed 2026-08-12, fifteen assertions: the startup floor refuses a stale schema and says why, the target-bounded apply crosses (18, 19] with obligations acknowledged by id, the run is journalled with the configuration fingerprinted, and users, archive rows and the donor's node id all survive). upgrade-release-e2e is retained as a POST-PUBLICATION gate bound to no lock claim: it verifies the final registry refs, the release assets, the lock AS DOWNLOADED, and an operator following UPGRADING.md. GateCoverage.PostPublication makes this structural — ReleaseCandidateReady ignores such a gate, CompletionReady requires it, ValidateClaimCoverage refuses an intent that binds a claim to one, and AssertEvidenceSupportsClaim refuses its evidence. A lock can now be cut; P2-M7.3 is still not complete, and does not become ✅ until upgrade-release-e2e passes against the published release. Its result is never folded back: the v0.3.0 lock is not re-cut. The coordinator-side SAFE REACTIVATION path also landed here, as a product defect rather than a test gap: a supported donor offline past eviction was useless indefinitely and could not be taught otherwise, since its agent never re-registers and ignores the refusal; reactivation now restores participation and not standing, forces a snapshot, credits no replica, and is audited. The claim that P2-M7.4 picks up _admin automatically is removed: that milestone needs an explicit route migration and a parity test. AMENDED 2026-08-12 — the release workflow is now wired end to end (D-M7.3-23…30, plan Phase 7). The milestone shipped the release documents and a state machine describing them, and a workflow that could not execute one: the evidence job exited unconditionally with "not implemented" and the promotion job downloaded a release-bundle nothing produced, so a dispatch would have pushed and signed three images and then failed. Now: the gate list is DERIVED from the intent (novarel plan; GateCoverage carries MakeTarget, AcceptanceScenarios and RequiredForRelease, and a test asserts every target exists) rather than five literal make lines a later release's new claim would silently skip; the gates run against the PUSHED IMAGES (scripts/lib/candidate.sh extracts the binaries from the candidate digest — downloading a descriptor and rebuilding the same commit demonstrates that a file exists, not what an operator will pull); evidence names the reviewed decision (IntentDigest is required, and a claim conditional on capabilities needs evidence that recorded exercising them); platforms are declared and realized EXACTLY in both directions, with full descriptors including child manifests and BuildKit's attestation manifests excluded — Nova was single-platform because a YAML line said so, and donors are volunteer hardware; every external write is no-overwrite with exact-match resume in scripts/release-publish.sh, including the absent-versus-unreadable distinction (creating a tag because a transient error made it look missing is the same overwrite) — and release-publish-injection stands up a fake registry, Git remote and GitHub, interrupts the sequence at all seven points and asserts the retry converges, with a fake docker that refuses to build; the approval gate shows the reviewer the lock digest, every descriptor, every claim and its statement digest, rather than a job name; and completion state 4 runs in the same dispatch — an on: release: published workflow would never fire, because events created with GITHUB_TOKEN do not trigger further runs, so the gate would silently never run and the milestone would sit at state 3 looking finished. A failing transition leaves the release published, completion at state 3, and the lock un-re-cut. make release-pipeline-rehearsal drives the whole document pipeline offline for TWO releases — the real intent and a synthetic v0.3.1 differing in predecessor kind, predecessor count, claim set and platform count — because a pipeline that only worked for the first release would pass every check written against the first release. What remains unprovable locally is stated rather than faked: OIDC, environment approval, GHCR permissions and immutable releases are GitHub's, and the first dispatch is what confirms them. docs/RELEASING.md carries the procedure, the refusals, the retry semantics and the required repository configuration. Still 🟡 at completion state 1: nothing has been dispatched. Design: docs/superpowers/specs/phase2/2026-08-11-phase2-m7.3-upgrade-release-lifecycle-design.md. Plan: docs/superpowers/plans/phase2/2026-08-11-phase2-m7.3-upgrade-release-lifecycle.md.
P2-M7.4 Deployment & docs correctness — the "I followed the shipped documentation and it broke" class. Closes §2 §3 §4 §5 §6 §7 §8 §11. Individually small; collectively the difference between a quickstart that works and one that does not. §2novactl stops touching ~/.config for subcommands that need no client state (the documented novactl config set uploads.default_collection_id cannot run in the documented container: mkdir /root/.config: read-only file system), honours XDG_CONFIG_HOME, and the image sets it to a writable path. §3novactl collection list plus a --help that enumerates subcommands (today collection implements only create, and --help is rejected as an unknown subcommand, so an operator who created a collection has no CLI way to recover its UUID for uploads.default_collection_id or upload-token create --collection); the audit covers every other subcommand group for the same gap. §4 — certbot moves to its own acme profile, because prod publishes only 8442/8443 and nothing binds host :80, so HTTP-01 can never complete and the coordinator serves the wizard's 7-day self-signed cert indefinitely while a certbot container sits alongside looking healthy; prod is documented plainly as expecting an external terminating proxy. The move must not orphan the existing ACME account/lineage in the nova-letsencrypt volume (constraint 1). §5resolver 127.0.0.11 valid=10s plus a variable in proxy_pass in internal/setup/templates/nova.conf.tmpl and the nginx reference: nginx resolves coordinator:9000 once at config load, so restarting only the coordinator makes every API route return 502 indefinitely — a silent outage that looks like a coordinator fault. §6 — the wizard refuses or auto-resolves uploads.public_uploads: true with no uploads.default_collection_id, following the existing precedent in internal/config/operator_yaml.go for rejecting incoherent config; today that combination returns 201 committed on write and 401 on read, with no warning where the operator enabled it. §7xmlns="http://www.w3.org/2000/svg" added to nova-federation-glyph.svg and nova-mark.svg (present on nova-hero.svg, so this is inconsistency rather than house style) plus a lint check; without it a brand asset renders inline but fails as a standalone document, which is exactly how <img>, favicons and README images use it. §8 — the widget demo page at /widget/ is aligned to data-product="image" to match the quickstart, so copying from the first thing an operator opens no longer produces blobs that cannot serve resized derivatives. §11 — the admin/public prefix collision, fixed in code rather than documented around (scope decision 2026-08-08): the admin SPA and public API are served by different nginx vhosts with overlapping /api/v1/ prefixes, so an operator fronting Nova with one hostname routes /api/* to one upstream and loses half the API either way — /api/v1/uploads 404s on the admin vhost, the entire admin console 404s on the public vhost, and the split is discoverable only by probing both ports. The coordinator registers both /api/v1/admin/* and /api/v1/_admin/* with _admin canonical (route mounting is centralized in internal/api/server.go); nova.conf.tmpl routes both on the admin vhost, making the single-origin rule unambiguous (/api/v1/_admin/* → admin, all other /api/v1/* → public); web/admin moves to _admin; openapi.yaml documents _admin and marks the old prefix deprecated; the old prefix returns a Deprecation header and logs a startup notice, with removal scheduled for a named later release and recorded in docs/UPGRADING.md. Dual-serve rather than hard cutover is deliberate — it makes the change a zero-manual-step upgrade for the existing deployment (constraint 1) and turns single-origin routing into a capability gained rather than a config repaired. The exact prefix table is published in the quickstart, docs/legal/OPERATOR_CHECKLIST.md and docs/recipes/NGINX_REFERENCE.md regardless. Docs: docs/quickstart.md gets the constraint-2 split — its TLS-mode table and headless-setup appendix move to docs/reference/operator-configuration.md, leaving the shortest working path. Acceptance criterion: a clean-room operator following docs/quickstart.md end to end hits none of these.
P2-M7.5 Serving & ingest correctness. Closes §13 §20 §21 §17 §19. Sequenced before the admin-console milestone deliberately: §15's thumbnails warm the very cache §13 fails to invalidate. §13 — tombstoned blobs keep serving bytes from the coordinator hot cache (highest-severity finding: a moderation-correctness bug in which a completed takedown does not take the content down). The tombstone is honoured everywhere except the route that matters — /blob/<cid> returns 200 image/jpeg with x-cache-status: HIT while /blob/<cid>.json and /i/<cid>/p/thumb.webp return 410, and appending an unused query parameter (a different cache key) makes the same object correctly 410. Tombstone, quarantine, soft-delete and blocklist-add must all evict the CID from the hot cache in the same transaction that changes the state, not on a later sweep — max-age=31536000, immutable is correct for content addressing and should stay, which makes server-side invalidation the only lever, so if the coordinator's own cache ignores a tombstone no downstream cache will do better. Audit whether derivative entries (/i/<cid>/...) are evicted by name or merely miss because the DEK is gone; if the latter, the same class of bug is latent on any path that can answer without touching the key. Regression test warms the cache first — that step is the whole test, since without it the current code passes. Upgrade sweep (constraint 1): a one-time eviction pass at migration time clears CIDs already tombstoned and already cached, so the fix does not ship requiring a manual purge on live deployments. §20 — SVG cannot be stored in a form the coordinator will serve as an image, which matters because Nova is pitched at forums, wikis and archives whose diagrams, logos and icons are overwhelmingly SVG. Every combination currently fails or is useless: image/svg+xml is rejected as mime_rejected under every product because the sniffer (almost certainly net/http.DetectContentType, which has no SVG rule) classifies it as text/xml on the <?xml prefix or text/plain on a bare <svg>, and the two combinations that do store — text/xml, text/plain — serve back a MIME type no <img> will render. Fix: an explicit SVG rule ahead of the generic XML classification (root element svg in the SVG namespace, optionally behind an XML declaration, comments and a doctype), then accept and store image/svg+xml. The sanitiser is not optional — SVG is script-capable (<script>, event-handler attributes, <foreignObject>, xlink:href="https://github.com/nova-archive/nova/tree/main/docs/javascript:") and this deployment serves the admin console from the same origin, making attacker-supplied SVG a stored-XSS delivery mechanism; plan is strip-active-content at ingest plus a sandbox CSP on SVG responses, with the separate-origin option documented for defence in depth. §21 — .ico returns 500 under product=image while raw and document store the identical bytes, so the image product accepts the ICO by MIME and hands it to a decoder with no ICO support that fails uncaught. Decide whether ICO is a supported image input and reject at validation with 415 if not; and — the part that matters more — make the decode path return a typed error rather than a bare 500, which is indistinguishable from the coordinator being broken and a bad thing to hand an uploader over a file-type choice. §17a — a bare CID redirects 301 to the same URL carrying the extension for its stored MIME type, and both resolve; content sniffing already knows the type at ingest, and the payoff is everything downstream of the URL (file managers, wget, forum software deciding whether to inline a link, every tool that treats an extensionless URL as "unknown, do not preview"). §17b — a format or WxH suffix on the original's CID resolves to the corresponding derivative through the (original CID, transform) → derivative CID mapping that already exists in Postgres, generating on miss only within the existing whitelist, with Link: rel="canonical" pointing at the derivative's own address so caches and crawlers do not treat the two as unrelated objects. Each derivative keeps its own CID — that is what makes it verifiable — but no human tracks six CIDs per image, so the original's CID plus a suffix is the handle people will actually use. The whitelist bound is load-bearing: an alias route that generates on miss is a work amplifier pointed at the open internet, and a crawler walking w1.webp, w2.webp, w3.webp would otherwise mint unbounded derivatives. The alias route must also honour tombstones and must not become a cache-warming bypass — which is why §13 lands in the same milestone. §19durability_state is returned by POST /api/v1/blobs but absent from the UploadResult schema; it is exactly the field a careful client wants (how a caller distinguishes "accepted" from "durably placed"), so it is specified rather than discovered, with enum values confirmed against the code, as its own spec version bump (deliberately excluded from the documentation-only 0.2.0-beta.1+ad629d7 pass).
P2-M7.6 Admin console & uploader-facing capability. Closes §14 §15 §16 §18 §9 §10. §14 — the admin "Settings" control paints an opaque overlay and traps the UI: the viewport fills with background tan, nothing is clickable, Back does nothing, and recovery requires typing a different admin URL and reloading — a modal mounting its backdrop at full viewport while the panel fails to render, keeping pointer-events and its focus trap, opened without a history entry. Three fixes, all of them: find whatever throws during the panel's render (the backdrop appearing proves the open path ran); an error boundary around the panel so a render failure shows an error inside the frame rather than an opaque sheet over the app; and Escape-closes + backdrop-click-closes + a pushed history entry so Back works. Any one of the three would have made this recoverable rather than a dead end. §15 — the Blobs list shows CIDs and cannot answer "what am I looking at" without opening every row; it gains original-or-derivative as a column and a filter (the first question about any row), MIME type as a column, and thumbnails from the thumb.webp preset already generated at upload for every image, so this costs no new generation work. Rendering thumbnails means the admin page fetches content through the ordinary read path and warms the hot cache — decided deliberately rather than discovered, and safe only because P2-M7.5 lands the transactional eviction first. §16 — the blob detail view lets an operator copy the CID but has no link that simply opens the content; it gains one next to the copy control, plus the derivative URLs the upload API already returns and nothing in the admin UI currently surfaces. §18 — the audit log shows what happened and to what but not when, though the API carries the timestamp (a display omission, not a data one); and both Audit and Jobs gain export — CSV for spreadsheets, JSONL for anything programmatic, honouring the filters currently applied rather than dumping the table, and streaming rather than buffering because these tables grow without bound. An audit log is evidence, so the export carries enough context to be checkable: coordinator identity, the filter that produced it, and the time range, in a header row or JSONL preamble. §9 — signed URLs cannot be minted by the person who uploaded the content. Not a bug but a design gap: POST /api/v1/admin/signed-urls/sign is operator+moderator because minting needs the unwrapped HMAC secret only the coordinator holds, which is correct as far as it goes — but it leaves an ordinary uploader (anonymous, or holding a scoped nova_ut_ credential) with no way to obtain a signed URL for their own private upload, so "upload privately, share a time-limited link" is not expressible by a non-operator. Fix mirrors upload_tokens exactly: the operator mints a long-lived, revocable, origin-scoped read credential for a site, and that site's backend derives short-TTL signed URLs from it — signing stays server-side, the operator keeps control of trusted origins, and the per-URL human step disappears. Ships with a docs correction: aud is checked against the request's Origin/Referer, which are browser-supplied and forgeable with curl, so signed URLs stop embedding elsewhere but do not stop downloading — the same security class as CORS. SIGNED_URL_FORMAT.md says this under "What the format does not do"; the framing elsewhere reads stronger and is corrected. §10 — the documented embed cannot give the uploader their URL, which on a service with no accounts is total loss: the blob is unreachable forever because there is nothing to look it up by. The auto-bootstrap path <div data-nova-upload-widget> that docs/quickstart.md documents reads only non-secret data-* attributes via parseElementConfig, and there is no attribute for onComplete, so the UploadResult (and urls.original inside it) reaches only a callback the documented embed cannot supply — and the two paths cannot be combined, because mount() keeps one instance per element in a WeakMap and returns the existing handle, so an element claimed by autoBootstrap() on DOMContentLoaded silently discards a later mount(el, {onComplete}). Fix: the widget renders a receipt itself by default (the brand doc specifies exactly what one looks like in section 08 and calls it "the design centrepiece"), and mount() merges options into an existing instance rather than discarding them. Default-on receipt is a behavior change, so it ships with an opt-out attribute (constraint 1). Likely a migration for the read-tokens table.

Deferred out of this track to P2-M11 (after the streaming-envelope milestones P2-M8/M9/M10, which own docs/superpowers/specs/phase2/2026-06-11-phase2-federation-design.md's milestone slots): §12 — the admin console is single-factor. Operator auth is well built as far as it goes (argon2id in internal/auth/password/password.go, per-IP rate limiting at internal/api/server.go:142) but there is no TOTP, no WebAuthn, no recovery codes; and the console is not a content surface — it controls master-key rotation, moderation and takedown, runtime configuration and upload-token minting, so a single password stands in front of the highest-value endpoint in the system. Rate limiting stops online brute force and does nothing against a reused, phished or session-hijacked password, and every external gate available to an operator today is worse than a real second factor (HTTP Basic re-sends a base64 credential on every request and is phishable with no revocation story; mTLS is strong but negotiated during the handshake so it cannot be path-scoped to /admin without a separate hostname; IP allowlists break travel and mobile networks). TOTP for operator/moderator plus recovery codes closes it against the existing users table; WebAuthn is the better end state and stays Phase 5. Ships with a docs note that Caddy redacts Authorization from access logs by default but not every proxy does, so a generic "put basic auth in front of it" recipe on nginx or Apache can write reversible credentials into access logs on every request. P2-M11 also carries the findings' implementable "cheaper wins": proactive diversity-debt rebalancing (a slow background pass scoring provider/ASN/principal/region/host diversity per object over already-healthy but concentrated data, copy-verify-then-demote, obeying contributor budgets, pausing during real repair emergencies, and never rejecting an upload for imperfect diversity — the 64% → 7.5% simulation result implies roughly an 8.5× reduction in provider-purge exposure at no permanent storage cost, the highest-return item available); an explicit repair_reserve_percent SLO surface (time-to-exhaustion at current ingress, time-to-restore under surviving egress budgets, admission warning with optional non-critical upload pause, separate reserves per replication class — restoring full redundancy after losing 40% of capacity from survivors alone requires 0.6C ≥ U); delayed hedged donor reads on the P2-M4.1 read path (start the highest-ranked donor, start a second fetch at the p95 threshold, take the first verified envelope — at a 5% slow/failure rate with independent sources, a ~20× reduction in the tail event while extra traffic stays near the hedge-trigger rate rather than doubling every read); automated warm-standby validation without automated promotion (verify WAL freshness, verify every master-key version is present, verify standby federation certificates, exercise restore/failover quarterly — promotion stays manual until Phase 6 fencing exists); and periodic encrypted signed authority recovery checkpoints (Postgres base backup + WAL position, blob/manifest index, key-version inventory, configuration and federation CA state, and the result and timestamp of the last restore test, with the existing offline Shamir escrow guarding the recovery key so only backup recovery is threshold-gated and the running coordinator stays operator-controlled). Quorum-before-commit already ships default-off, so it is a /settings surfacing and recommendation item rather than new code. P2-M11 also tightens HEALING_PROTOCOL.md's R-default rationale, which currently reads as though a high default R answers the 40%-failure result on its own (see the durability-claims discipline note under Phase 6).

Not in scope anywhere. The findings' "Operator to-do" — registering a designated agent with the U.S. Copyright Office for the § 512(c) safe harbour — is an operator action for a specific deployment, not a Nova issue.

v2 additions: - HTTPS+mTLS auth inside Nebula with separate federation client certs (Nebula cert authorizes overlay; federation cert authorizes HTTP API). - Donor-to-donor controlled repair transport with HMAC-signed, source-and-destination-pinned repair tokens. No Bitswap-backed repair fetch. - Five-state node liveness (active / suspect / unreachable / evicted / revoked) with separate timers; healing engages at unreachable (~1h), not at evicted (30d). - Possession audits (per POSSESSION_AUDIT.md): challenge-response spot-checks, donor reputation tracking, audit-aware placement. - Incremental change-log endpoint (/fed/v1/pins/changes) plus snapshot recovery path with snapshot_epoch consistency. - pkg/node graduates to a public, semver-stable Go library alongside cmd/node.

v3.1 promotions into Phase 2: - Streaming-AEAD envelope (v2 wire format). Chunk size aligned to the existing 256 KiB IPFS block boundary so chunk N == block N; per-chunk XChaCha20-Poly1305 with chunk-counter-derived nonces; AAD binds chunk_index || total_chunks || cid to defeat reordering and substitution. Encrypted blobs become Range- serveable. See docs/specs/ENCRYPTION_ENVELOPE.md § "Planned v2: Streaming-AEAD". This was previously listed as Phase 6+ research; pulled forward because single-shot AEAD restricts nova-video, nova-audio, large nova-archive objects, and modern web media patterns to full-object fetch. Federation is the right pairing because the per-block crypto semantics share infrastructure with possession audits and donor-to-donor repair.

Phase 3 — Dedup and moderation

Go-native 256-bit perceptual hash (pHash, goimagehash ExtPerceptionHash) index and BK-tree for near-duplicate detection and dedup. Content-moderation pipeline scaffolding.

Phase 4 — Adapters, SDKs, and severe-content workflow

Adapter packages for fediverse and forum software (separate repositories). Auto-generated client SDKs in TypeScript, Python, Swift.

v2 addition: full severe-content workflow per SEVERE_CONTENT_PROCEDURE.md: - PDQ hash computation and scan against the StopNCII/NCMEC external blocklist at upload (synchronous reject for clear matches, quarantine + legal-hold for ambiguous). Note: PDQ is distinct from the Phase-3 Go-native pHash — PDQ is for external blocklist matching only, not dedup. - NCMEC CyberTipline report generation. - Admin SPA legal-hold clearance UI. - Audit-log export for evidence packaging.

Phase 5 — Hardening

Chaos testing, security audit, documentation polish, public 1.0.

Phase 6 — Multi-coordinator, single-authority HA (post-1.0)

Remove the coordinator as an availability single point of failure without ever allowing two authorities to diverge. Several active coordinators behind redundant ingress read one strongly-consistent Postgres authority (primary + fenced streaming standbys); exactly one fenced control-plane leader (monotonic control-term token) runs orchestration, liveness transitions, audits, lifecycle sweeps, master-key rotation, and cert revocation. Reads and donor-API traffic are active-active. Builds on and automates the manual docs/recipes/COLD_STANDBY.md pattern with mechanical fencing.

Groundwork (surfaced by the second-pass resilience analysis so it is not built as accidental tech debt): job-queue + control-plane fencing tokens (lease_id/generation, coordinator_leases(term)), origin-location tracking with a transactional outbox for the Kubo-pin/Postgres-commit boundary, multi-endpoint donor config with since_seq cursor preservation, replicated or shared upload staging, cross-instance signed-URL revocation, and redundant Nebula lighthouses + Kubo bootstrap peers. Reframes T1.27; explicitly rejects independent writable masters. Design + simulation evidence: docs/superpowers/specs/phase6/2026-06-12-resilience-and-post-1.0-architecture-design.md.

Sequencing (2026-08-08, from the first-deployment architecture analysis). Implement in the order HA → peering → erasure coding. That is not the order they are usually discussed in; the reason is that each fixes a different failure and only HA is on the availability path. HA is the largest availability win, and Nova's own simulation shows adding coordinator processes stalls after two because Postgres, keys and ingress remain the floor (1 coordinator 98.4459% / 136.2 h·yr⁻¹; 2 → 99.2335% / 67.2 h; 3 → 99.2413% / 66.5 h). With this phase's requirements actually met — replicated fenced Postgres 99.99%, redundant ingress 99.99%, key availability 99.999% — two active coordinators reach ≈99.969% (2.7 h/yr) and three ≈99.979% (1.9 h/yr); under the simulation's 20% correlated-failure assumption both fall to ~20 h/yr, which is the real finding: provider and configuration diversity matters more than the third replica. Two coordinators is the target; a third buys little until evidence shows two-host maintenance overlap is hurting the SLO. Throughput is not the reason to do this — one 1 Gbps coordinator serves 9.82 TiB/day against a year-five average of 0.97 TiB/day, roughly 10× headroom — availability, maintenance windows and burst are.

Durability-claims discipline (applies to all public and internal wording). R=5 does not mean the corpus survives losing 40% of nodes. Under independent uniform loss, P(all five copies lost) = 0.4⁵ = 1.024% — 98.976% survival per object, which across ~34 million blobs at 50 TiB is hundreds of thousands of unrecoverable objects. R=5 tolerates four failed holders for a given object; corpus-level survival is a property of placement diversity, repair speed, and whether an external copy exists — which is precisely why diversity rebalancing (P2-M11) and peering (Phase 7) carry the weight that raising R does not. Any claim about surviving a percentage of node loss must be worded per-object or backed by the placement/repair/peering argument.

Phase 7 — Opaque inter-federation replica peering (post-1.0)

Off-site durability and disaster recovery across operators without merging trust domains. A peer/v1 protocol (distinct from donor fed/v1) in which a peer stores opaque ciphertext only — never keys, plaintext, catalog, moderation state, or assignment history. Invariants: every object has exactly one home federation; peers count as at most one failure domain each (lease- and audit-gated); no transit / no re-export without home authorization; signed, generation-ordered tombstones propagate crypto-shred even to peers that no longer hold the object; optional encrypted DR packages (Postgres base backup + WAL + manifests, encrypted under a recovery key the peer does not hold) turn peering from ciphertext durability into full federation reconstruction. Peering replicates bytes, not authority. Reframes T1.28.

Why this is second, and two constraints that are not optional (2026-08-08). Peering is the largest disaster win because it covers the one failure replication cannot: every local holder disappearing together. From the Phase-7 simulation, under deliberately fragile bandwidth-weighted placement, no peers leaves 31,847 of 50,000 objects with zero local holders (48 GiB of healing egress) while two opaque custodians leave 0 (189 GiB). The "slower recovery" with peers is an accounting artefact — without them the lost objects are simply abandoned and generate no repair traffic. Current diversity-optimised placement already cuts local zero-holder exposure from ~64% to ~7.5%, so at 50 TiB a peer reseeds ~3.75 TiB rather than 32 TiB — 9.2 hours at 1 Gbps rather than 3.3 days, with 2–3× line rate budgeted for real conditions. Cost per full-corpus peer: initial seed up to 50 TiB, ~10.1 GiB/day ongoing, +20% network storage relative to local R=5, no change to read throughput or the coordinator bill. The two constraints: peering must ship with the encrypted DR packages, because opaque corpus bytes alone recover nothing (Postgres, wrapped DEKs, manifests, assignment history and the master key all have to travel too, separately encrypted); and peers must not count toward the home federation's local R floor — peering is the disaster layer, not a hidden runtime dependency.

Phase 8+ — Research

Speculative directions: end-user client direct integration, browser-resident pinning via WASM, FFI bindings for non-Go embedding, additional product modules (nova-video, nova-audio, nova-archive, nova-document), formal Provable Data Possession / Proof of Retrievability, hot-tier / cold-tier auto-migration, optional S3 read-only adapter product layer, erasure coding for large archival objects.

Erasure coding — a capacity optimisation, and a trap if done cheaply (2026-08-08). It goes last, after HA (Phase 6) and peering (Phase 7), and it stays a selective cold-capacity layer rather than a universal substrate. Do not ship a 1.5× code while 40% simultaneous loss is the advertised target: at that overhead more than half the stripes become unrecoverable (RS(6+3) 48.261% survival, RS(8+4) 43.818%, against R=5's 98.976% per object). Matching R=5's tolerance costs 2.5–3× (RS(5+10) 3.0× / 99.065%, RS(12+18) 2.5× / 99.170%), which still saves 40–50% of federation storage and ingest traffic — 2,560 donor nodes at 100 GiB each becomes 1,280–1,536 — and coalesced reconstruction could cut mass recovery from ~200 TiB to ~100–110 TiB. The catch, and the reason it goes last: for ordinary one-node churn Reed–Solomon is worse, because rebuilding one fragment reads k surviving fragments — roughly 3× replication's repair traffic per lost byte for RS(5+10), ~6.5× for RS(12+18) — while metadata scales from 170M assignment rows at R=5 to 511M or 1.02B, and a cache-miss read becomes a k-way fan-out. LRC or regenerating codes reduce the repair penalty; Nova would need its own heterogeneous-link benchmarks before choosing, and no choice should be made before real size/churn/cache traces exist. Shape when it does land: hybrid policy — R=5 for small/hot originals, R=2–3 or cache-only for regenerable derivatives, EC for large cold originals, replication plus peer custody for important ones. Apply EC after encryption so fragments stay donor-blind, give every fragment a deterministic CID, store a signed versioned stripe manifest, bind repair grants to (stripe, fragment index, assignment generation, source, destination), let a capable destination reconstruct rather than routing bytes through the coordinator, verify the original envelope root before decryption, align stripes with streaming-AEAD chunks for Range reads, and preserve crypto-shred by deleting the DEK and propagating fragment tombstones. Making EC the universal substrate saves the most bytes and increases metadata scale, repair fan-out, tail latency and operational complexity precisely where volunteer infrastructure is weakest.

(v3.1: streaming-AEAD envelope was promoted from Phase 6+ research to a Phase 2 deliverable. v3.2 — 2026-06-12: multi-coordinator HA and inter-federation peering were promoted from the Phase 6+ research grab-bag into deliberate post-1.0 Phases 6 and 7, and the remaining research items renumbered to Phase 8+; the earlier "read-only secondary coordinator" research line is superseded by Phase 6. See the 2026-06-12 resilience design.)

Source: docs/ROADMAP.md