Plan v2: public ledger, create-time PIN locks, and expiring pages

Target branch: feat/page-pin-locks, cut from master at 7ad6b01 (PR #15 merged).

This plan supersedes v1 of Plan: Per-page PIN locks. It incorporates the code review and the subsequent product/security decisions. It is intended to be executable by another agent without needing the design conversation that produced it.

1. Outcome and product contract

Agent-pages becomes a public publish ledger whose page bodies may optionally be protected by a fixed six-digit PIN and whose pages may optionally expire.

1.1 Listing and access

• The ledger at / and /page/{n} is public. It lists every active page, including PIN-locked pages.
• A locked row exposes the page title, slug, format, update time, expiry (when set), and the fact that it is locked. The PIN protects the body, version history, and raw representation—not the existence or title of the page.
• An open page requires no viewer credential.
• A locked page uses exactly six ASCII digits (^[0-9]{6}$). PIN length is fixed in code and is not configurable.
• PINs are set only when a page is created. They cannot be added, removed, or changed later.
• Content/title updates preserve the original access state and expiry.

1.2 MCP publication contract

publish_page gains locked: bool = False, pin: str | None = None, and expiry (default never). Its behavior is exact:

The MCP server—not the language model—generates a missing PIN:

generated_pin = f"{secrets.randbelow(1_000_000):06d}"

This includes leading zeroes. A successful generated-PIN tool result must say, plainly:

Published locked page at https://…
PIN: 004821

It must also include generated_pin in structured tool output. generated_pin appears only when MCP generated it; a user-supplied PIN is not echoed redundantly. Later API/MCP reads never return a PIN.

Generate once before the HTTP call and do not automatically retry an ambiguous publish transport failure. If the MCP-to-API result is ambiguous, the tool error should say the outcome is unknown and repeat the generated PIN so the operator can recover the newest page through list_pages. Idempotent publish remains out of scope.

1.3 Direct API publication contract

POST /v1/pages accepts optional pin and expiry. The direct API does not generate PINs; callers wanting a locked page must supply one. The MCP server resolves locked into a concrete PIN before calling the API.

Examples:

{ "title": "Open", "format": "markdown", "content": "…" }

{
  "title": "Locked",
  "format": "html",
  "content": "…",
  "pin": "004821",
  "expiry": "1_week"
}

Accepted expiry values:

Store an absolute UTC expires_at; do not store the duration label. Updates cannot extend or change it.

1.4 Expiration contract

As soon as expires_at <= now, before physical cleanup runs, a page:

1. Disappears from the public ledger and its counts.
2. Disappears from authenticated API/MCP get_page and list_pages.
3. Returns the normal styled 404, not 410, from latest, historical, raw, and unlock URLs. This deliberately does not confirm that the page used to exist.
4. Rejects updates with API 404.
5. Performs no R2 body read on any rejected viewer request.

Physical deletion is a separate, retryable CLI operation run by host cron. There is no scheduler or background cleanup loop inside FastAPI.

1.5 Editing authority and PIN boundary

The PIN is a viewer credential only. It is not an ownership or edit credential.

• PUBLISH_API_TOKEN remains instance-wide administrative authority.
• The MCP client needs AGENT_PAGES_MCP_AUTH_TOKEN; the MCP service uses AGENT_PAGES_API_TOKEN/PUBLISH_API_TOKEN downstream.
• Anyone with a usable publishing path can call authenticated list_pages, obtain UUIDs, and update any unexpired page without its PIN.
• The public ledger exposes slugs, not page UUIDs.
• A slug, a PIN, or both never authorize an update.
• Updates remain PUT /v1/pages/{page UUID} and require the publish bearer.

This is accepted for the current single-owner personal instance. Per-user ownership, per-page edit secrets, create-only tokens, and multi-tenant authorization are out of scope and must not be implied by the PIN feature.

2. Threat model and security choices

2.1 Six-digit PINs

A uniformly generated PIN has 1,000,000 possibilities (~19.9 bits). User-provided PINs may have lower effective entropy; accept every six-digit value, including 000000, without a weak-PIN blocklist. Document that generated PINs are preferable for sensitive pages.

The online defenses are per-client throttling and bounded cryptographic work. There is deliberately no instance-wide per-page counter: a global page counter would let any anonymous visitor lock everyone out by spending the page budget repeatedly.

2.2 Two client-scoped rate limits

Use the existing atomic fixed-window SQLite implementation and publish_rate_limits table, generalized enough that PIN error messages do not say “publish.” Consume both counters before scrypt verification:

1. Page + client IP: 5 attempts per 15 minutes.
   • Key: sha256("pin-unlock-page-ip:" + page_id + ":" + client_ip).
   • Limits brute force against one page from one network identity.
2. Global client IP: 30 attempts per 15 minutes across all pages.
   • Key: sha256("pin-unlock-ip:" + client_ip).
   • Bounds one address attempting many pages and bounds local resource consumption.

Both count submitted unlock attempts, including a successful one. Once unlocked, the cookie avoids further submissions. Check the global-IP budget first, then page+IP, so a client sweeping pages cannot create/use a fresh page bucket to bypass the global ceiling.

Suggested settings (PIN length itself is not a setting):

pin_unlock_rate_limit_max = 5
pin_unlock_rate_limit_global_max = 30
pin_unlock_rate_limit_window_seconds = 900
pin_unlock_hash_concurrency = 4
pin_unlock_hash_wait_seconds = 0.25
pin_unlock_cookie_ttl_seconds = 604800  # 7 days

Environment aliases should follow the existing uppercase pattern. Validate every count/window/concurrency value as >= 1 and wait as >= 0.

Move resolve_client_ip() and trusted-proxy helpers out of app/viewer/session.py into a neutral module such as app/client_ip.py, because the index session is being retired but PIN limiting still needs the reviewed rightmost-walked XFF logic.

• Untrusted peers cannot control XFF.
• A trusted proxy with missing/unusable XFF uses its peer address as a shared bucket with the same conservative ceilings.
• A missing request.client becomes an explicit shared unknown bucket; it must not receive a looser allowance.
• Preserve/add the production warning when HTTPS is configured while TRUSTED_PROXY_CIDRS is empty. Log during application startup/lifespan, not as an import side effect.

2.3 Rate-limit row retention

Every distinct rotating IP creates counter keys. Expired fixed-window rows do not disappear automatically. Reuse the indexed probabilistic purge and calculate retention using every feature sharing the table:

retention_seconds = max(
    settings.publish_rate_limit_window_seconds,
    settings.pin_unlock_rate_limit_window_seconds,
)

The index sign-in window disappears when index auth is removed. Page+IP keys also rotate, so invoke best-effort cleanup from the PIN attempt path. Cleanup failure must never turn an otherwise valid attempt into 500.

2.4 Bounded scrypt work

Scrypt is intentionally memory-hard. With the chosen parameters each operation uses roughly 16 MiB. A rate limit controls requests over time but does not prevent many first requests from arriving simultaneously across different IPs/pages.

Add one process-wide threading.BoundedSemaphore(4) around both hash creation and verification. This deployment is one API process, so four slots bound concurrent expected scrypt memory near 64 MiB.

Unlock order:

1. Validate six ASCII digits before any expensive work.
2. Consume global-IP and page+IP counters.
3. Acquire a scrypt slot with a short bounded wait (default 250 ms).
4. If no slot is available, return a retryable 503 with Retry-After: 1; the already-consumed attempt remains consumed.
5. Run scrypt and constant-time comparison.
6. Release the semaphore in finally on every path.

Publishing a locked page also uses the semaphore. An authenticated publish that cannot acquire capacity returns a retryable 503 and must not upload to R2 or create DB metadata.

If multi-worker/multi-host deployment is introduced later, the limit is per process and must be divided by worker count or moved to shared infrastructure.

2.5 PIN hashing

Add app/pins.py with a self-describing stored format:

scrypt$16384$8$1$<base64-salt>$<base64-derived-key>

Use:

N = 2**14
R = 8
P = 1
DKLEN = 32
SALT_BYTES = 16

• Use hashlib.scrypt, a random per-page salt, strict base64 decoding, and hmac.compare_digest on derived bytes.
• Reject malformed/unknown stored formats safely rather than raising into a viewer 500.
• Bound parsed numeric parameters before invoking scrypt so corrupt/malicious DB values cannot request extreme memory. Accept only the reviewed parameter set in v1.
• Never log a supplied PIN, hash, cookie, or generated derived key.
• A DB leak still permits offline enumeration of a six-digit PIN; scrypt raises its cost but is not equivalent to a high-entropy secret.

2.6 Unlock cookie

Use one page-scoped opaque cookie:

name:  agent_pages_unlock
value: {expires_at}.{HMAC(pin_hash, "agent-pages-unlock-v1|" + slug + "|" + expires_at)}
path:  /p/{slug}
flags: HttpOnly; SameSite=Lax; Secure when PUBLIC_BASE_URL is HTTPS
TTL:   7 days, no sliding renewal

The server enforces the signed absolute expiry, not only Max-Age. Changing a PIN is unsupported; physical deletion removes the validating row. Bind the slug into the MAC and compare cookies in constant time.

Keep the cookie path without a trailing slash so it reaches /p/{slug} as well as nested version/raw routes. Add the RFC 6265 prefix test proving a cookie for foo-a1b2c3d4 is not sent to foo-a1b2c3d4-e5f6g7h8.

3. Data model

3.1 pages additions

pin_hash: Mapped[str | None] = mapped_column(String(255), nullable=True)
expires_at: Mapped[datetime | None] = mapped_column(
    DateTime(timezone=True), nullable=True, index=True
)
purge_started_at: Mapped[datetime | None] = mapped_column(
    DateTime(timezone=True), nullable=True
)

• pin_hash IS NULL means open; non-NULL means locked.
• PIN is page-level, so every historical version is protected consistently.
• expires_at IS NULL means never expires.
• purge_started_at is a cleanup claim/lease and update-race guard.
• Do not add pin_set_at: PINs are create-only and it has no consumer.

The legacy visibility column remains physically present for migration compatibility but is deprecated and ignored for listing/access decisions. Add comments/docs making clear that locked pages are still publicly listed. Do not undertake a SQLite table rebuild merely to drop it in this change.

3.2 Durable deletion record and attempts

Use two tables so the unique slug reservation and append-only attempt history do not conflict.

page_deletion_logs — one durable record per expired page:

id                  UUID primary key
page_id             UUID snapshot, no FK (source row is deleted)
slug                VARCHAR(128) UNIQUE NOT NULL
expires_at          DATETIME NOT NULL
first_attempted_at  DATETIME NOT NULL
completed_at        DATETIME NULL
status              pending | deleted
version_count       INTEGER NOT NULL
objects_deleted     INTEGER NOT NULL default 0
attempt_count       INTEGER NOT NULL default 0
last_error_summary  TEXT NULL, capped/sanitized

page_deletion_attempts — append-only details for each CLI attempt:

id                  INTEGER primary key/autoincrement
deletion_log_id     FK page_deletion_logs.id, RESTRICT
attempted_at        DATETIME NOT NULL
completed_at        DATETIME NOT NULL
status              succeeded | failed
objects_attempted   INTEGER NOT NULL
objects_deleted     INTEGER NOT NULL
error_summary       TEXT NULL, capped/sanitized

Do not store page bodies, PINs, PIN hashes, cookies, or full exception traces in either table. Application logs may carry tracebacks; DB error summaries must exclude credentials and be capped.

page_deletion_logs.slug is the permanent tombstone. Slug allocation checks active pages.slug and tombstoned page_deletion_logs.slug. An expired URL can therefore never resolve to unrelated future content.

3.3 Expiration helper

Centralize UTC/SQLite-naive handling and active predicates. A page is active only when:

Page.expires_at.is_(None) | (Page.expires_at > now)

Use the same definition in viewer lookup, API get/list, index stats/list, and update checks. Cleanup deliberately queries the inverse. Avoid scattering subtly different comparisons across modules.

4. Migration mechanics

create_all() does not add columns to existing SQLite tables. Extend agent-pages init-db after Base.metadata.create_all():

1. Inspect PRAGMA table_info(pages).
2. Add missing nullable columns individually:

ALTER TABLE pages ADD COLUMN pin_hash VARCHAR(255);
ALTER TABLE pages ADD COLUMN expires_at DATETIME;
ALTER TABLE pages ADD COLUMN purge_started_at DATETIME;

1. Create the expiry index explicitly for existing volumes:

CREATE INDEX IF NOT EXISTS ix_pages_expires_at ON pages (expires_at);

1. Let create_all() create both new deletion tables on fresh and existing databases; explicitly assert required indexes/uniques in tests.
2. Preserve the existing explicit publish_rate_limits.window_start index creation.

No data backfill is needed:

• Existing pin_hash = NULL → open.
• Existing expires_at = NULL → never expires.
• Existing purge_started_at = NULL → not being purged.
• The now-public ledger lists every existing active test page immediately.

Run init-db twice in the migration test. Both runs must succeed and leave one copy of every column/index/table.

This is another hand-written schema evolution. Record in MEMORY.md that the next nontrivial schema change should introduce Alembic rather than extending the DDL block again.

5. API and service changes

5.1 Schemas

Add:

ExpiryOption = Literal["never", "1_day", "1_week", "1_month", "1_year"]

PageCreateRequest gains:

pin: str | None = None
expiry: ExpiryOption = "never"

Validate PIN with an ASCII-specific full match, not \d, which accepts Unicode digits. Continue stripping titles and enforcing UTF-8 body size.

PageSummary gains:

has_pin: bool
expires_at: datetime | None

PageResponse gains the same two fields. No schema contains pin_hash, raw PIN, purge_started_at, or deletion internals.

PageUpdateRequest does not gain PIN or expiry fields. Unknown fields remain rejected according to the chosen Pydantic model policy; add an explicit test that attempts to mutate either cannot silently succeed.

5.2 Create service

In create_page:

1. Consume the authenticated publish limit.
2. Convert expiry label to one absolute UTC timestamp using a captured now.
3. If PIN supplied, acquire the bounded scrypt slot and hash before any R2 upload. Capacity failure is retryable and leaves no object/row.
4. Plan and upload version 1 using the existing unique R2 key and compensation discipline.
5. Generate the slug and check both active page and deletion-log reservations.
6. Insert Page, including pin_hash and expires_at, plus PageVersion atomically.
7. Retain unique-conflict retries for concurrent slug allocation.

Define a specific SlugAllocationError. After the existing bounded retry count is exhausted, compensate the uploaded object and return a retryable API response such as 503 with an actionable detail (Could not allocate a unique page URL; retry the publish). MCP must preserve that meaning. Do not return a misleading content-validation error.

5.3 Update service

Updates continue to create immutable content versions. Add lifecycle checks twice:

1. Before the R2 upload, reject missing, expired, or purge_started_at != NULL as PageNotFoundError.
2. After upload, when reacquiring the page for the version commit, recheck expiry against a fresh now and confirm purge_started_at IS NULL. If it became expired/claimed during network I/O, roll back and compensate only the newly uploaded unique key.

Updates preserve pin_hash and expires_at exactly. They do not renew the unlock cookie, change access, or extend lifetime.

5.4 Authenticated get/list

• GET /v1/pages/{id} returns 404 for expired/claimed/deleted pages.
• GET /v1/pages excludes expired/claimed pages and its total counts only active pages.
• Responses expose has_pin and expires_at, never secret material.

5.5 Authority tests

Keep and explicitly test the boundary:

• Slug only → cannot update.
• Slug + correct viewer PIN → cannot update.
• UUID without publish bearer → 401.
• UUID with wrong bearer → 401.
• UUID with valid publish bearer → update succeeds without viewer PIN while active.
• Valid bearer + expired UUID → 404.
• Public ledger markup does not contain page UUIDs.

6. Public ledger migration

Remove the opt-in visibility switch and make / public in this deployment.

6.1 Remove index authentication surface

• / and valid /page/{n} render the ledger without a cookie.
• Remove sign-in and sign-out controls/templates from navigation.
• Retire POST /session and POST /session/signout; they should fall through to the styled 404.
• Remove AGENT_PAGES_INDEX_TOKEN, index cookie signing, and index sign-in rate-limit settings/docs.
• Delete index session code after moving reusable client-IP resolution to app/client_ip.py.
• Preserve path-based pagination, UTC grouping, the Page.id secondary sort key, mobile table semantics, and no-JavaScript behavior.

6.2 Ledger data and presentation

• Index queries and index_stats filter active, non-purging pages using the shared predicate.
• Locked rows get a mono lock sigil plus real accessible text (Locked); do not rely on glyph/color alone.
• Show expiry in UTC when present and No expiry/omit the marker when absent. Keep the compact mobile layout usable and update explicit grid-row assignments and ARIA roles if a column is added.
• The format remains the only format color. A lock/expiry marker should not introduce a competing semantic palette.
• Keep X-Robots-Tag: noindex, nofollow, noarchive. Publicly reachable does not mean search-engine discovery.
• The anonymous page/count secrecy rules are intentionally retired.

The physical legacy visibility column is not consulted. Update AGENTS/MEMORY language from “unlisted” to “public ledger; optionally body-locked.”

7. Viewer gating and unlock flow

7.1 Gate every body-bearing route

All four routes must enforce active lifecycle and PIN access before R2:

Check lifecycle before access so an unlock page or cookie cannot reveal an expired page. Check access before storage.get_object(). Refactor load_page_view into metadata resolution plus explicit body loading, or otherwise provide one auditable boundary; do not fetch metadata twice merely to avoid the refactor.

HTML host behavior remains two requests: the unlocked outer shell loads immutable /v/{n}/raw; the raw request independently verifies the cookie.

7.2 Unlock routes

Add app/viewer/unlock.py:

The form may show title and locked state, since both are already public. It shows no body, versions, PIN hint, hash detail, or expiry-internal state. Use inputmode="numeric", a six-character limit/pattern, and accessible error text. Server validation remains authoritative.

Validate next more narrowly than the global theme helper: it must be the canonical, historical, raw, or historical raw URL for the same slug. Reject other pages and all schemes, hosts, query strings, fragments, backslashes, and control characters. Default to /p/{slug}.

Use the existing form-enabled shell CSP after renaming it from home-specific terminology if helpful. Only unlock and any remaining legitimate shell form may use form-action 'self'. Raw/artifact CSP remains unchanged: sandbox, script-src 'none', font-src 'none', form-action 'none'.

7.3 Cache safety

Every /p/* response—open or locked, shell or raw, success/error/redirect/unlock—gets:

Cache-Control: private, no-store
Vary: Cookie

This prevents unlocked content crossing credential states and prevents cached open content surviving expires_at. Preserve base security headers and CSP on every response. Ensure global error handlers add the same cache policy for /p/* validation, 404, 405, and 500 responses.

7.4 Browser verification

Before declaring the HTML path complete, verify in real Chrome and Firefox that the page-scoped SameSite=Lax cookie is attached to the same-site request for an iframe with sandbox="". The expected behavior is that sandboxing makes the resulting document origin opaque but does not strip cookies from the same-site network request.

If measurement disproves that expectation, stop and redesign; do not silently add unlock credentials to query strings. Any token fallback requires separate review because URLs leak through copying/history/logs.

8. MCP client and tools

8.1 publish_page

FastMCP signature:

async def publish_page(
    title: str,
    format: PageFormat,
    content: str,
    locked: bool = False,
    pin: str | None = None,
    expiry: ExpiryOption = "never",
) -> CallToolResult:

Resolution:

generated_pin = None
effective_pin = pin
if locked and effective_pin is None:
    generated_pin = f"{secrets.randbelow(1_000_000):06d}"
    effective_pin = generated_pin
# effective_pin being supplied implies locked even when locked=False

Forward pin only when effective_pin is non-NULL and forward expiry. API validation remains canonical. Do not implement scrypt or database logic in MCP; it stays a thin HTTP client.

Successful structured content includes normal page metadata (id, slug, url, version, has_pin, expires_at) and conditionally generated_pin. Ensure the human-readable result prominently returns the generated PIN with the URL. Tool docs warn that supplied/generated PINs appear in the agent transcript and must be shared appropriately.

8.2 update_page

Do not add PIN or expiry parameters. Its docstring explicitly says:

• requires UUID and publishing authority;
• cannot change PIN/expiry;
• fails after expiry;
• creates a new immutable version.

8.3 get_page and list_pages

Surface has_pin and expires_at; exclude expired pages according to the API. Keep page UUIDs in authenticated MCP results. Document that publishing credentials are administrative and permit updates without viewer PINs.

9. Physical cleanup CLI and host cron

9.1 Command

Add:

agent-pages purge-expired --batch-size 100 --claim-timeout-seconds 3600

Exit behavior:

• 0 when the batch completes with no failures, including nothing due.
• nonzero when any claimed page fails cleanup, so cron monitoring can alert.
• Print concise structured/log-friendly totals: examined, claimed, deleted, failed, objects deleted.
• Never print bodies, PINs, hashes, cookies, or R2 credentials.

Document a host cron example (frequency may be adjusted operationally):

*/15 * * * * cd /path/to/agent-pages && docker compose exec -T app agent-pages purge-expired --batch-size 100

The app container already has the SQLite volume and R2 credentials. No second long-running Compose service and no in-app scheduler are required.

9.2 Claim and retry algorithm

For each eligible page in deterministic expiry/id order:

1. In a short SQLite transaction, conditionally claim it only when expires_at <= now and purge_started_at is NULL or older than the claim-timeout cutoff. Set purge_started_at = now atomically. A stale lease allows recovery after process death.
2. Create or load the durable page_deletion_logs row for page ID/slug. The unique slug begins permanent reservation immediately. Increment/record attempt state without losing prior attempts.
3. Read every committed PageVersion R2 key and expected count after the claim.
4. Delete every key. Treat an already-missing object as success; R2 deletion is idempotent. Continue attempting remaining keys after one failure so the attempt record is complete.
5. Append one page_deletion_attempts row with sanitized result counts/error.
6. On any object failure: update the durable log to pending/error, clear purge_started_at (or allow lease expiry if the DB write itself fails), commit the attempt, and retain Page/PageVersion metadata for retry.
7. On total R2 success: in one DB transaction mark the durable log deleted/completed, append the successful attempt, and delete Page; versions cascade. The tombstone remains forever.

Crash properties:

• Crash before claim commit: another run claims normally.
• Crash after claim: stale lease reclaims.
• Crash after some/all R2 deletes: retry treats missing keys as success.
• Crash before Page deletion: metadata retains the authoritative key list.
• DB failure after R2 success: retry repeats idempotent deletes, then finalizes.

Never delete the Page row before every R2 key is successfully removed; doing so loses the cleanup manifest.

9.3 Update/cleanup race

The cleanup claim and update’s final lifecycle recheck form the boundary:

• Cleanup claims only expired pages.
• An update started before expiry may upload outside the DB lock.
• At commit, update rechecks current time and purge_started_at. If the page expired or was claimed, it aborts and compensates its unique new R2 key.
• Cleanup sees only committed PageVersions after its claim.

Add a deterministic test that pauses update between upload and commit, claims expiry, resumes update, and proves the new object is compensated and never becomes an orphan/version.

10. Slug allocation and permanent reservation

Current slugs are readable title bases plus eight random lowercase/digit characters (36^8 possibilities). Preserve this format.

Allocation must reject a candidate present in either:

pages.slug
page_deletion_logs.slug

Keep database uniqueness as the final concurrency guard. Because the tables are separate, perform both existence checks during each retry; the deletion-log unique constraint permanently protects tombstones. A creator cannot submit a slug through API/MCP.

Tests monkeypatch slug generation to cover:

• collision with an active page → retry and succeed;
• collision with a pending/deleted audit slug → retry and succeed;
• all retries collide → no DB page/version, uploaded object compensated, meaningful retryable API/MCP error;
• after physical deletion, requesting the old viewer URL remains 404 while a later same-title page receives a different slug.

11. Detailed test plan

Create focused files rather than one monolith:

11.1 tests/test_pins.py

• Hash round trip for values including 000000 and 999999.
• Different salts produce different stored strings for the same PIN.
• Wrong PIN rejects.
• Non-six-digit/API Unicode-digit input rejects before hashing.
• Malformed, truncated, unknown-algorithm, invalid-base64, and out-of-bounds-parameter stored hashes reject safely.
• Comparison uses derived bytes; patch/spy where practical.
• Semaphore releases after success and exception.
• Fifth concurrent operation cannot enter when four slots are occupied and receives the retryable capacity result.

11.2 tests/test_pin_publish.py

• API no PIN → has_pin=false, DB NULL, page open.
• API valid PIN → has_pin=true, DB stores a hash that contains neither plaintext PIN nor recoverable raw value.
• API rejects five/seven digits, whitespace, signs, Unicode numerals, numbers instead of strings.
• Every expiry option maps to the exact fixed duration; omitted/never stores NULL.
• Response includes has_pin and expires_at, never PIN/hash.
• Hash-capacity failure happens before R2 put and DB insert.
• Content/title limits and R2 compensation behavior remain unchanged.

11.3 MCP tests (tests/test_mcp_server.py and client tests)

• false/omitted locked + no PIN forwards no PIN and publishes open.
• locked=true + no PIN patches secrets.randbelow to 4821, forwards 004821, and returns URL plus PIN: 004821 and structured generated_pin.
• supplied 004821 with locked omitted implies locked, forwards exactly it, and does not add generated_pin to output.
• locked=true + supplied PIN does not call generator.
• expiry is forwarded exactly.
• invalid supplied PIN error comes from API without MCP-side storage logic.
• generated PIN is generated once and no automatic HTTP retry occurs.
• ambiguous transport error identifies unknown outcome and preserves generated PIN in the error guidance.
• update tool has no PIN/expiry mutation arguments.
• get/list surface has_pin/expires_at and retain UUIDs only on authenticated tool access.

11.4 tests/test_pin_unlock.py

For each of the four viewer routes:

• open active page remains normal;
• locked/no cookie produces the specified 303 or raw 403;
• locked/forged, expired, truncated, other-page cookie rejects;
• locked/valid cookie produces expected content;
• page A cookie rejects for page B;
• current and every historical version use the same page PIN;
• HTML host iframe raw request succeeds with cookie in server tests;
• unlock GET open/already unlocked redirects to validated same-page target;
• unknown and expired unlock URLs return indistinguishable styled 404s;
• invalid next target defaults/rejects safely and cannot redirect to another slug.

Instrument FakeR2Storage with get_calls; objects alone does not record reads. Assert get_calls == [] whenever lifecycle/access gates reject before body fetch.

11.5 tests/test_pin_rate_limit.py

• Page+IP allows exactly five attempts, sixth gets 429 with Retry-After.
• Different IP on same page has an independent budget; there is no global per-page lockout.
• Same IP on different page has a separate page+IP budget.
• Global IP ceiling blocks the 31st attempt across many pages.
• Both budgets count a successful submitted attempt.
• Rate-limit rejection never calls scrypt.
• Hash-capacity rejection returns 503 and releases no extra slot.
• Trusted rightmost XFF resolution, spoofed prepended XFF, missing/invalid XFF shared bucket, untrusted peer, IPv4/IPv6, and unknown request client behave as designed.
• Rotating-IP rows are purged after the longest active retention window; purge failure is best effort.
• Concurrent attempts cannot exceed atomic fixed-window counts.

11.6 tests/test_expiry.py

At expires_at - epsilon, expires_at, and expires_at + epsilon, cover:

• public ledger presence/counts;
• API get/list/total;
• MCP behavior through mocked API;
• latest Markdown and HTML host;
• historical version;
• latest raw and historical raw;
• unlock GET/POST;
• update acceptance/rejection;
• no R2 read on expired viewer paths;
• all expired viewer paths are styled 404, never 410 and never reveal expiry.

Use injected/captured time helpers rather than fragile wall-clock sleeps.

11.7 tests/test_purge_expired.py

• No-op batch exits success and writes no audit rows.
• Active and never-expiring pages are untouched.
• Eligible page is claimed, every version object deleted, Page/versions removed, durable log marked deleted, successful attempt appended.
• One R2 failure retains metadata/key manifest, records failed attempt/error summary, and retries successfully later.
• Already-missing R2 key is treated as success.
• Crash/stale purge_started_at lease can be reclaimed; fresh lease cannot.
• DB finalization failure after R2 success is safely retryable.
• Batch size and deterministic order work across multiple runs.
• PIN/hash/body/R2 credentials are absent from audit records and captured logs.
• Update-versus-cleanup race compensates only the update’s unique key.
• CLI exit code nonzero when any page fails and zero otherwise.
• Durable log slug is unique; attempt rows are append-only.

11.8 tests/test_slug_reservation.py

• Active collision retry.
• Pending/deleted tombstone collision retry.
• Retry exhaustion maps to actionable retryable response and R2 compensation.
• Deleted slug is never reused.

11.9 Public ledger/session tests

Rewrite test_home_index.py/test_home_session.py expectations:

• Anonymous / renders rows and counts.
• Anonymous path pagination works and preserves tie-break ordering.
• Existing pages are public immediately after migration.
• Locked marker has accessible text; public markup contains slug/title but no UUID, PIN, or hash.
• Expired/claimed rows and counts disappear.
• /session and /session/signout are styled 404s.
• No index cookie or token environment dependency remains.
• Mobile grid/table ARIA structure remains correct.

11.10 Security/cache/CSP tests

• Every /p/* response class includes Cache-Control: private, no-store and Vary: Cookie: open, locked, unlock, raw, historical, redirects, 401/403/404/405/422/429/500.
• Unlock CSP allows form-action 'self' but no script.
• Raw/artifact CSP remains sandbox, script-src 'none', font-src 'none', form-action 'none' and never loads shell assets.
• Markdown/Mermaid/HTML sandbox boundaries remain unchanged.
• Security headers remain on public ledger and cleanup-independent errors.
• Correct PIN cannot authorize API update; only valid publish bearer can.

11.11 Migration tests

Construct a pre-feature SQLite database, insert representative existing Page/PageVersion rows and fake R2 bodies, then run init_db():

• New columns/tables/indexes exist.
• Existing pin_hash, expires_at, and purge_started_at are NULL.
• Existing pages are open, public in ledger, and bodies reachable.
• Running init_db() twice is idempotent.
• New tombstone uniqueness and attempt FK behavior work.

11.12 Manual browser acceptance

In Chrome and Firefox:

• locked Markdown unlock/read;
• locked HTML unlock → sandboxed iframe receives cookie and renders body;
• fullscreen immutable raw works after unlock;
• wrong PIN, rate-limit, expiry, and 404 presentation;
• light/dark/system shell themes;
• cookie path prefix isolation;
• back/refresh does not reveal cached body after expiry (also inspect response headers).

12. Success criteria

The feature is complete only when all are true:

1. Anonymous visitors can browse the active ledger without an index credential; legacy test pages remain open and visible after migration.
2. Open-by-default publication works through direct API and MCP.
3. A supplied six-digit PIN implies a locked page; invalid PIN shapes are rejected.
4. locked=true without a PIN causes MCP to generate a zero-padded cryptographically random six-digit PIN and return it prominently with the URL exactly once.
5. No plaintext PIN or PIN hash appears in DB plaintext fields, later responses, application logs, ledger HTML, or deletion audit data.
6. All current/historical/raw body routes gate locked content before R2 and unlock with one page-level cookie.
7. Expiration is enforced logically at the exact deadline across viewer, index, API, MCP, and updates, always presenting missing-page 404 semantics.
8. Every /p/* response is private, no-store and varies on Cookie; CSP/sandbox protections do not regress.
9. PIN attempts are bounded per page+IP (5/15m) and globally per IP (30/15m), with no globally attacker-consumable page lockout.
10. At most four scrypt operations run concurrently in the single API process and capacity exhaustion is retryable without memory growth.
11. Rate-limit rows from rotating IPs are eventually purged without affecting active counters or request success.
12. purge-expired removes all immutable R2 versions before deleting metadata, is crash/retry safe, exits observably, and records durable sanitized audit/attempt rows.
13. Deleted slugs remain reserved forever and can never resolve to different future content.
14. An update/expiry/cleanup race leaves no orphaned R2 object and never resurrects an expired page.
15. Viewer PINs never grant edit authority; update remains protected by instance publish credentials and UUID.
16. MCP remains an HTTP-only thin client with no DB/R2 imports.
17. Full python -m pytest passes, new migration tests pass from a pre-feature schema, and Chrome/Firefox iframe-cookie acceptance is recorded.

13. Recommended implementation sequence

Keep commits reviewable and preserve security boundaries at every step:

1. Browser spike: verify cookie attachment to opaque sandboxed iframe in Chrome/Firefox; record result.
2. Models + migration: Page columns, deletion log/attempt tables, indexes, old-DB idempotence tests.
3. Lifecycle helpers: central active/expired predicate/time handling; service query tests.
4. PIN module: fixed validation, scrypt serialization/parser bounds, semaphore, unit tests.
5. Create/API contract: pin hash + expiry calculation + response metadata + collision/tombstone checks.
6. MCP contract: locked/pin decision table, secure generation, one-time result, client forwarding/tests.
7. Public ledger: remove index gate/session, move IP resolver, public active filtering, lock/expiry UI.
8. Unlock route/cookie/CSP: form, validation, cookies, same-page next policy.
9. Four-route viewer gate: lifecycle then PIN then R2; cache policy across all /p/* responses.
10. Rate limiting: page+IP/global-IP counters, retention purge, trusted proxy warning, semaphore capacity errors.
11. Update lifecycle: pre/post upload expiry/purge checks and race compensation.
12. Cleanup CLI: claim lease, R2 deletion, durable log + append-only attempts, tombstone reservation, cron docs.
13. Integration/security tests: expiry matrix, authority boundary, cache/CSP, cleanup races, manual browsers.
14. Docs/decisions: update AGENTS.md, MEMORY.md, README, architecture, .env.example, Compose/host cron guidance.
15. Final verification: full tests, lint/type checks used by repo, clean secret scan, manual acceptance record.

Steps 1–10 are the viewer/security core. Do not expose PIN-locked pages before all four routes, cache headers, and rate/concurrency controls land together.

14. Documentation and locked-decision updates

Update:

• AGENTS.md: Privacy/Index/Index-auth/limits/retention decisions, routes, expiry CLI, new models, and do-not-regress rules.
• MEMORY.md: public listing is intentional; PIN is create-only/read-only access control; six fixed digits; MCP generation; per-client limits without global page lockout; four scrypt slots; expiry 404 semantics; permanent slug tombstones; audit structure; instance-wide publish authority remains admin.
• docs/architecture.md: viewer lifecycle/PIN boundary before R2, public ledger, cleanup state flow, MCP generation boundary, credential diagram.
• README.md / .env.example: rate/concurrency/cookie settings, public index behavior, expiry options, CLI and cron, trusted proxy security, credential authority warning.
• Docker/operations docs: exact docker compose exec -T app agent-pages purge-expired cron command, exit codes, retry behavior, database/R2 backup implications.

Add explicit do-not-regress entries:

1. Active/expiry and PIN checks cover all four routes before R2.
2. Every /p/* response is private/no-store and varies on Cookie.
3. No instance-wide per-page unlock counter may be introduced.
4. PIN verification remains bounded to four concurrent scrypt operations per process.
5. PIN/expiry remain create-only and page-level.
6. Expired pages are indistinguishable from missing pages externally.
7. Cleanup never deletes DB metadata before all R2 objects and retains retry/audit state.
8. Deleted slugs remain permanently reserved.
9. PINs never authorize writes; publish bearer remains the write boundary.
10. MCP continues to have no storage/model imports and generated PINs are returned with successful URLs.

15. Explicitly out of scope

• Locking an existing open page.
• Removing, rotating, recovering, or changing a PIN after publication.
• Changing/adding/extending expiry after publication.
• Manual page deletion.
• Per-version PINs or expiry.
• User accounts, SSO, multiple tenants, per-page ownership/edit secrets, create-only tokens.
• Making PIN-locked titles/slugs private.
• Search-engine indexing or public search beyond the ledger.
• CAPTCHA, proof-of-work, distributed rate limiting, multi-process semaphore coordination.
• Weak-PIN blocklists.
• Idempotent publish/update requests.
• Alembic introduction in this feature (but it is the recorded next-schema-change trigger).
• In-process/background cleanup scheduler or a second long-running Compose worker.
• Reusing expired slugs under any circumstance.