Skip to content

internal/ctlog: prerender get-roots responses - #77

Open
lsautia-godaddy wants to merge 2 commits into
FiloSottile:mainfrom
lsautia-godaddy:feat/prerender-get-roots
Open

lsautia-godaddy wants to merge 2 commits into
FiloSottile:mainfrom
lsautia-godaddy:feat/prerender-get-roots

Conversation

@lsautia-godaddy

@lsautia-godaddy lsautia-godaddy commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Each get-roots request currently collects the accepted certificates and serializes the entire response to JSON. This repeats the certificate traversal and base64/JSON encoding even when the accepted roots have not changed.

Render the response when a log loads and when its accepted root set changes. The existing /ct/v1/get-roots handler now writes that immutable response directly, removing serialization from the request path.

Implementation

  • Add a rootsJSON snapshot to each log and initialize it from persisted roots in LoadLog, including an empty certificates array when there are no roots.
  • In SetRootsFromPEM, skip an identical PEM refresh entirely. If the PEM changes but the accepted certificate set is unchanged (ordering, comments, or duplicates), preserve the existing JSON buffer while persisting the updated PEM.
  • When certificates are added or removed, serialize the replacement response and publish it together with the new root pool under the existing rootsMu, after the backend upload succeeds. Parsing or upload failures retain the previous roots and response.
  • The handler takes the snapshot under a read lock and releases the lock before writing to the client. In-flight responses can finish using the previous immutable buffer while an update publishes its replacement.
  • Keep serialization and root-set comparison helpers beside the root-update code.

The retained memory is one serialized JSON response per log, approximately the base64 size of the certificates plus JSON syntax (roughly 4/3 of the total DER bytes). Requests share that buffer. A changed root set allocates a replacement; an old snapshot remains reachable only while existing requests use it, then becomes eligible for garbage collection. An unchanged accepted set reuses the buffer.

The response retains its JSON structure, trailing newline, content type, and CORS header. HTTP cache headers remain under operator control; this change adds no Cache-Control policy.

Validation

Added tests for response-buffer reuse, root additions/removals, PEM reordering/comments/duplicates, startup from persisted and empty roots, invalid PEM/DER, backend upload failure and retry, updates during an in-flight response, and concurrent readers/root updates. The full CT log suite, go test -short -race ./internal/ctlog, command builds, and command -h smoke checks passed.

Also ran the actual application locally over verified HTTPS from clean commit ddfeef7, with independent TLS and mark logs, filesystem storage, and SQLite checkpoint/cache databases. All 14 runtime checks passed, including:

  • Repeated get-roots responses with the expected bytes and headers.
  • TLS and mark submissions with cryptographic verification of returned SCT signatures; duplicate submissions do not add leaves.
  • Root addition/removal updating both the response and chain acceptance; unchanged and reformatted PEM refreshes preserving the response; invalid reloads preserving a usable log.
  • 400 HTTPS requests from 8 workers during 10 root replacements, with complete snapshots and isolation between logs.
  • A process restart preserving roots, checkpoint sizes, and duplicate submission timestamps/extensions.

The runtime check uses custom PEM reloads through SIGHUP. Live CCADB fetching, the 15-minute timer, and remote object storage were not exercised. These are functional checks; no production performance or memory measurements are claimed.

@lsautia-godaddy lsautia-godaddy changed the title Feat/prerender get roots internal/ctlog: prerender get-roots responses Nov 12, 2026
@lsautia-godaddy
lsautia-godaddy force-pushed the feat/prerender-get-roots branch from ddfeef7 to f9e5863 Compare November 12, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant