feat(flags-core): add header-driven Vercel mode on the shared cache

.changeset/embedded-flags-fetch-time.md

@@ -0,0 +1,8 @@
+---
+"@vercel/prepare-flags-definitions": patch
+"@vercel/flags-core": patch
+---
+
+Record `fetchedAt` when a datafile fetch completes and preserve it in generated flag definitions. Loading the bundle retains the original timestamp so the Flags SDK can determine its age.
+
+Expose optional `fetchedAt` metadata on datafiles. Record it for accepted live updates and preserve valid timestamps when loading provided or bundled definitions, without mutating the input.

.changeset/header-driven-vercel-mode.md

@@ -0,0 +1,11 @@
+---
+"@vercel/flags-core": minor
+---
+
+Add a header-driven `vercel` client mode, enabled by default when `VERCEL=1`. Initialization loads provided/bundled definitions; request versions trigger refreshes and an empty cache fetches on its first read. Explicit offline/build behavior is preserved.
+
+An evaluation with a missing or empty version header permanently starts streaming when enabled, otherwise polling. Concurrent reads share startup, and pending header refreshes are cancelled without losing cached data or resetting stale-if-error. Initialization and snapshot reads do not trigger this switch.
+
+The controller supplies a source freshness-status callback and optional fetch callback to the cache. HeaderSource keeps version observations; the cache handles serving, background/blocking refreshes, shared work, cancellation, and stale-if-error. The cache owns age and resets it on accepted updates or confirmations, without rewriting `fetchedAt`. Polling becomes stale after its interval, streaming after 30 seconds; stream pings reset age and clear stale-if-error failures because each connection sends `primed` or a datafile first. Source schedules stay unchanged.
+
+Use `staleWhileRevalidate` (default 10) and `staleIfError` (default Infinity) in **seconds**, including fractions. Setting either to `0` disables its stale allowance. Datafiles preserve optional `fetchedAt` epoch-millisecond timestamps across serialization and bundled/provided reuse.

packages/prepare-flags-definitions/src/index.test.ts

@@ -1,5 +1,8 @@
-import { readFile } from 'node:fs/promises';
-import { describe, expect, it, vi } from 'vitest';
+import { mkdtemp, readFile, rm } from 'node:fs/promises';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import { pathToFileURL } from 'node:url';
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
 import { version as pkgVersion } from '../package.json';
 import {
   generateDefinitionsModule,
@@ -8,6 +11,16 @@ import {
   prepareFlagsDefinitions,
 } from './index';
 
+const FETCH_TIME = 1_700_000_000_000;
+
+beforeEach(() => {
+  vi.useFakeTimers({ toFake: ['Date'], now: FETCH_TIME });
+});
+
+afterEach(() => {
+  vi.useRealTimers();
+});
+
 function createOidcToken(projectId: string): string {
   const header = Buffer.from(JSON.stringify({ alg: 'none' })).toString(
     'base64url',
@@ -118,6 +131,35 @@ describe('generateDefinitionsModule', () => {
 });
 
 describe('prepareFlagsDefinitions', () => {
+  it('embeds fetch completion time and preserves it when loaded later', async () => {
+    const cwd = await mkdtemp(join(tmpdir(), 'flags-fetched-at-'));
+    try {
+      await prepareFlagsDefinitions({
+        cwd,
+        env: { FLAGS: 'vf_server_timestamp' },
+        fetch: vi.fn().mockResolvedValue({
+          ok: true,
+          json: async () => {
+            vi.setSystemTime(FETCH_TIME + 5_000);
+            return { configUpdatedAt: 123, fetchedAt: 456, definitions: {} };
+          },
+        }),
+      });
+      vi.setSystemTime(FETCH_TIME + 365 * 24 * 60 * 60 * 1_000);
+      const url = pathToFileURL(
+        join(cwd, 'node_modules/@vercel/flags-definitions/index.js'),
+      ).href;
+      const bundle = await import(/* @vite-ignore */ url);
+      expect(bundle.get(hashSdkKey('vf_server_timestamp'))).toEqual({
+        configUpdatedAt: 123,
+        fetchedAt: FETCH_TIME + 5_000,
+        definitions: {},
+      });
+    } finally {
+      await rm(cwd, { recursive: true, force: true });
+    }
+  });
+
   it('returns { created: false, reason: "no-flags-entries" } when no flags auth is in env', async () => {
     const result = await prepareFlagsDefinitions({
       cwd: '/tmp/test',
@@ -148,7 +190,7 @@ describe('prepareFlagsDefinitions', () => {
     expect(definitionsJs).toMatchInlineSnapshot(`
       "const memo = (fn) => { let cached; return () => (cached ??= fn()); };
 
-      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true}}"));
+      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true},\\"fetchedAt\\":1700000000000}"));
 
       const map = {
         "faab116281fa4201059a73f3ca8b7cad7fce9e1132988008784883fa2c78d64a": _d0,
@@ -242,7 +284,7 @@ describe('prepareFlagsDefinitions', () => {
     expect(definitionsJs).toMatchInlineSnapshot(`
       "const memo = (fn) => { let cached; return () => (cached ??= fn()); };
 
-      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true}}"));
+      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true},\\"fetchedAt\\":1700000000000}"));
 
       const map = {
         "3790790d2dc9b23c4539a9f3c49eb5820e4216daebdd7eeee9136f3ceccc31a3": _d0,
@@ -298,7 +340,7 @@ describe('prepareFlagsDefinitions', () => {
     expect(definitionsJs).toMatchInlineSnapshot(`
       "const memo = (fn) => { let cached; return () => (cached ??= fn()); };
 
-      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true}}"));
+      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true},\\"fetchedAt\\":1700000000000}"));
 
       const map = {
         "prj_oidc_test": _d0,
@@ -338,7 +380,7 @@ describe('prepareFlagsDefinitions', () => {
     expect(definitionsJs).toMatchInlineSnapshot(`
       "const memo = (fn) => { let cached; return () => (cached ??= fn()); };
 
-      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true}}"));
+      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true},\\"fetchedAt\\":1700000000000}"));
 
       const map = {
         "faab116281fa4201059a73f3ca8b7cad7fce9e1132988008784883fa2c78d64a": _d0,
@@ -440,8 +482,8 @@ describe('prepareFlagsDefinitions', () => {
     expect(definitionsJs).toMatchInlineSnapshot(`
       "const memo = (fn) => { let cached; return () => (cached ??= fn()); };
 
-      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true}}"));
-      const _d1 = memo(() => JSON.parse("{\\"flag_b\\":{\\"value\\":true}}"));
+      const _d0 = memo(() => JSON.parse("{\\"flag_a\\":{\\"value\\":true},\\"fetchedAt\\":1700000000000}"));
+      const _d1 = memo(() => JSON.parse("{\\"flag_b\\":{\\"value\\":true},\\"fetchedAt\\":1700000000000}"));
 
       const map = {
         "faab116281fa4201059a73f3ca8b7cad7fce9e1132988008784883fa2c78d64a": _d0,

packages/prepare-flags-definitions/src/index.ts

@@ -183,7 +183,9 @@ async function fetchDatafile(
     }
 
     if (res.ok) {
-      return res.json() as Promise<BundledDefinitions>;
+      const definitions = (await res.json()) as BundledDefinitions;
+      // Preserve fetch time so loading the bundle does not make old data fresh.
+      return { ...definitions, fetchedAt: Date.now() };
     }
 
     if (res.status === 404) {

packages/vercel-flags-core/CLAUDE.md

@@ -18,6 +18,7 @@ src/
 ├── controller/           # Controller (state machine) and I/O sources
 │   ├── index.ts              # Controller class
 │   ├── stream-source.ts      # StreamSource (wraps stream-connection)
+│   ├── header-source.ts       # Request version checks and on-demand refresh
 │   ├── polling-source.ts     # PollingSource (wraps fetch-datafile)
 │   ├── bundled-source.ts     # BundledSource (wraps read-bundled-definitions)
 │   ├── stream-connection.ts  # Low-level NDJSON stream connection
@@ -43,7 +44,7 @@ src/
 ```
 createClient(sdkKey, options)
   → Controller (state machine, selects data origin and coordinates sources/cache)
-    → StreamSource / PollingSource / BundledSource (emit raw DatafileInput)
+    → StreamSource / PollingSource / HeaderSource / BundledSource (raw DatafileInput)
   → create-raw-client (ID-based indirection for 'use cache' support)
     → controller-fns (lookup by ID, evaluate, report)
   → FlagsClient (public API)
@@ -88,7 +89,9 @@ type ControllerOptions = {
   datafile?: Datafile;  // Initial datafile for immediate reads
   stream?: boolean | { initTimeoutMs: number };      // default: true (3000ms)
   polling?: boolean | { intervalMs: number; initTimeoutMs: number };  // default: true (30s interval, 3s timeout)
-  staleIfError?: number;  // Seconds of fallback after stream/poll failure; default: Infinity
+  vercel?: boolean; // default: process.env.VERCEL === '1'; replaces stream/poll at runtime
+  staleWhileRevalidate?: number; // Seconds of header refresh grace; default: 10
+  staleIfError?: number; // Seconds of fallback after update failure; default: Infinity
   buildStep?: boolean;  // Override build step auto-detection
   metricEnvironment?: string; // Environment attached to ingested evaluation metrics
   waitUntil?: (promise: Promise<unknown>) => void;  // default: @vercel/functions waitUntil
@@ -112,7 +115,28 @@ Behavior differs based on environment:
 
 Build-step reads are deduplicated: data is loaded once via a shared promise (`buildDataPromise`) and all concurrent `evaluate()` calls share the result. The entire build counts as a single tracked read event (`buildReadTracked` flag in Controller).
 
-**Runtime** (default, or `buildStep: false`):
+**Vercel runtime** (`vercel: true`, default when `VERCEL=1`):
+- Load provided or bundled definitions during initialization, then select Vercel mode.
+- Do not start stream/poll; the first read fetches if the cache is empty.
+- HeaderSource parses the request's project version and owns `highestObserved`. The cache owns freshness age.
+- A matching header confirms freshness only when no newer version has been observed.
+- The controller passes `getStatus` and `fetch` callbacks to `cache.resolve()`.
+  The cache selects cached/background/blocking behavior and shares refresh work.
+- A newer header permits background refresh within `staleWhileRevalidate` seconds of the
+  latest accepted fetch or valid confirmation; unknown/expired cache age blocks for refresh.
+- Every returned entry passes through `DatafileCache.read()`. Refresh errors use its
+  `staleIfError` allowance; expiry forces blocking recovery on the next newer-header read.
+- Evaluations without a version header (including an empty header) permanently
+  start streaming if enabled, otherwise polling, using the existing startup timeouts.
+  Concurrent reads share source startup. Pending header fetches are cancelled without
+  clearing stored data or the failure deadline; their readers resume through the new source.
+  `resolveData()` checks header availability and uses `resolveDataWithFallbacks()`
+  to start the configured source. Handover retains cached data before considering seeds.
+- Present malformed/unrelated headers use cached data without fetching, subject to stale-if-error.
+- `getDatafile()` remains a snapshot read: it enforces the same failure policy but does
+  not inspect request headers. Disabling both stream and polling selects offline mode.
+
+**Other runtime** (default outside Vercel, or `vercel: false`):
 1. **Stream** - Real-time updates via NDJSON streaming, wait up to `initTimeoutMs`
 2. **Polling** - Interval-based HTTP requests, wait up to `initTimeoutMs`
 3. **Provided datafile** - Use `options.datafile` if provided
@@ -232,9 +256,9 @@ When updating tests for new behavior, preserve the strength of existing assertio
 ### Stream Connection
 
 - Uses fetch with streaming body (NDJSON format)
-- Callbacks: `onDatafile` (new data), `onPrimed` (server confirmed revision is current), `onDisconnect`, and `onError` (failure evidence for cache policy)
+- Callbacks: `onDatafile` (new data), `onPrimed` (server confirmed revision is current), `onPing` (resets age and clears failure), `onDisconnect`, and `onError` (failure evidence for cache policy)
 - Sends `X-Revision` header with the current revision number on every connection (including reconnects), allowing the server to respond with a lightweight `primed` message instead of a full datafile when the revision is current
-- The `primed` message confirms the client's data is up-to-date; it resolves the init promise (like `datafile`) but does not update data — only transitions state to `streaming`
+- The `primed` message confirms the client's data is up-to-date; it resolves the init promise (like `datafile`) but does not update data — resets cache age and clears a failure when revision/identity match, then transitions state to `streaming`
 - Reconnects with exponential backoff (base: 1s, max: 60s, max retries: 15)
 - Retries on transient errors both before and after initial data is received. Before initial data, retries continue until max retries are exhausted or the abort controller is aborted (e.g., by the Controller's init timeout). The init promise rejects when the loop exits without data.
 - Default `initTimeoutMs`: 3000ms
@@ -260,7 +284,11 @@ The Controller selects the origin. Initial/fallback snapshots are tagged before
 - `'fetched'` → `'remote'`
 - `'bundled'` → `'embedded'`
 
-`tagData` mutates the input object in-place via `Object.assign` (callers always pass freshly-created data).
+`tagData` returns a shallow copy. Accepted fetched/stream/poll data is stamped with
+`fetchedAt`; provided and bundled data preserves valid finite nonnegative timestamps.
+The cache seeds its own freshness age from that timestamp; missing/invalid timestamps
+mean unknown age. Accepted updates and valid confirmations reset cache age without
+rewriting the stored `fetchedAt`. Equal/older responses do not replace or retag data.
 
 ### Usage Tracking
 
@@ -293,16 +321,28 @@ The DatafileCache rejects incoming data (from stream or poll) if its `configUpda
 
 ### Cache read policy
 
-`DatafileCache.read()` is the only full-entry read. The cache is configured once
+Every served entry passes through `DatafileCache.read()`, including `resolve(policy)` results. The cache is configured once
 with the internal `staleIfErrorMs`, normalized from the public `staleIfError`
 option in seconds. Evaluations and `getDatafile()` share the same serving
 boundary. `hasData` and `revision` expose coordination metadata even after expiry,
 so retained data is not replaced by fallback and stream reconnects can still send
 `X-Revision`. `seed()` never clears failure. Accepted source updates or valid
 version/revision confirmations clear it; repeated errors/disconnects do not renew
-the first-error deadline. Stream opening/pings and initialization timeout alone
+the first-error deadline. Stream opening and initialization timeout alone
 are not recovery/failure evidence respectively.
 
+`cache.resolve(policy)` receives a mode-specific `getStatus` callback returning
+`fresh`, `stale`, `expired`, or `unknown`, and an optional `fetch` callback.
+It owns background/blocking decisions, `waitUntil`, shared revalidation, and cancellation on clear. HeaderSource supplies small version/age
+checks and a fetch callback; it does not read the cache. Header confirmations are
+forwarded through controller event wiring. Stream/poll modes omit on-read revalidation
+and retain their existing schedules. New public time windows use seconds; internal
+normalized durations, cache age, and `fetchedAt` use milliseconds. Polling is fresh
+through its interval; streaming through 30 seconds. Accepted updates, valid confirmations,
+and stream pings reset cache age. Pings also clear failures: each connection sends
+`primed` or a datafile before pings, so they confirm recovery without rewriting `fetchedAt`.
+Polling errors use the shared source-error handler without logging each failed poll.
+
 ### Evaluation Safety
 
 - Regex comparators (`REGEX`, `NOT_REGEX`) limit input string length to 10,000 characters to prevent ReDoS

packages/vercel-flags-core/README.md

@@ -33,10 +33,40 @@ export default app;
 
 Outside Vercel, pass an SDK key explicitly: `createClient(process.env.FLAGS)`.
 
-## Cached stream and polling reads
+## Header-driven reads on Vercel
+
+When `VERCEL=1`, the client defaults to `vercel: true`. Initialization loads provided
+or bundled definitions without starting a stream or polling. Request version headers
+indicate when cached definitions need refreshing. If an evaluation has no version
+header (or an empty one), the client permanently switches to streaming when enabled,
+otherwise polling. Concurrent evaluations share that startup and later headers do
+not switch the client back. A present but malformed or unrelated header keeps the
+existing cached-read behavior, fetching only when the cache is empty.
+
+```ts
+const client = createClient(process.env.FLAGS!, {
+  vercel: true,
+  staleWhileRevalidate: 10, // Seconds of background-refresh grace.
+  staleIfError: 60, // Seconds of cached fallback after a refresh failure.
+});
+```
+
+`staleWhileRevalidate` defaults to 10 seconds and accepts finite, nonnegative values,
+including fractions. `0` makes refreshes block. The window starts at the latest
+accepted fetch or valid confirmation, including an equal-version fetch response.
+The cache tracks this age independently of `fetchedAt`. Bundled/provided definitions
+preserve their original `fetchedAt`; unknown or expired cache age requires a blocking
+refresh when a newer request version arrives. Refresh failures use `staleIfError`.
+A newer-header read attempts blocking recovery after that failure allowance expires.
+
+`getDatafile()` remains a snapshot read: it applies stale-if-error but does not inspect
+headers. Use `vercel: false` to select the existing stream/poll behavior. Disabling both
+stream and polling still selects offline mode, and builds retain their existing loading.
+
+## Cached reads after errors
 
 `staleIfError` controls how many seconds evaluations and `getDatafile()` may use
-cached flag definitions after a stream/poll failure or stream disconnect:
+cached flag definitions after a stream/poll/header-refresh failure or stream disconnect:
 
 ```ts
 const client = createClient(process.env.FLAGS!, {
@@ -54,8 +84,9 @@ The allowance starts at the first consecutive failure. Repeated errors,
 disconnects, and provided or bundled fallback data do not renew it. An accepted
 source update, or a finite equal version for the same project and environment,
 clears the outage. A stream `primed` message also clears it when its finite numeric
-revision and identity match the cached entry. Opening a connection or receiving
-a ping alone does not clear a failure. A later failure starts a new allowance.
+revision and identity match the cached entry. Pings clear failures too: the server
+sends `primed` or a datafile before pings on each connection. Opening a connection
+alone does not clear a failure. A later failure starts a new allowance.
 Responses are observed in completion order, with existing version acceptance.
 
 After expiry, `evaluate()` returns the caller's default with reason `error`, or
@@ -66,9 +97,12 @@ retained for recovery, including its revision for stream reconnection. A clean
 stream close or ping timeout records `stream: disconnected` if no earlier failure
 exists. `getFallbackDatafile()` remains an independent bundled-data export.
 
-There is no age-based expiry while the source is healthy, and reads do not trigger
-an extra refresh after expiry. Build/offline behavior, source scheduling, retries,
-timeouts, metrics categories, and logging are unchanged. An initialization timeout
+Polling data is marked stale after the polling interval; streaming data after 30
+seconds. Accepted updates and valid confirmations reset cache age without rewriting
+`fetchedAt`. Stream pings also reset age and clear any failure.
+Age alone does not prevent stream/poll reads or trigger extra requests. Source scheduling,
+retries, timeouts, and build/offline behavior remain unchanged. Poll errors feed the
+shared failure handler without logging each failed poll. An initialization timeout
 alone does not start the allowance. Existing startup limitations remain: when
 initial polling times out, no recurring interval is started, even if that in-flight
 request later completes.