feat: emit LaunchDarkly context identity on AI SDK feature_flag spans - #60
feat: emit LaunchDarkly context identity on AI SDK feature_flag spans#60ccschmitz-launchdarkly wants to merge 4 commits into
Conversation
Python twin of js-ai-sdk's packages/client/src/context.ts, sharing its fixtures so the two ports cannot drift. Deliberately does not use ldclient.Context: ldclient is an optional import here, and an attribute that vanishes for custom clients is worse than no attribute.
Python counterpart of the js-ai-sdk change. Adds feature_flag.context.id and feature_flag.contextKeys to the feature_flag event, plus one context.contextKeys.<kind> span attribute per kind. Keys only. Context attribute values are not emitted. AIC-3230
The scanner was lowercase-only and namespace-limited, so it could not see feature_flag.contextKeys or the interpolated context.contextKeys.<kind>. Widened both, and added the keys to EXPECTED_VOCABULARY in the same commit, because fixing either half alone turns the suite red. Also asserts every handler writes the per-kind attributes and that tool spans still do not, so a context-scoped query still finds one span per run. AIC-3230
Documents feature_flag.context.id, feature_flag.contextKeys and the per-kind context.contextKeys.<kind> span attributes, and why the identity is emitted in two shapes rather than one. AIC-3230
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a820aa8. Configure here.
| # consumer may match on it as text. | ||
| feature_flag_attrs["feature_flag.contextKeys"] = json.dumps( | ||
| context_keys, separators=(",", ":") | ||
| ) |
There was a problem hiding this comment.
JSON encoding mismatches TypeScript output
Low Severity
json.dumps still uses ensure_ascii=True, so non-ASCII context keys are emitted as \uXXXX escapes. JSON.stringify writes those characters as-is, so feature_flag.contextKeys will not be byte-identical to js-ai-sdk or the browser SDK for the same context. That string is stored verbatim in ClickHouse and may be matched as text.
Reviewed by Cursor Bugbot for commit a820aa8. Configure here.


What
Python counterpart of the
js-ai-sdkchange (filed alongside this). Carries the identity of the context an AI Config was evaluated for onto the run's root span, so observability can filter a config's traces down to a single context. This PR also carries theTELEMETRY-CONTRACT.mdupdate for both SDKs, since that document is canonical for both.Three attributes, none of them a new LaunchDarkly convention:
feature_flageventfeature_flag.context.idgo-server-sdk/ldotelemitsfeature_flageventfeature_flag.contextKeyscontext.contextKeys.<kind>Why the identity is emitted in two shapes
feature_flag.context.idis a composite for a multi-kind context (kind:key:kind:key, sorted), so it cannot answer "filter this config's traces to this one user" — which is exactly the question AI Config Monitoring's group-by asks, since that groups per kind. Andfeature_flag.contextKeyslands in a ClickHouseStringcolumn holding JSON, so filtering it per kind would be substring matching. The per-kind span attributes are what give an exact match on a single kind.Why keys only
Context keys are identifiers, and LaunchDarkly's other OTel integrations already expose them. Context attribute values are where the personal data lives, so none are read or emitted, and no capture option is added — there is nothing to gate. A test asserts that a context carrying
nameandemailleaks neither.How it works
No plumbing.
execute_and_track/execute_and_streamalready mergeldContextinto thevariablesdict next to__ld, after the caller's own variables so it can't be clobbered.set_ld_span_attributesreads it there, and all six handler packages reach it through theirspans.start_root_span, so provider coverage is uniform by construction.Deliberately not on
TrackData: that dict is thedatapayload of every$ld:ai:*track call, where context keys would duplicate the call's own context argument and bloat every analytics event.New
ld_context.pyis a port ofgetCanonicalKey/getContextKeysfrom the observability browser SDK's LaunchDarkly integration, sharing fixtures with the TypeScript port so the two cannot drift. It deliberately does not go throughldclient.Context.fully_qualified_key:ldclientis an optional import here (seeto_ld_context), and an attribute that silently vanishes for anyone using a custom client is worse than no attribute.context_identitynever raises — this runs on the emit path of every run, so a malformed context degrades to emitting nothing.Two details worth a reviewer's eye
json.dumpsneeds explicit separators. Python defaults to", "/": ";JSON.stringifyemits neither. Withoutseparators=(",", ":")the same context would produce a different string in Python than in TypeScript, in a column consumers may match as text. There is a test pinning the exact bytes.The vocabulary lock needed widening, in the same commit.
tests/test_cross_handler_parity.pystatically scans attribute literals against a committed list. Its regex was lowercase-only (feature_flag\.[a-z_.]+) and limited to thegen_ai/launchdarkly/feature_flag/ldnamespaces, so it could seefeature_flag.context.idbut notfeature_flag.contextKeys(camelCase) orcontext.contextKeys.<kind>(f-string,contextnamespace). Adding the keys toEXPECTED_VOCABULARYwithout widening the scanner fails one lock test; widening without adding fails the other — so both halves are in one commit. Verified that widening surfaces nothing else: there are no pre-existingset_attribute("context…literals and no uppercase letters in any existing"feature_flag.…"literal.RecordedSpan.add_eventalso discarded event attributes (eventswas alist[str]), so nothing could assert on thefeature_flagevent's payload. It is now a name-keyed dict; all existing usages are membership checks and survive unchanged.Testing
make test— 1135 passed, 11 skippedmake lint— cleanTestVocabularyLocktests passtest_ld_context.py(shared fixtures with the TS port), 9 intest_span_attributes.py(nothing coveredset_ld_span_attributesbefore), plus 2 cross-handler cases asserting all six handlers write the per-kind attributes and that tool spans still do notPre-existing failure, unrelated:
make typecheckfails withDuplicate module named "conftest". This repo has 8 identically-namedconftest.pyfiles and no per-package mypy config; the failure reproduces on unmodifiedorigin/mainand this branch touches none of them. Every new and modified file passesmypy --strictindividually. Worth its own ticket.Still outstanding, tracked on the ticket: end-to-end verification in staging that the
context.contextKeys.user=…filter returns the run.Links
js-ai-sdkcounterpart: filed alongside thisNote
Overview
Adds evaluation context identity to the Python AI SDK’s root
invoke_agentspan and itsfeature_flagevent so AI Config Monitoring can filter traces by a single context kind (e.g. user), not only by config.ld_context.pyports canonical-key and per-kind-key derivation from the observability browser SDK /js-ai-sdk, with shared test fixtures so emitters stay aligned.set_ld_span_attributesreadsvariables["ldContext"]and, when identity is valid, setsfeature_flag.context.id, compact JSONfeature_flag.contextKeys(matchingJSON.stringify), and one span attribute per kind ascontext.contextKeys.<kind>. Malformed or missing context emits nothing and never raises; only context keys are written—no custom attribute values.TELEMETRY-CONTRACT.mddocuments the new fields and the keys-only rule. Cross-handler parity tests assert all six handlers get context on the root only (not tool spans), and the vocabulary lock scanner was extended for camelCasefeature_flag.contextKeysand thecontextnamespace.Reviewed by Cursor Bugbot for commit a820aa8. Bugbot is set up for automated code reviews on this repo. Configure here.