Conversation
1 task
…metry Co-Authored-By: Claude <noreply@anthropic.com>
…etry Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
joe4dev
force-pushed
the
devx-1004-error-code-telemetry
branch
from
September 25, 2026 07:56
97e23b2 to
84b933e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
With
proxied,proxy_exit_codeandcancelled(#499), telemetry can tell whose failure a failedlstk_commandwas, but not why:lstk aws --account 123 s3 ls,awsmissing from PATH, Docker being down and a genuine lstk defect all land in the same "lstk failed" bucket, which is why that bucket cannot be read as a reliability metric. The repo already has the why axis,output.ErrorCodewith its categories, but it rode theErrorEventthe sink consumed, not the error value the telemetry hook receives, and only 20 of 67 error sites set one.Solution
Stacked on #499.
lstk_commandevents gainresult.error_codeandresult.error_category, present when the failing site classified its error, absent otherwise.SilentErrorgains aCode, set byoutput.Fail(sink, event, err), which emits the event and returns the silent error in one call, replacing the two-stepsink.Emit(...); return output.NewSilentError(err)that dropped the code. Domain code that returns errors for the caller to render usesoutput.WithCode(err, code), a non-silent carrier.commandResultreads either viaoutput.ErrorCodeOfand emits the code and its category.ErrorEventin the tree now sets aCode.emitValidationErroris split by cause (VALIDATION_ERRORfor arguments,NETWORK_ERRORfor an unreachable--endpoint-url,EMULATOR_WRONG_TYPEfor a wrong emulator behind it); the fifteenruntime not healthypreflights shareruntime.UnhealthyError; terraform backend provisioning isIAC_DEPLOY_FAILED; Azure setup and interception failures areEMULATOR_NOT_RUNNING,INTERNAL_ERRORorVALIDATION_ERROR; snapshot, auth, config, dependency and container sites got their matching codes. Three definitions were widened:DEPENDENCY_MISSING(also an unusable tool version),SNAPSHOT_INVALID_REF(also an invalid or incompatible snapshot file),EMULATOR_START_FAILED(also a container-name conflict or unremovable leftover container).TestEveryNewErrorEventSetsACodeparsescmd/andinternal/and fails when anErrorEventliteral has noCode. Failures that reach Cobra's fallback printer without any event stay unclassified; the pipe reports that share.Not emitted:
CANCELLED(thecancelledfield is the observation) and anything for proxied tool exits (noErrorEventis shown for them;proxy_exit_codeis their axis).Analytics side
The reliability and coverage queries, and the category split (
USAGE,CONFIG,AUTH,RUNTIMEare the user's environment; the rest is a team decision), are inopenspec/changes/distinguish-proxied-command-errors/design.md, Decision 8.Testing
Unit tests for the carriers and the builder; integration tests for
VALIDATION_ERROR,RUNTIME_UNAVAILABLE,NETWORK_ERROR,EMULATOR_WRONG_TYPE,INTERNAL_ERROR,EMULATOR_NOT_RUNNING,IAC_DEPLOY_FAILED,INTEGRATION_NOT_SET_UP,AUTH_REQUIREDandSNAPSHOT_NOT_FOUND. The 40-scenario manual matrix on #499 was run with these commits and every classified row matched.Docs
Every
ErrorEventnow carries anoutput.ErrorCode, so--jsonenvelopes that previously fell back toINTERNAL_ERRORreport their real code, and three code definitions indocs/structured-output.mdwere widened (see above). The telemetry payload itself is internal and undocumented.Review
Human review advised: it touches 46 files across most packages, changes documented
--jsonerror codes, and makes taxonomy calls (INTERNAL_ERRORfor lstk's own failing subprocesses,IAC_FILE_NOT_FOUNDfor a missing provider) that deserve a second opinion.Todo
mainonce Record proxied, proxy_exit_code and cancelled on command telemetry #499 mergeserror_code/error_categoryand add the coverage query (localstack-dwh)🤖 Generated with Claude Code
Follow-up of DEVX-1004