Skip to content

docs: explain how CycloneDX and SPDX SBOM attestations differ - #420

Open
AlexKantor87 wants to merge 3 commits into
mainfrom
claude/docs-sbom-format-differences
Open

AlexKantor87 wants to merge 3 commits into
mainfrom
claude/docs-sbom-format-differences

Conversation

@AlexKantor87

Copy link
Copy Markdown
Contributor

The SBOM section describes the subject and the package count as though both come back the same way whatever the format. They do not.

A reader who follows this page to write a policy meets the difference as a rule that silently never matches. That is the worst way to find out.

The two differences

The subject's digest is usually missing from CycloneDX. In SPDX the subject is a package and its checksum sits on that package. So subject.sha256 is filled. CycloneDX has a hashes field for the same job. Snyk and Syft both leave it empty. They write the digest into version instead, where it reads as a version string. Kosli does not infer a checksum from a version, so the field stays empty.

Package counts are not comparable. package_count counts what each format calls a package. CycloneDX entries describing a file are skipped. The SPDX equivalents are not.

Measured

Ran the CLI parser over three real documents rather than reasoning from the schemas.

document subject.sha256 package_count
Snyk CycloneDX 1.6, server image empty 1023
Syft CycloneDX 1.6, kosli_Linux_arm64.rpm empty 1
Syft SPDX 2.3, same rpm the artifact fingerprint 2

The first reproduces the live attestation on the server flow exactly. The last two are the same artifact from the v2.41.0 release. That pair is where the count difference shows up.

One file, prose only. No new pages and no new links, so navigation is unaffected.

🤖 Generated with Claude Code

The page describes the subject and the package count as though both come
back the same way whatever the format. They do not, and a reader following
this to write a policy meets the difference as a rule that never matches.

In SPDX the subject is a package and its checksum sits on it, so
subject.sha256 is filled. CycloneDX has a hashes field for the same job,
but Snyk and Syft both leave it empty and put the digest in version. Kosli
does not infer a checksum from a version, so the field is empty there.

package_count counts what each format calls a package. CycloneDX skips
entries describing a file and SPDX does not, so one artifact reports a
different count in each format.

verified: ran internal/sbom.ProcessSBOMFile from cli origin/main over three
real documents -- Snyk CycloneDX 1.6 from a server build (sha256 null,
count 1023, matching the live attestation), and Syft CycloneDX 1.6 (sha256
null, count 1) and Syft SPDX 2.3 (sha256 = the artifact fingerprint, count
2) for the same rpm from the v2.41.0 release. Read subjectFromComponent,
subjectFromSPDX and packageCount for why.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@AlexKantor87
AlexKantor87 requested a review from a team as a code owner September 15, 2026 15:13
@mintlify

mintlify Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
kosli 🟢 Ready View Preview Sep 15, 2026, 3:55 PM

@claude

claude Bot commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

Claude finished @AlexKantor87's task in 3m 1s —— View job


Doc review — PR #420

Read getting_started/attestations.md at 560b7bc. One file, prose plus a Rego snippet in the SBOM accordion; no new pages, so config/navigation.json is untouched and correct. link-rot is NEUTRAL again, so I checked the one new link by hand: /policy-reference/rego_policy#input-data resolves — policy-reference/rego_policy.mdx:116 is ## Input data.

Two claims verified against the sources they lean on: --attestations really is "plain name for trail-level, dot-qualified (artifact.name) for artifact-level" (client_reference/kosli_evaluate_trail.md:29), and environment policy expressions really expose only flow.name, flow.tags, artifact.name, artifact.fingerprint (policy-reference/environment_policy.mdx:126-145). Both new sentences are right.

The three findings are all on the worked example, and on the same theme: it is now the site's only source for paths a policy silently depends on.

getting_started/attestations.md

1. Improvement — artifact.artifact_fingerprint (line 336) is unsourced, and the example turns on it. rego_policy.mdx:148-150 describes an artifacts_statuses entry only as having "its own attestations_statuses map" and names no fingerprint field; the artifact_fingerprint keys in client_reference/kosli_get_trail.md:83 are in the trail events array, a different object; the nearest documented spelling is artifact.fingerprint (environment_policy.mdx:144). If the key differs, the == compares against undefined and the rule never matches — the exact failure this section was written to prevent, arriving via its own example. Pin it with --show-input on a live trail and document the confirmed key in the reference. (inline)

2. Improvement — the snippet doesn't demonstrate the presence assertion the prose asks for (lines 331-338). Line 318 says to "assert the field is present before comparing it"; the snippet compares immediately, so on the CycloneDX case the section is about the body is undefined and does not fire. Also if without import rego.v1 (both examples in rego_policy.mdx carry it), artifact unbound where the site shows the every name, artifact in ... artifacts_statuses binding (rego_policy.mdx:258), and line 340's attestations_statuses[name] disagreeing with the snippet's own alias. (inline)

3. Improvement — attestation_data exists nowhere but this accordion (lines 328-338). rego_policy.mdx:145 tells policy authors that a status entry "contains the attestation's data, including type-specific fields", and its own example reads snyk.processed_snyk_results.results with no wrapper. SBOM needs the extra attestation_data.document step. The merge behaviour makes both true, but a reader who starts at the Rego reference — where policy authors start — has no way to learn that this type sits one level down; and a getting-started accordion is a poor durable home for a field path. One sentence under ## Input data naming attestation_data.document.subject.sha256 and attestation_data.document.package_count, with this accordion pointing at it, puts the path where it will be looked up and keeps the two pages from drifting.

Fix this →

3 Improvements, 0 Critical, 0 Suggestions. Nothing blocks the merge; the prose half of the change is sound and cross-checks clean. The Rego example is where the risk sits — it is the only place on the site giving these paths, so a wrong field name there is worse than no example.
· branch claude/docs-sbom-format-differences

Comment thread getting_started/attestations.md Outdated
Comment thread getting_started/attestations.md Outdated
Comment thread getting_started/attestations.md Outdated
Comment on lines +322 to +324
*Package counts are not comparable between formats.* `package_count` counts what each
format calls a package. CycloneDX entries describing a file are not counted; the SPDX
equivalents are. The same artifact can report a different count in each format.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion — "CycloneDX entries describing a file are not counted; the SPDX equivalents are" is too abstract to act on: a reader cannot map "the SPDX equivalents" onto anything they can look for in their own document, so they can't predict a count or explain one to an auditor.

The measurements in the PR description do this work and aren't on the page. Naming the mechanic (a CycloneDX component with type: file is skipped) and giving the measured pair — the same kosli_Linux_arm64.rpm reporting 1 from Syft's CycloneDX output and 2 from its SPDX output — turns a caveat into something the reader can check against their own SBOM.

Fix this →

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken. The abstraction was hiding the one thing a reader can check.

The paragraph now names the mechanic and gives the measured pair. A CycloneDX component with type: file is skipped, while the SPDX package describing that same file is counted. Syft reports one package for kosli_Linux_arm64.rpm in CycloneDX and two for the same file in SPDX.

Kept as prose, no table, as suggested.

verified: ran internal/sbom.ProcessSBOMFile from cli origin/main over the two SBOMs the
v2.41.0 release produced for that rpm. CycloneDX 1.6 gives package_count 1,
SPDX 2.3 gives 2. Read packageCount, which skips ComponentTypeFile and recurses
into nested components, against the SPDX path which is len(doc.Packages) with
no filter.

no mutation: this reply adds no test. The claim is a pair of parser runs over two real
documents, and the numbers above are that output.

search: grep -nE "package_count|type: file" getting_started/attestations.md
:321 and :322-323, this paragraph only. No other page states a package count rule,
per the site-wide grep in the first thread.

…y path

Three review findings, all taken.

The takeaway generalised to the format after the prose had correctly blamed
the tools. A reader using a generator that fills CycloneDX hashes would have
read the field as untrustworthy. It now says Kosli reads the hashes entry in
both formats, that Snyk and Syft leave it empty, and that this is a tool
choice.

The page told readers the differences matter for a policy without saying
which kind can reach the fields. Environment policy expressions expose only
the flow and the artifact name and fingerprint, so only Rego can. The
artifact-scoped path is given, with the trail-scoped variant named.

The package count rule named no mechanic a reader could look for. It now
says a CycloneDX component with type file is skipped, and gives the measured
pair for one rpm.

verified: ran internal/sbom.ProcessSBOMFile from cli origin/main over four
          documents. Live Snyk CycloneDX 1.6, subject.sha256 null. The same
          document with a SHA-256 added to metadata.component.hashes,
          populated. Syft CycloneDX 1.6 and SPDX 2.3 for kosli_Linux_arm64.rpm,
          package_count 1 and 2. Resolved the documented Rego path with opa
          eval against the live attestation shape.

mutation: drop the added hashes entry -> subject.sha256 returns null, so the
          populated result comes from hashes and not the version beside it.

search: grep -rnE "subject\.sha256|package_count" over all md and mdx finds
        only this file. mint broken-links reports one broken link, in
        tutorials/working_with_controls.mdx, present on main and untouched here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread getting_started/attestations.md Outdated
Comment on lines +326 to +330
Of the two policy mechanisms, only [Rego](/policy-reference/rego_policy#input-data) can read
these fields; environment policy expressions expose the artifact's name and fingerprint but
not attestation content. An artifact-scoped SBOM is reachable at
`input.trail.compliance_status.artifacts_statuses.<artifact>.attestations_statuses.<name>.attestation_data.document`,
and a trail-scoped one at the same path without the `artifacts_statuses.<artifact>` step.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement — the new path disagrees with every other Rego example on the site, and omits the prerequisite that makes it populated.

1. The attestation_data.document step appears nowhere else. rego_policy.mdx:145 describes the attestations_statuses value as the object that "contains the attestation's data, including type-specific fields enriched via --attestations" — fields directly on the status object, no wrapper. Both its own example (rego_policy.mdx:252) and the tutorial (tutorials/evaluate_trails_with_opa.mdx:157, :221) read snyk := artifact.attestations_statuses[...] then snyk.processed_snyk_results.results. A site-wide grep finds attestation_data only on this line. So either this page adds a step the reader's rule will trip over, or the other three places are describing a shape SBOM doesn't follow — and a reader has no way to tell which. Given that a wrong path here produces a rule that silently never matches, it's worth settling from kosli evaluate trail --show-input --output json on a real trail carrying an SBOM attestation rather than from an input object assembled by hand, since building the shape by hand assumes the answer.

2. --attestations filters by name. The flag "limits which attestations are enriched with full detail" (tutorials/evaluate_trails_with_opa.mdx:393, client_reference/kosli_evaluate_trail.md:30), and the tutorial recommends narrowing with it for speed. A reader already passing --attestations dashboard.snyk-container-scan gets an undefined lookup for the SBOM — the same silent non-match this section exists to prevent. One clause naming the flag closes it.

Two smaller things in the same sentence: the path is dot notation with <artifact>/<name> placeholders, while the site's examples use bracket lookup with a params-aliased name — which rego_policy.mdx makes a design rule, so a missing param fails the rule rather than proceeding. And the path stops at .document, leaving the reader to join it to the two fields the section is about (.document.subject.sha256, .document.package_count).

Fix this →

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Point 2 and the two smaller ones taken. Point 1 I checked and it does not hold.

attestation_data is not a step this page invented. Evaluation merges each attestation record onto its status entry: rehydrateAttestationMap copies every top-level key of the detail record across, unless the entry already has that key. pull_requests, processed_snyk_results and html_url are top-level fields of an attestation record. So is attestation_data. The other three places and this one read top-level keys of the same merged object, so there is nothing to settle.

Production confirms it resolves. kosli-dev/server evaluates SDLC-CTRL-0004 with artifact.attestations_statuses[name].attestation_data.content. On the trail this PR takes its numbers from, the live decision records allow: true and violations: null. A non-resolving path empties lock_content, which empties lock_pins, which fires the "records no exact (==) version pins" rule. allow would be false. It is not.

I could not run --show-input, having no CLI token here.

Point 2 is now on the page, dot-qualified for artifact scope. The example also moved to bracket lookup with a params-aliased name, and reaches the two fields.

verified: read internal/evaluate/transform.go rehydrateAttestationMap for the merge rule.
The CLI golden check in cmd/kosli/evaluateTrail_test.go asserts
input.trail.compliance_status.artifacts_statuses.cli.attestations_statuses.art-att.html_url,
which is the artifacts_statuses step. Ran opa check on the snippet now in the
page, then opa eval three ways.

mutation: set subject.sha256 to null -> allow undefined, the silent non-match. Remove the
attestation from attestations_statuses, as --attestations would -> allow
undefined again. Populate the digest -> allow true. So both warnings on the
page describe the behaviour the snippet has.

search: grep -rn "attestation_data" over the kosli-dev/server control policies finds
SDLC-CTRL-0004 only. The other three read status, is_compliant, has_audit_package
and pull_requests directly off the entry. Both idioms are live and green in one
pipeline, which is what the merge rule predicts.

The path was prose with placeholders, stopped at the document object, and
said nothing about input filtering. A reader narrowing the input with
--attestations and leaving the SBOM out gets a rule that never matches,
which is the failure this section exists to prevent.

It is now a snippet in the site's own idiom: bracket lookup with a
params-aliased attestation name, reaching subject.sha256 and comparing it
to the artifact fingerprint. The trail-scoped path is named beside it. A
closing line says to include the SBOM in --attestations, dot-qualified for
artifact scope, because anything left out is absent from the input.

The attestation_data step stands. Evaluation copies every top-level key of
an attestation record onto its status entry, so pull_requests and
attestation_data sit at the same level, and the site's other examples and
this one are both reading top-level keys.

verified: read internal/evaluate/transform.go rehydrateAttestationMap for
          the merge rule. cmd/kosli/evaluateTrail_test.go asserts
          input.trail.compliance_status.artifacts_statuses.cli.attestations_statuses.art-att.html_url,
          confirming the artifacts_statuses step. kosli-dev/server's
          SDLC-CTRL-0004 reads attestation_data.content and its live
          decision on this trail is allow true, violations null, which an
          unresolved path would have made false. opa check passes on the
          snippet.

mutation: subject.sha256 null -> allow undefined. Attestation removed from
          attestations_statuses -> allow undefined. Digest populated ->
          allow true. Both warnings match the snippet's behaviour.

search: grep -rn attestation_data over the server control policies finds
        SDLC-CTRL-0004 only; the other three read fields directly off the
        entry. mint broken-links reports one broken link, in
        tutorials/working_with_controls.mdx, present on main and untouched.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

sbom_describes(artifact) if {
sbom := artifact.attestations_statuses[sbom_attestation_name]
sbom.attestation_data.document.subject.sha256 == artifact.artifact_fingerprint

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvementartifact.artifact_fingerprint is the one field in this snippet that nothing on the site backs up, and it is the field the comparison turns on.

  • rego_policy.mdx:148-150 documents an artifacts_statuses entry as having "its own attestations_statuses map" and names no other field on it, so the fingerprint key is undocumented there.
  • The only in-repo occurrences of artifact_fingerprint are in the trail events array (client_reference/kosli_get_trail.md:83, :404) — a different object from an artifact status entry, so it is not evidence for this path.
  • The nearest documented spelling is the environment-policy one, artifact.fingerprint (policy-reference/environment_policy.mdx:144).

If the key on a status entry is fingerprint (or absent, with the name-keyed map being the only artifact identity in the input), the == compares against undefined, sbom_describes never fires, and the reader gets the silent non-match this whole section was written to prevent — from the example meant to show them the way. Worth pinning with kosli evaluate trail --show-input --output json | jq '.input.trail.compliance_status.artifacts_statuses[] | keys' on a real trail rather than a hand-built input, and then documenting the confirmed key in rego_policy.mdx's ## Input data so the example has a second source.

Fix this →

Comment on lines +331 to +338
```rego
sbom_attestation_name := data.params.sbom_attestation_name

sbom_describes(artifact) if {
sbom := artifact.attestations_statuses[sbom_attestation_name]
sbom.attestation_data.document.subject.sha256 == artifact.artifact_fingerprint
}
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Improvement — the snippet does not do the thing the paragraph above it advises, and it can't be run as written.

Line 318 tells the reader to "assert the field is present before comparing it, so a missing digest fails the rule instead of skipping it". The snippet compares straight away. On a Snyk or Syft CycloneDX SBOM — the case the section is about — subject.sha256 is absent, the == is undefined, and the body does not fire. Under a positive allow that fails safe; inside a violations rule it is the silent skip rego_policy.mdx:84 warns about. Since the snippet shows neither call site, the reader can't tell which they get. A presence check makes the advice visible in the code:

sbom_describes(artifact) if {
    sbom := artifact.attestations_statuses[sbom_attestation_name]
    digest := sbom.attestation_data.document.subject.sha256
    digest != ""
    digest == artifact.artifact_fingerprint
}

Two smaller things in the same block: if in a rule head needs import rego.v1, which both examples in rego_policy.mdx (:212, :245) and the tutorial carry but this fragment doesn't mention; and artifact arrives unbound, where the site's idiom shows the every name, artifact in trail.compliance_status.artifacts_statuses that produces it (rego_policy.mdx:258-262). Either add that line or say the fragment plugs into the Snyk example's shape.

Line 340 then writes the trail-scoped path as attestations_statuses[name], using name where the snippet aliases sbom_attestation_name, and dropping the input. prefix the reference uses throughout.

Fix this →

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant