Docs proposal: review object-detection augmentation changes in pull requests with CML

[ternaus]

Feature request

Would you be open to a focused guide or small example repository for ML teams that need to review an object-detection augmentation-policy change in a pull request before merging it or spending compute on model training?

User problem

A change to stochastic preprocessing can alter an image while silently mishandling its bounding boxes or class labels. A unit test can check shapes and coordinate ranges, but it does not let a reviewer see whether the transformed training samples still make sense. Discovering the problem from model metrics is late and makes the cause harder to isolate.

CML already turns generated images, tables, and Markdown into a pull-request comment or check. AlbumentationsX can apply one label-aware policy to an image, its boxes, and their class IDs, use a stable sample-level invocation_seed, and expose both the serialized policy and the transforms realized for a sample.

Together, they could give a reviewer a compact augmentation report directly in the pull request:

• the fixed input scene and its annotations;
• transformed samples for a small set of stable seeds;
• pass/fail annotation invariants;
• the transforms and sampled parameters realized for each preview; and
• the resolved augmentation policy used by CI.

The reviewer could inspect the data effect of the code change before approving it. The example would demonstrate a CML review workflow rather than a generic augmentation snippet.

Proposed workflow

The example would use a generated detection scene, avoiding dataset downloads and data-license dependencies. A small Python script would:

1. construct an AlbumentationsX policy for an image, Pascal VOC bounding boxes, and class_labels;
2. apply it to the same fixture with stable invocation_seed values such as 17 and 29;
3. assert that returned boxes and labels remain paired and that coordinates are valid for the output image;
4. render the original and transformed annotations as contact-sheet images;
5. write a compact applied-transform table from AX run_with_trace() output; and
6. save A.to_dict(policy) as JSON for policy provenance.

The GitHub Actions job would then assemble a Markdown report and publish it with the existing CML interface:

- uses: actions/checkout@v4
- uses: iterative/setup-cml@v1
- uses: actions/setup-python@v5
  with:
    python-version: "3.12"
- name: Build augmentation review
  run: |
    pip install -r requirements.txt
    python review_augmentation.py --output artifacts

- name: Publish CML report
  env:
    REPO_TOKEN: ${{ secrets.GITHUB_TOKEN }}
  run: |
    python build_report.py --artifacts artifacts --output report.md
    cml comment create report.md

The report could contain a small table such as:

Input	AX policy, seed 17	AX policy, seed 29
Original boxes and labels	Transformed boxes and labels	Transformed boxes and labels

Below the images, it would show the annotation checks and collapsible policy/trace details. No model training, external service, or DVC dependency would be required.

Why this fits CML

• CML owns the CI and pull-request review surface: it publishes the generated evidence where the code change is being discussed.
• AlbumentationsX owns synchronized image/box/label transformation, repeatable sample realizations, execution details, and policy serialization.
• The user gets an early, reviewable answer to “does this augmentation change still produce valid labeled training samples?”

The existing tabular getting-started example would remain unchanged. This could be a standalone advanced guide in cml.dev or a separate focused example repository, whichever is easier for the maintainers to own.

Dependency boundary

• The public albumentationsx package is AGPL-3.0-only and imports as albumentations.
• Importing it requires PyTorch. PyTorch is intentionally not selected by the package metadata because users need to choose the appropriate CPU, CUDA, or MPS build.
• These dependencies would remain scoped to the example and would not become CML runtime dependencies.

I searched the current iterative/cml and iterative/cml.dev code, Issues, and pull requests and found no existing Albumentations or AlbumentationsX integration or overlapping augmentation-review proposal.

If this use case fits the CML documentation, I would be happy to implement and run the example after the maintainers confirm the preferred placement.