Skip to content

feat: shift the scan inside the patch and fill what lies outside it with EM-synthesised background - #144

Draft
Hendrik-code wants to merge 4 commits into
mainfrom
hm/fov-extension
Draft

Hendrik-code wants to merge 4 commits into
mainfrom
hm/fov-extension

Conversation

@Hendrik-code

@Hendrik-code Hendrik-code commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Why

A model trained on a narrow field of view has never been shown what lies outside it. On whole-body axial NAKO VIBE, the SpineGAN-trained models lose most of their Dice to false positives far from the spine. The contrast augmentations are not failing here: the network has never been asked to reject that context.

Diagnosis, measured on Ours (fp32) (paper config c69b9872 on 01bf0b3), NAKO VIBE test split:

vertebra instance precision, all four training sets, in-phase and water 0.01, median 76-125 false-positive components per subject (recall ~0.90)
FP voxels more than 50 mm from the reference spine (803 in-phase, n=10) 60%
FP voxels outside the body 1%
FP voxels by VibeSeg-100 class (803 in-phase / 804 water, n=15 each) unlabelled 31/41%, muscle 14/21%, gluteals 9/6%, subcutaneous fat 8/4%, intestine 5/11%, femur 4/4%, sacrum 2/1%
Dice if only the largest connected component is kept (803 in-phase, n=15) vertebra 0.729 → 0.804, IVD 0.616 → 0.696, canal 0.697 → 0.808

The cause is the training geometry. Preprocessed 803 has a median of 85 (LR) × 297 × 187 voxels at 1 mm, with a 128³ patch. The foreground leaves only 22 mm of LR free, so nnU-Net centres the slab and pads the rest:

  • data is padded with 0;
  • seg is padded with -1, which the tail then turns into 0;
  • so every training window contains spine, and beyond ±42 mm laterally the network has only seen a flat slab labelled background.

What

RandomFOVExtensionGPU is a new GEO transform in smauglab/transforms/gpu/context.py. It runs directly after RandomAffineGPU and before every intensity transform. For each selected sample:

  1. Shift. Image and mask move by the same whole-voxel offset, up to shift of the patch per axis. At shift: 1.0 the spine can leave the window entirely.
    • With spare_label_faces (default true), the shift never vacates a window face that a label comes within margin voxels of. A structure can still leave through that face, but synthetic tissue is never placed against it.
    • The spine touches both faces along its length, so it is not shifted along that axis. Which axis that is gets read off each window's labels; it is not configured.
  2. Fill region. Everything that is not scan: the padding (every channel exactly pad_value) plus whatever the shift vacated.
  3. Synthesise it SynthSeg-style, from the patch as it arrived, before the shift.
    • EM splits the patch's background into a random 3-10 regions. Only voxels labelled 0, beyond margin of any label and off the padding seam are used.
    • A 3×3×3 majority filter and a random diffeomorphic field (SynthSeg's random_svf_field) post-process the region map.
    • layers randomly rotated, scaled, mirrored and translated copies of that fixed map are composited into the fill.
    • Each region is rendered from its own mean and std. The texture is spatially correlated (σ = 1 voxel), then blurred as SynthSeg blurs.
    • The fill is labelled 0.

Everything outside the fill region is the shifted input, bit for bit, and with shift: 0 only the padding changes.

Commits:

  • 6d889b9 adds the transform.
  • 85b971b fixes two defects that the first screen exposed (see Evaluation):
    • the fill is now built from the pre-shift patch;
    • spare_label_faces is added;
    • _shift now gathers on the device, so sparing adds no host sync.
  • 5a8b044 sets the shipped config to the best arm.

Other details:

  • Per-sample selection is pinned. Without a pipeline section the trainer forces same_on_batch=True onto every child, which would make p one coin per batch and could empty both samples of a step of spine.
  • The EM is batched and weighted. It is a form of _em_gmm_1d with no boolean indexing and no host read.
  • The EM moments use a broadcast product instead of resp.mT @ design: 0.04 ms against ~7.5 ms on this shape. Without it the transform cost 190 ms per batch.
    • synthseg.functional._em_gmm_1d has the same pattern and probably the same cost. It is untouched here and left for a separate PR.
  • New config: transform_params_one-sequence-to-segment-them-all-fov.json (d35b9d91). It is c69b9872 plus the transform with shift 1.0, regions [3, 10], margin 4, layers 6, spare_label_faces true and p 0.5, and nothing else. It is byte-identical to the screen's fov2_spare arm. FOV stays opt-in, and the paper config is unchanged.
  • The README gets a "Field of view beyond the scan" section, and the matrix and all_augmentations.json are regenerated.

Checks

All on 5a8b044.

  • pytest unit_tests, including the slow tests, with one A40 visible: 497 passed, 13 skipped, 1570 subtests. main (7169464) in the same venv: 469 passed, 13 skipped, 1519 subtests.

  • The 28 new tests in test_fov_extension.py cover:

    • image and mask moving together, checked with a sentinel intensity under the label;
    • no foreground intensity reaching the fill;
    • bit-identity outside the fill;
    • the no-background case;
    • per-sample selection under same_on_batch=True;
    • seed determinism and bf16 autocast;
    • EM against two known populations;
    • the fill using background the shift removed;
    • no single region dominating a large-shift fill;
    • a fully vacated window getting a varied fill;
    • a through-label not being cut, and a one-face label shifting only away from its face;
    • the touch tolerance equalling margin;
    • sparing per row, with no RNG consumed.

    The fill and sparing tests fail on 6d889b9.

  • The registry sweeps pick the transform up automatically.

  • Pre-commit hooks (ruff) pass over 7169464..5a8b044. mypy smauglab/ reports no issues, run locally with mypy 2.1.0 (CI pins 2.4.0).

  • Fill check on 8 real 803 patches shifted so that about 11% or 0% of the scan stays in the window:

    6d889b9 85b971b
    share of the fill covered by the layers 8-39% 97.6% (median)
    largest region's share of the fill 75-99% 54% (median)
    fully vacated windows flat 0 none flat

    The SI axis was spared on both faces in 8/8 patches, the other two axes in 0/8.

  • Cost on an A40, from the training logs: median epoch 60.5-62.1 s on the four training sets, against 50.5-52.0 s for the paper config on the same set and card model (+19-23%).

Evaluation

The results are in their own comment: Dataset803 (both screens), and the shipped config on 804, 806 and 807. On all four training sets:

  • it removes most far false positives on NAKO VIBE and raises raw class-mean Dice by 0.06-0.12 over the paper config, with no post-processing;
  • its in-domain cost is -0.0023 to -0.0056 Dice, TOST-equivalent at ±0.01 but outside replicate noise where that is measured;
  • it does not beat the paper config plus largest-connected-component post-processing;
  • its recall cost ranges from inside replicate noise on every FN readout but one (804) to 7.6x the reference's FN more than 5 mm from any prediction (806).

Known limits

The limits of the evaluation itself are at the end of the Evaluation comment.

  • Background that the scanner itself zeroed normalises to a constant other than 0, so it counts as scan and stays. An example is the masked air of a Dixon reconstruction.
    • It forms constant bands along some SpineGAN slab edges, and on some 803 patches it is 30-41% of the fill sources, where it shows as flat straight bands in the fill.
    • nnU-Net's -1 marks only part of it, about 17% in the case checked, so -1 is not used.
  • There is no femur or pelvis texture among the sources, so the ~5% of FPs on bone may persist.
  • A label-to-image generator later in the pipeline (SynthSeg, PALETTE synthesis) overwrites the fill.

🤖 Generated with Claude Code

Hendrik-code and others added 3 commits October 28, 2026 16:15
…ith EM-synthesised background

A model trained on sagittal spine slabs has never been shown what lies beside them.
nnU-Net centres the ~8 cm slab in a 128 mm patch and pads the rest with 0, so every
training window contains spine and everything beyond +-4 cm is a flat slab labelled
background. On whole-body axial NAKO VIBE the Ours setup then predicts vertebrae in
muscle, gluteals, fat and bowel: vertebra instance precision 0.01 with ~100 false-positive
components per subject, and 60% of the false-positive voxels more than 5 cm from the
spine. That is missing context, not a contrast failure, and no intensity augmentation
reaches it.

`RandomFOVExtensionGPU` (GEO, placed right after `RandomAffineGPU`) does, per selected
sample:

* shifts image and mask by the same whole-voxel offset, up to `shift` of the patch per
  axis, so the spine reaches the window edge or leaves it;
* fills everything that is not scan -- the exact-`pad_value` padding plus the vacated
  area -- SynthSeg-style: EM splits the patch's own background (label 0, beyond `margin`
  of any label) into 3-10 intensity regions, a majority filter and a random diffeomorphic
  field post-process the region map, warped copies of it are composited into the fill,
  and each region is rendered from its own mean and std with spatially correlated
  texture, then blurred as SynthSeg blurs. The fill's label is 0.

Row selection is pinned per sample: without a `pipeline` section the trainer forces
`same_on_batch=True` onto every child, which would make `p` one coin per batch and could
take the spine out of both samples of a step.

The EM is a batched, weighted form of `_em_gmm_1d` with no boolean indexing and no host
read. Its moment accumulation is a broadcast product rather than `resp.mT @ design`:
cuBLAS takes ~7.5 ms for that reduction shape, the product ~0.04 ms.

`transform_params_one-sequence-to-segment-them-all-fov.json` (ae065824) is the paper's
Ours config (c69b9872) plus the transform at its defaults and nothing else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t a label at the window face

Two changes to RandomFOVExtensionGPU, both found reading the 803 screen of #144.

The fill was fitted on the patch after the shift. A far shift leaves 3-8% of
the scan in the window, so the EM, the atlas and the render statistics came
from that sliver, and each layer warped what the previous layers had written:
on real 803 patches one region filled 75-99% of a far-shifted window, and a
window with nothing left stayed flat 0. Under shift 1.0 about 60% of shifted
samples keep less than 10% of the scan, so most spine-free training windows
were one homogeneous blob. The EM, majority filter, deformation and region
statistics now run on the patch as it arrives, and every layer warps that
fixed full map.

spare_label_faces (default true) stops the shift vacating a window face that
a label comes within `margin` voxels of. A structure may still leave through
that face, as at any window edge, but never gets synthetic tissue against it.
The spine touches both faces along its length, so it is no longer shifted
along that axis: on NAKO VIBE the shift-1.0 model lost recall at the superior
end of the spine and in the posterior elements. Offsets are computed on the
device from params["seg"] in both hooks, so image and mask cannot disagree
and sparing costs no host sync; `_shift` is now a per-axis gather for that.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The shipped -fov config is now byte-identical (d35b9d91) to fov2_spare, the best
arm of the Dataset803 screen: the paper config plus RandomFOVExtensionGPU at the
defaults, except shift 1.0. spare_label_faces is written out although it is the
default.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@NathanMolinier

Copy link
Copy Markdown
Collaborator
Without the transform With the transform
image (1) image (2)

@Hendrik-code

Copy link
Copy Markdown
Collaborator Author

Evaluation and results

Updated 2026-11-13. The shipped arm fov2_spare now has runs on all four SpineGAN training sets: Dataset803 (Dixon in-phase), 804 (CT), 806 (Dixon fat) and 807 (Dixon water). Before, only 803 had been screened on v2.

Setup. Fold 0, one run per arm.

  • Arms are chosen on NAKO VIBE val and reported on test.
  • The class-mean Dice is panoptica's global Dice for the paper cohorts. For NAKO VIBE it is recomputed against VibeSeg-100, four parts pooled (436 subject-parts on test, 62 on val).
  • TOST is at ±0.01, alpha 0.05. Intervals are 95% t-intervals of the paired per-subject delta.

References, all the paper config c69b9872:

  • On 803 and 804, four runs: ms0_base, the paired reference, on 7169464; Ours (fp32); two more replicates on 7169464. Their spread is the noise floor.
  • On 806 and 807, Ours (fp32) alone. No replicate and no v1 arm ran there, so nothing measures the run-to-run term on those two sets.

Criteria, written down before each launch:

  1. Against v1 fov_shift100: total, superior-fifth and posterior-element FN lower; far FP at most 2x.
  2. Raw NAKO VIBE Dice ≥ the reference's after largest-connected-component (LCC) post-processing, on vertebra, IVD and canal.
  3. Every paper cohort TOST-equivalent to the reference.

fov2_spare on the four training sets

NAKO VIBE test, means per subject-part, reference → fov2_spare. The reference is the range over the four replicates on 803/804 and Ours (fp32) on 806/807.

803 Dixon in-phase 804 CT 806 Dixon fat 807 Dixon water
class-mean Dice, raw 0.683-0.715 → 0.795 0.692-0.744 → 0.767 0.695 → 0.796 0.674 → 0.790
reference's class-mean Dice after LCC 0.810-0.815 0.783-0.796 0.807 0.808
FP > 20 mm from the spine 30.1k-43.3k → 2.32k 13.1k-34.3k → 5.18k 43.4k → 0.15k 49.0k → 2.15k
FN, all 12.0k-12.7k → 15.5k 8.18k-9.53k → 9.31k 12.7k → 19.3k 14.2k → 19.1k
FN > 5 mm from any prediction 305-408 → 632 78-156 → 195 427 → 3251 832 → 2519
in-domain Dice vs the paired reference -0.0056 -0.0023 -0.0050 -0.0053
criterion 1 / 2 / 3 pass / fail / fail pass / fail / fail open / fail / fail open / fail / pass
  • The far-FP removal holds on every training set.

    • Far FP falls 2.5-6.6x on 804, 13-19x on 803, 23x on 807 and about 290x on 806.
    • Raw class-mean Dice over the paired reference: +0.080 (803), +0.058 (804), +0.101 (806), +0.115 (807).
    • The four replicates span 0.032 (803) and 0.052 (804) in raw class-mean Dice. The 806/807 gains are about twice the wider of the two, though that bar is borrowed.
  • The LCC baseline is not beaten on any of them.

    • Raw fov2_spare is 0.011-0.029 below the paired reference after LCC in class mean.
    • It is below on every class of every set (criterion 2, test, vertebra / IVD / canal): 803 -0.003 / -0.012 / -0.030, 804 -0.022 / -0.025 / -0.038, 806 -0.011 / -0.004 / -0.020, 807 -0.022 / -0.013 / -0.019.
  • The recall cost depends on the training set. Against Ours (fp32), the one reference all four share:

    803 804 806 807
    FN, all +22% +14% +52% +35%
    FN > 5 mm from any prediction 1.8x 2.5x 7.6x 3.0x
    • On 804, total, superior-fifth and posterior FN stay inside the replicate range.
    • On 806 and 807 the FN rise (+6607 and +4959 voxels) is 3.7-9.3x the FN spread of the four replicates on 803 (0.71k) and 804 (1.35k).
  • On 806 and 807 the lost IVD recall goes to background, not to vertebra. On 803 about two thirds of it was IVD labelled as vertebra, and on 804 nearly all of it. On 806/807, as a share of reference IVD voxels:

    • IVD labelled as vertebra does not rise (0.120 → 0.112, 0.155 → 0.149);
    • IVD labelled as background does (0.062 → 0.102, 0.056 → 0.074).
  • In-domain cost on all four, from -0.0023 to -0.0056.

    • Each is TOST-equivalent with p < 0.001.

    • Each is outside the replicate range where one exists: -0.0001 to +0.0004 on 803, -0.0006 to -0.0002 on 804.

    • The source domain is fitted more slowly on all four. From the training logs, set by set:

      803 804 806 807
      pseudo-Dice, mean of the last 100 epochs, Ours (fp32) → fov2_spare 0.904 → 0.897 0.909 → 0.906 0.889 → 0.883 0.888 → 0.880
      epoch where the EMA first reaches Ours (fp32)'s plateau - 0.014 (0.890 / 0.895 / 0.875 / 0.874) 157 → 610 73 → 204 200 → 579 309 → 696
  • Criterion 3 never fails in domain:

    • on 803, only SpineGAN CT, on the upper side (n = 27);
    • on 804, the three SpineGAN Dixon cohorts, which are out of domain there;
    • on 806, only SpineGAN Dixon water;
    • on 807, none.
  • Cost. All runs are on A40s. The median epoch is 60.5-62.1 s, against 50.5-52.0 s for the paper config on the same set (+19-23%).

What this transform is, on this evidence. It is in-model removal of far false positives on wide-field-of-view data, on all four training sets.

  • On NAKO VIBE it raises raw class-mean Dice by 0.06-0.12 over the paper config, with no post-processing.
  • It does not beat the paper config plus LCC post-processing on any of them.
  • Its in-domain cost (-0.0023 to -0.0056) is inside the ±0.01 margin, but outside replicate noise where that is measured.
  • Its recall cost ranges from inside replicate noise (804, except FN > 5 mm) to 7.6x the FN more than 5 mm from any prediction (806).

Dataset803 (SpineGAN Dixon in-phase)

First screen (6d889b9, L40S)

Dice delta vs ms0_base, test:

arm NAKO VIBE fat / in / out / water SpineGAN Dixon in-phase (in-domain) paper cohorts TOST-equivalent
fov_fill (shift 0) -0.003 / +0.032 / +0.015 / -0.013 -0.0001 all
fov_shift050 (shift 0.5) +0.014 / +0.043 / +0.059 / +0.058 -0.0060 all but SpineGAN CT
fov_shift100 (shift 1.0) +0.066 / +0.089 / +0.083 / +0.081 -0.0092 all but Dixon in-phase
  • fov_shift100 removed 94% of the far FPs: voxels more than 20 mm from the reference spine fell from 37.9k to 2.30k per subject-part.
  • It failed two of the three pre-stated criteria:
    • recall fell, with FN up 43%, concentrated in the posterior elements and the superior fifth of the spine;
    • ms0_base with LCC post-processing still beat it on every class;
    • in-domain Dixon in-phase fell by 0.0092, so TOST failed.
  • Two causes in the code:
    1. The fill was built from what the shift left. After a large shift only a sliver of scan remains, so one EM region filled 75-99% of the window, and a fully vacated window stayed flat 0. Under U[-1, 1]³ about 60% of shifted samples keep under 10% of the scan.
    2. Every SI shift cut the spine. The spine runs through the window along SI, so the shift gave it an artificial end with background-labelled tissue beyond.

85b971b fixes both. The criteria for the second screen were written down before it launched.

Second screen (85b971b, A40)

Both arms use fov_shift100's settings. fov2_nospare turns sparing off, so fov2_spare - fov2_nospare isolates it. NAKO VIBE test, means per subject-part:

4 replicates v1 fov_shift100 fov2_nospare fov2_spare (shipped)
class-mean Dice, raw 0.683-0.715 0.795 0.789 0.795
class-mean Dice, after LCC 0.810-0.815 0.805 0.791 0.807
FP > 20 mm from the spine 30.1k-43.3k 2.30k 0.64k 2.32k
FN, all 12.0k-12.7k 17.4k 20.7k 15.5k
FN > 5 mm from any prediction 305-408 1487 3812 632
FN, superior fifth of the spine 2.49k-2.69k 4.45k 5.97k 3.51k
FN, posterior elements 5.97k-6.53k 9.06k 9.86k 7.94k
SpineGAN Dixon in-phase, Dice vs ms0_base -0.0001 to +0.0004 -0.0092 -0.0079 -0.0056
pre-stated criterion fov2_spare fov2_nospare
1. vs v1: total, superior-fifth and posterior FN lower; far FP at most 2x pass (val and test) fail: FN above v1
2. raw Dice ≥ ms0_base + LCC on vertebra, IVD and canal fail: -0.003 / -0.012 / -0.030 fail
3. every paper cohort TOST-equivalent to ms0_base fail, only on SpineGAN CT fail, only on SpineGAN CT
  • Sparing is the change that recovers recall.

    • fov2_spare - fov2_nospare, all FN: -5259 [-5816, -4701].
    • FN more than 5 mm from any prediction: -3180 [-3599, -2760].
    • Every paper cohort gains +0.0006 to +0.0035 Dice.
    • Far FP rises by 1686 voxels.
  • The fill fix alone makes the model more conservative than v1: FN +3283 and far FP -1665.

  • Against v1, fov2_spare recovers part of the recall it lost, measured as the share of v1's excess over the replicate mean:

    • total FN: 39%;
    • FN more than 5 mm from any prediction: 75%;
    • superior-fifth FN: 51%;
    • posterior-element FN: 41%.

    Far FP stays at v1's level (+21 [-327, 368] voxels), and the in-domain cost is cut from -0.0092 to -0.0056.

  • Criterion 3 fails on the upper side. On SpineGAN CT, fov2_spare is +0.0053 with interval [-0.0015, +0.0122] (n=27), so equivalence cannot be shown. Every other paper cohort is equivalent with p < 0.001.

  • Criterion 2 fails on IVD. With LCC applied to every model, fov2_spare is:

    • above all four replicates on spinal canal (0.830 vs 0.824-0.828);
    • 0.002-0.005 below them on vertebra;
    • 0.011-0.020 below them on IVD.

    About two thirds of its lost IVD recall is IVD predicted as vertebra (17.6% of reference IVD voxels, vs 14.1-14.6% for ms0_base and one replicate). That is a class-boundary error, not missed tissue.

Dataset804 (SpineGAN CT)

fov2_spare only, from the same checkout as on 803 (85b971b). Sparing was settled on 803, so fov2_nospare did not run. The references are the four replicates and both v1 arms: fov_shift100 (the same leaves, and the criterion-1 reference) and fov_shift050. All three FOV arms trained on A40s, so fov2_spare - fov_shift100 has no card term here, unlike on 803. It still carries a different random stream.

criterion val test
1. recall recovered, far FP kept (vs fov_shift100) pass (far FP 0.76x v1) pass (far FP 1.31x v1)
2. raw Dice ≥ ms0_base + LCC, vertebra / IVD / canal fail: -0.020 / -0.025 / -0.033 fail: -0.022 / -0.025 / -0.038
3. every paper cohort TOST-equivalent to ms0_base fail on the three SpineGAN Dixon cohorts fail on the three SpineGAN Dixon cohorts

NAKO VIBE test, means per subject-part. The share recovered is (v1 - v2) / (v1 - replicate mean), against fov_shift100.

4 replicates v1 fov_shift050 v1 fov_shift100 fov2_spare spare - fov_shift100 recovered
FP > 20 mm 13.1k-34.3k 16.5k 3.96k 5.18k +1218 [+715, +1721]
FN, all 8.18k-9.53k 8.02k 10.63k 9.31k -1326 [-1406, -1246] 71%
FN > 5 mm 78-156 127 395 195 -200 [-236, -164] 71%
FN, superior fifth 1.78k-2.12k 1.70k 2.67k 2.08k -592 [-649, -536] 79%
FN, posterior elements 4.17k-5.26k 4.37k 5.83k 5.01k -821 [-874, -768] 67%
Dice vertebra / IVD / canal, raw 0.720-0.778 / 0.727-0.769 / 0.629-0.690 0.758 / 0.764 / 0.694 0.780 / 0.741 / 0.773 0.780 / 0.755 / 0.767 -0.000 / +0.013 / -0.007
Dice vertebra / IVD / canal, LCC 0.794-0.802 / 0.761-0.782 / 0.794-0.805 0.793 / 0.772 / 0.805 0.789 / 0.746 / 0.804 0.792 / 0.756 / 0.805 +0.003 / +0.010 / +0.001
class mean, raw / LCC 0.692-0.744 / 0.783-0.796 0.739 / 0.790 0.765 / 0.780 0.767 / 0.784
  • Criterion 1.

    • Against v1 at the same shift, fov2_spare recovers 67-79% of the recall loss on test and 62-76% on val. On 803 that was 39-75%.
    • Only two FN deltas are larger than the spread of the four replicates: FN > 5 mm (200 vs 78) and superior-fifth FN (592 vs 339). Total and posterior FN are not (1326 vs 1354, 821 vs 1090). The criterion asks only for the direction, so it passes, but on 804 those two cannot be told from run-to-run noise. On 803 all four were resolved.
  • Recall against the replicates. FN, superior-fifth FN and posterior FN are inside the replicate range. Only FN > 5 mm is above it (195 vs at most 156). On 803 every FN readout stayed above the replicates.

  • The raw NAKO gain over the replicates is mostly canal.

    • Raw canal Dice is +0.08 over the best replicate.
    • The canal volume ratio falls from 1.28-1.47 to 1.01: the replicates over-segment the canal, and both shift-1.0 arms do not.
    • Raw vertebra is 0.002 above the best replicate. Raw IVD is inside the range.
  • Paired against ms0_base, raw class-mean Dice is +0.042 to +0.070 per VIBE part. The other three replicates span -0.033 to +0.049, and fov2_spare is above their range on every part, by 0.016-0.031.

  • Against v1 at the same shift, the class-mean Dice is the same, as on 803: raw 0.767 vs 0.765. v2 is better on IVD (+0.013) and worse on canal (-0.007).

  • The IVD deficit is the 803 pattern again. Share of reference IVD voxels, test:

    ms0_base rep2 v1 shift 0.5 v1 shift 1.0 fov2_spare
    predicted IVD 0.752 0.752 0.730 0.693 0.712
    predicted vertebra 0.221 0.223 0.245 0.272 0.259
    predicted background 0.026 0.025 0.024 0.035 0.029

    The IVD-as-vertebra share grows with the shift. At shift 1.0, v2 takes back about a quarter of v1's excess over the replicates (0.037 vs 0.050).

Paper cohorts, test Dice delta vs ms0_base:

cohort n other three replicates v1 shift 0.5 v1 shift 1.0 fov2_spare
SpineGAN CT (in-domain) 27 -0.0006 to -0.0002 -0.0010 -0.0051 -0.0023 [-0.0034, -0.0012]
SpineGAN Dixon in-phase 29 -0.0081 to -0.0014 -0.0171 -0.0075 -0.0109 [-0.0176, -0.0042]
SpineGAN Dixon fat 29 -0.0072 to -0.0023 -0.0121 -0.0114 -0.0132 [-0.0188, -0.0076]
SpineGAN Dixon water 29 -0.0017 to +0.0023 -0.0075 +0.0014 -0.0079 [-0.0149, -0.0009]
Myelom CT 620 -0.0017 to +0.0001 +0.0022 -0.0011 +0.0028
SPIDER T1w 171 -0.0028 to +0.0030 -0.0022 +0.0024 -0.0016
SPIDER T2w 219 -0.0036 to +0.0054 -0.0029 +0.0010 -0.0004
  • In-domain CT passes TOST (p < 0.001). The cost is half of v1's at the same shift (-0.0023 vs -0.0051), as on 803, but still outside the replicate range.
  • Criterion 3 fails on the Dixon cohorts, which are out of domain on 804.
    • Two of the nine replicate cells there fail the same test: Ours (fp32) on fat (p = 0.13) and rep2 on in-phase (p = 0.19).
    • v1 at shift 1.0 fails it on fat and in-phase.
    • fov2_spare is below the replicate range on all three Dixon cohorts, by 0.003-0.006.
    • On Dixon water it is 0.009 below v1 at shift 1.0. This is the one paper cohort where v2 is clearly worse than v1.
  • The fill on CT has one dominant region, and that comes from the data, not the v1 defect. On 804 patches the largest region holds a median 0.72 of the fill, against 0.54 on 803.
    • The CT background near the spine is almost one tissue: its normalised intensity has a std of 0.09-0.18, against 0.66-1.07 on 803.
    • The largest EM region already holds 54-92% of the source background.
    • The pre-flight gate therefore compares the fill with its source: the largest region may hold at most 0.15 more of the fill than of the source background. 804 exceeds its source by a median 0.007, 803 by 0.022, 806 by 0.025 and 807 by 0.056. v1 would still fail: it put 75-99% of the fill into a region that held 37-62% of the background.

Dataset806 (SpineGAN Dixon fat) and Dataset807 (SpineGAN Dixon water)

fov2_spare only, from the same checkout (85b971b), against Ours (fp32). Criterion 1 is stated against fov_shift100, which has no run here, so it stays open.

criterion 806 val 806 test 807 val 807 test
1. vs fov_shift100 open open open open
2. raw Dice ≥ Ours (fp32) + LCC, vertebra / IVD / canal fail: -0.007 / -0.000 / -0.019 fail: -0.011 / -0.004 / -0.020 fail: -0.025 / -0.014 / -0.032 fail: -0.022 / -0.013 / -0.019
3. every paper cohort TOST-equivalent to Ours (fp32) fail on Dixon water fail on Dixon water pass pass

NAKO VIBE test, means per subject-part, fov2_spare minus Ours (fp32):

806 Ours (fp32) 806 fov2_spare 806 delta 807 Ours (fp32) 807 fov2_spare 807 delta
FP > 20 mm 43.4k 151 -43206 [-45517, -40894] 49.0k 2.15k -46827 [-48389, -45265]
FN, all 12.7k 19.3k +6607 [+6134, +7081] 14.2k 19.1k +4959 [+4766, +5151]
FN > 5 mm 427 3251 +2824 [+2418, +3230] 832 2519 +1687 [+1559, +1814]
FN, superior fifth 2.76k 5.82k +3053 [+2734, +3371] 2.89k 4.48k +1599 [+1491, +1706]
FN, posterior elements 6.30k 9.22k +2913 [+2779, +3046] 7.78k 11.09k +3310 [+3182, +3438]
Dice vertebra / IVD / canal, raw 0.720 / 0.671 / 0.694 0.810 / 0.781 / 0.797 +0.090 / +0.109 / +0.103 0.707 / 0.715 / 0.601 0.796 / 0.766 / 0.807 +0.089 / +0.051 / +0.205
Dice vertebra / IVD / canal, LCC 0.820 / 0.784 / 0.817 0.809 / 0.780 / 0.795 -0.011 / -0.004 / -0.022 0.818 / 0.779 / 0.826 0.801 / 0.770 / 0.820 -0.017 / -0.009 / -0.006
class mean, raw / LCC 0.695 / 0.807 0.796 / 0.795 0.674 / 0.808 0.790 / 0.797
  • Far FP is almost gone, and the raw gain is the largest of the four sets. Raw class-mean Dice rises on every VIBE part: +0.071 to +0.123 on 806 and +0.105 to +0.122 on 807.

  • The recall cost is also the largest of the four sets.

    • Superior-fifth FN is 2.1x Ours (fp32)'s on 806 and 1.55x on 807.
    • Posterior-element FN is 1.46x and 1.43x.
    • Without a v1 arm it is open how much of this v2 already recovered, as it did on 803 and 804.
  • The lost recall is missed tissue. Share of reference voxels, Ours (fp32) → fov2_spare:

    806 807
    vertebra predicted as background 0.111 → 0.166 0.127 → 0.175
    IVD predicted as background 0.062 → 0.102 0.056 → 0.074
    IVD predicted as vertebra 0.120 → 0.112 0.155 → 0.149
    vertebra volume ratio, predicted / reference 1.35 → 0.95 1.37 → 0.97

    Ours (fp32) over-segments vertebra by about a third, and fov2_spare under-segments it slightly.

Paper cohorts, test Dice delta vs Ours (fp32). In-domain in bold:

cohort n 806 807
SpineGAN Dixon fat 29 -0.0050 [-0.0074, -0.0027] +0.0018 [-0.0007, +0.0043]
SpineGAN Dixon water 29 -0.0078 [-0.0134, -0.0022], not equivalent (p = 0.21) -0.0053 [-0.0075, -0.0031]
SpineGAN Dixon in-phase 29 -0.0042 [-0.0064, -0.0020] -0.0031 [-0.0048, -0.0014]
SpineGAN CT 27 +0.0003 [-0.0026, +0.0032] +0.0020 [-0.0048, +0.0087]
Myelom CT 620 -0.0037 [-0.0046, -0.0029] -0.0047 [-0.0052, -0.0042]
SPIDER T1w 171 -0.0006 [-0.0014, +0.0001] -0.0000 [-0.0010, +0.0009]
SPIDER T2w 219 -0.0008 [-0.0014, -0.0002] -0.0009 [-0.0016, -0.0002]
  • In-domain, the cost is -0.0050 on 806 and -0.0053 on 807. Both are TOST-equivalent with p < 0.001. Both are larger than any in-domain replicate difference measured on 803 or 804 (at most 0.0006 against ms0_base), but that bar is borrowed.
  • Criterion 3 fails on 806 only on Dixon water, an out-of-domain Dixon part, whose interval crosses -0.01. On 804, two of nine replicate cells failed the same test on the out-of-domain Dixon cohorts, so one failure there may be run-to-run noise. 806 has no replicate to check this.

Limits of this evaluation

  • One run per arm. On 803, v1 ran on an L40S and v2 on an A40, so v1-vs-v2 deltas there also carry a card term and a different random stream. On 804 both ran on A40s, so only the random stream differs. The replicate spread bounds these deltas only loosely.
  • The 2x2 has no "v1 fill + sparing" arm, so it is open whether the fill fix still helps once sparing is on. The v1 fill was degenerate in any case, as shown in Checks.
  • fov2_nospare ran on 803 only.
  • 806 and 807 have no replicate and no v1 arm. Criterion 1 is untested there, and every resolution bar on those two sets is borrowed from 803 and 804.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants