Skip to content

Fix the marking and accuracy columns, and watch each cycle for parsing errors - #7

Closed
RISCfuture wants to merge 7 commits into
mainfrom
dof-column-fix-and-cycle-watch
Closed

RISCfuture wants to merge 7 commits into
mainfrom
dof-column-fix-and-cycle-watch

Conversation

@RISCfuture

Copy link
Copy Markdown
Owner

Testing the parser against the current distribution (cycle 20260802, published 2026-09-14) reported zero errors — while misreading one field on every record. This fixes that, then adds the workflow that would have caught it.

The bug

Per the FAA README shipped inside the distribution, the tail of each record is:

Column (1-idx) Spec Parser read it as
98 Horizontal Accuracy (19) ✓ correct
100 Vertical Accuracy (AI) marking
102 Mark Indicator (P W M F S N U) faaIndicator — declared, never read ✗

MarkingType's raw values were AI, coinciding exactly with the vertical accuracy code set, so every record parsed "successfully" with a wrong value and vertical accuracy was never exposed. A column tally over all 652,785 records confirms it: column 100 holds only AI, column 102 only P W M F S N U. The test fixture pinned the wrong answer — … R 5 D M … asserted marking == .paintAndFlags, reading the D that means ±50 ft height tolerance.

The distribution data is correct; the parser was wrong.

Changes

  • HorizontalAccuracy (19) and VerticalAccuracy (AI) replace AccuracyCategory, which mixed both code sets. Breaking.
  • MarkingType rewritten to the real codes. Breaking.
  • Obstacle.verticalAccuracy added; marking now reads column 102.
  • Cases are named for the code the FAA publishes. These are not a file-format encoding — the FAA has used Accuracy Codes since 1979 and still writes them as "Accuracy Code 1A" (PARC recommendation); the spec's own columns read Code and Tolerance, never "category".
  • DOF Cycle Watch workflow, modelled on SwiftNASR's, parses each cycle as it is released and opens an issue on parse errors, count drift, or a wrong currency date.
  • SwiftDOF_E2E --report / --baseline produce the JSON the workflow reads; the tool now exits non-zero when any line fails to parse.

Verification

  • All 652,785 records cross-checked against an independent read of the raw columns — horizontal accuracy, vertical accuracy, marking, lighting, action, verification, AGL, MSL, quantity: 0 mismatches.
  • The previous distribution (20260607, 641,207 obstacles) also parses with 0 errors.
  • All 11 supplemental files parse with exact record counts and 0 errors.
  • Cycle arithmetic checked against the FAA's published schedule, including DOF (Next); live probe returns 200 for released cycles and 404 for the unreleased one.
  • Drift and parse-error detection both exercised with negative tests.
  • Drift tolerances calibrated on real data: a normal 56-day cycle moves the total 1.8%, against a 5% tolerance, so a clean cycle reports no drift.
  • swift test (64), swiftlint --strict, swift format lint --strict, actionlint, yamllint, and DocC all clean.

Notes

  • The combined 4D / 1A accuracy code is not exposed as a property; say the word if it would be useful. The two codes dominate the file (322,448 and 159,603 obstacles).
  • Unrelated FAA data inconsistency spotted: obstacle 01-000525 is in DOF.DAT but missing from the per-state 01-AL.Dat extract.

🤖 Generated with Claude Code

RISCfuture and others added 7 commits September 14, 2026 20:38
Adopt typed throws across the parsing surface: `DOF.init(data:)`,
`DOF.init(url:)`, the `DOF.from(…)` factories, `DOFByteParser`, and the
DOF file line reader's `AsyncIteratorProtocol.Failure` now carry
`DOFError`, while `AsyncBytesLineReader` propagates its source
sequence's own `Failure` type.

Extract the chunked file reader into `FileLineReader` so both
`AsyncDOFLineReader` and the new synchronous streaming initializer share
one implementation, and route `DOF.from(filePath:)` through it instead
of reading the whole file into memory.

Match the "CURRENCY DATE = " header marker against an
`InlineArray<16, UInt8>`, removing a heap allocation and the crash on
input shorter than the marker.

Replace the force-unwraps in `Cycle.previous`, `Cycle.next`, and the
cycle datum date with a shared failable helper and a precondition.

Add Swift 6.4 CI legs alongside the newest leg for each OS already in
the matrix.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbZbx5D2gGeXT8UxuiKEdq
Extracting the chunked reader out of the async iterator left
`AsyncIterator.bytesRead` forwarding to a reader nothing asks it about;
progress reporting reads `FileLineReader.bytesRead` directly. Periphery
flags it, failing the strict scan.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbZbx5D2gGeXT8UxuiKEdq
`main` lowered this package's floor to what its code there requires. The
byte-parsing work on this branch uses `InlineArray`, which is macOS 26, so the
branch declares the floor its own code needs.

Merging this therefore raises the floor. That is the trade the branch asks
for and it should be decided on the merge, not worked around in the source.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbZbx5D2gGeXT8UxuiKEdq
Enable `.strictMemorySafety()` (SE-0458) alongside the existing upcoming
feature flags and audit every unsafe construct it surfaces, marking each
with the `unsafe` expression marker.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbZbx5D2gGeXT8UxuiKEdq
`main` lowered the floor to macOS 15 and the README followed. This branch
raises it to 26 for `InlineArray`, so the README has to say so too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EbZbx5D2gGeXT8UxuiKEdq
Obstacle.marking was read from column 100, which holds the vertical
accuracy code, and the real mark indicator in column 102 was declared as
a field and never read. The old MarkingType raw values were A-I, exactly
the vertical accuracy code set, so every record in every cycle parsed
without error and reported a marking derived from the obstacle's height
tolerance: a +/-50 foot obstacle (D) read as .paintAndFlags.

Split AccuracyCategory, which mixed both code sets, into HorizontalAccuracy
(1-9) and VerticalAccuracy (A-I), and expose the vertical column that had
no representation at all. Cases are named for the code the FAA publishes,
which it has used since 1979 and still writes as "Accuracy Code 1A".

Verified against cycle 20260802: all 652,785 records now agree with an
independent read of the raw columns across every parsed field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The FAA releases each 56-day DOF the day after it takes effect, so there
is no window in which the incoming cycle can be parsed early. A daily
probe checks whether the cycle in effect is downloadable, and the first
time it is, parses it end to end. The report artifact, named for the
cycle, is both the record that the cycle was checked and the baseline the
next cycle's counts are compared against.

SwiftDOF_E2E gains the report the workflow reads: --report writes counts,
per-region totals, and the field each failed line failed on, --baseline
records counts that moved more than their tolerance, and the tool now
exits non-zero when any line fails to parse. Periphery retains Codable
properties, since jq is the only reader the report has.

Tolerances are calibrated against real data: cycle 20260607 to 20260802
moved the total by 1.8%, well inside the 5% allowed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@RISCfuture

Copy link
Copy Markdown
Owner Author

Superseded — these commits went directly into main ahead of Modernize for Swift 6.4 as 0cde9b0 and 1b28a73.

@RISCfuture RISCfuture closed this Sep 17, 2026
@RISCfuture
RISCfuture deleted the dof-column-fix-and-cycle-watch branch September 17, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant