Skip to content

[GH-3383] Preserve GeoParquet XYZ layouts and document M conversion - #3390

Merged
jiayuasu merged 1 commit into
apache:masterfrom
jiayuasu:codex/oss-geoparquet-dimensions
Sep 23, 2026
Merged

jiayuasu merged 1 commit into
apache:masterfrom
jiayuasu:codex/oss-geoparquet-dimensions

Conversation

@jiayuasu

Copy link
Copy Markdown
Member

Did you read the Contributor Guide?

Yes, I have read the Contributor Rules and Contributor Development Guide.

Is this PR related to a ticket?

Closes #3383. The PR title follows the [GH-XXX] format.

What changes were proposed in this PR?

GeoParquet writes currently infer dimensionality from the first coordinate's Z, so declared empty XYZ geometries, NaN Z values, and later finite Z values can become XY. Preserve the supported XY/XYZ layouts with the existing patched WKB writer and derive geometry_types from the emitted WKB header. Reuse one configured writer per geometry-column closure.

Ignore empty envelopes when accumulating the file bbox and omit bbox metadata when no geometry contributes a spatial extent. Empty geometries still contribute their geometry types.

Document the existing measured-input conversion: GeoParquet 1.1 does not support M, so writes project XYM to XY and XYZM to XYZ. The writer remains limited to XY/XYZ with no embedded SRID, retaining the existing WKB encoding flavor. The tutorials advise storing needed M values separately before writing.

The existing JTS limitation for zero-member collections remains: with no coordinate sequence or member, a declared collection layout cannot be retained. Collections with typed empty members are covered by the regression tests.

How was this patch tested?

  • Reproduced four failures before the fix: two file-write Z-header failures and two empty-bbox failures; both reader fixtures passed.
  • Spark 4.1.3 / Scala 2.13, Java 17: 89 selected GeoParquet tests passed across six suites.
  • Spark 3.5.8 / Scala 2.12, Java 17: the same 89 tests passed.
  • Both clean Maven reactor builds and Spotless checks passed. git diff --check passed.
  • A focused compatibility probe confirmed ordinary two-coordinate point constructors remain XY after geometry serialization.

File tests construct independent ISO WKB fixtures and check both stored WKB and read-back geometries, footer geometry_types, and bbox. Coverage includes empty XY/XYZ primitives, NaN Z, a finite Z after the first coordinate, multipart/nested empty members, both byte orders, null values, measured empty primitives, XYM/XYZM projection, and plain/dictionary input files.

Did this PR include necessary documentation updates?

Yes. Updated the English and Chinese GeoParquet Spark tutorials with XYZ preservation and the explicit M conversion policy.

@jiayuasu jiayuasu added this to the sedona-2.0.0 milestone Sep 23, 2026
@jiayuasu
jiayuasu merged commit 986558a into apache:master Sep 23, 2026
41 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preserve declared Z layouts in GeoParquet writes and handle measured inputs explicitly

1 participant