Is your feature request related to a problem? Please describe.
Auron currently falls back from native round-robin shuffle when the shuffle payload contains a Spark MapType, including maps nested in arrays or structs.
Native shuffle serialization already supports Map values. However, round-robin shuffle sorts its input before repartitioning to provide stable partition assignment, and Arrow row encoding does not support MapType.
Describe the solution you'd like
Allow MapType payloads in native round-robin shuffle.
When the input schema contains a Map, use Auron's Spark-compatible Murmur3 hash over all payload columns as the internal sort key. Keep the existing all-column sort path unchanged for schemas without maps.
Apply the same behavior to both local and RSS shuffle writers, and add coverage asserting that Map repartitioning uses NativeShuffleExchangeExec.
Describe alternatives you've considered
Skipping the pre-sort for Map payloads was considered, but that would weaken stable round-robin partition assignment.
Adding Map ordering support to Arrow row encoding would be a significantly broader change, and Map is not an orderable Spark SQL type.
Additional context
This only allows Map values to be carried as round-robin shuffle payloads. It does not add support for using MapType as a range sort key or hash partitioning key.
Is your feature request related to a problem? Please describe.
Auron currently falls back from native round-robin shuffle when the shuffle payload contains a Spark
MapType, including maps nested in arrays or structs.Native shuffle serialization already supports Map values. However, round-robin shuffle sorts its input before repartitioning to provide stable partition assignment, and Arrow row encoding does not support
MapType.Describe the solution you'd like
Allow
MapTypepayloads in native round-robin shuffle.When the input schema contains a Map, use Auron's Spark-compatible Murmur3 hash over all payload columns as the internal sort key. Keep the existing all-column sort path unchanged for schemas without maps.
Apply the same behavior to both local and RSS shuffle writers, and add coverage asserting that Map repartitioning uses
NativeShuffleExchangeExec.Describe alternatives you've considered
Skipping the pre-sort for Map payloads was considered, but that would weaken stable round-robin partition assignment.
Adding Map ordering support to Arrow row encoding would be a significantly broader change, and Map is not an orderable Spark SQL type.
Additional context
This only allows Map values to be carried as round-robin shuffle payloads. It does not add support for using MapType as a range sort key or hash partitioning key.