[Variant] Enable direct top-level Variant projection in ordinary native Parquet scans

[peterxcli]

Problem

After the schema, normalization, and physical-reader layers are ready, Comet still needs a deliberately narrow JVM admission path for direct top-level Variant projection and Spark-compatible behavior for missing columns and unsupported consumers.

Parent: #5546
Depends on: #5550

Solution

• Admit only direct top-level VariantType fields in ordinary native Parquet scans.
• Reuse Unicode Parquet name matching from feat: support unicode case sensitive field names for reading parquet #5602 (Support Spark-compatible Unicode case-insensitive Parquet field matching #5495 is complete); do not port feat: project Spark 4 VARIANT columns in native Parquet scans [Splitting] #5407's non-ASCII-name fallback.
• Build the native data schema from Spark's pruned required schema so unread Variant roots remain removable.
• Preserve existence-default value/index pairing and transport a Variant default as a scan-only constant [value, metadata] struct.
• Apply a default only when the physical column is absent.
• Export the marked Field and reuse CometStructVector so inherited ColumnVector.getVariant works.
• Keep nested Variant, pushed VariantStruct (Support Spark's pushed one-field VariantStruct (whole-value pushVariantIntoScan rewrite) in native Parquet scans #5519), strict malformed reads, expressions, writes, C2R, shuffle/spill, Python operators, encryption, and Iceberg on explicit Spark fallback where applicable.
• Document the supported surface and remaining fallbacks.

Tests

• SELECT v and SELECT id, v, tail remain native and match Spark.
• Objects, arrays, scalars, JSON null, SQL NULL, nullable parents, and entirely-null columns round-trip.
• Spark sees logical VariantType with exact [value, metadata] Binary children.
• A missing Variant column receives its existence default without shifting later defaults.
• Unread Variant pruning from feat: Support native scans with unprojected Spark 4 VARIANT columns #5377 remains native.
• Case-insensitive Unicode Variant projection remains native and matches Spark, covering a non-ASCII Variant root, an ordinary sibling beside Variant, and physical Kelvin sign K matched to logical k. Preserve Spark's missing-column behavior for physical ſ read as logical s.
• Unsupported consumers above have focused fallback assertions.
• Spark 4.0/4.1 profiles pass and Spark 3.5 still compiles unchanged.

Completing this issue closes the ordinary-Parquet phase of #4295. Iceberg and the broader shredded optimization scope in #3983 remain separate.

[peterxcli]

take