Preserve PyArrow extension metadata when chaining Python scalar UDFs - #1287

Closed

kosiew wants to merge 18 commits into

apache:mainfrom

kosiew:udf-1172

Commits on Oct 22, 2025

Implement metadata-aware PySimpleScalarUDF
kosiew
committed
d0b02a6
Clone pyarrow.Field objects for FFI handoff
kosiew
committed
2325993
Updated the _function and _decorator helpers in ScalarUDF.udf to use concrete Callable[..., Any] and pa.DataType | pa.Field annotations, removing the lingering references to the deleted _R type var…
kosiew
committed
e0bce84
Updated the internal ScalarUDF.udf helper signatures so IDEs and type checkers surface support for pyarrow.Field inputs when defining UDFs
kosiew
committed
5fb08d6
Refactor imports in udf.rs for improved organization and clarity
kosiew
committed
6a89977
Add shared PyArrowArray alias and refine ScalarUDFs
kosiew
committed
5aacb41
Add feature flag for pyarrow UUID helper detection
kosiew
committed
bae5d54
Add assertions for UUID extension type in tests
kosiew
committed
153b5f1
Teach _wrap_extension_value to handle empty arrays
kosiew
committed
1baa2b7
Refactor type alias for PyArrowArray to use Union for better clarity
kosiew
committed
308e774
Fix Ruff errors
kosiew
committed
16224e2
Enhance documentation for PyArrowArray type alias to clarify usage and improve type-checking
kosiew
committed
7b9ced0

Commits on Oct 23, 2025

Add dev dependency pyarrow >= 19 and remove UUID extension availability check from tests
kosiew
committed
0f28465
Update return_type to raise error for metadata source
kosiew
committed
f068caa
Fix ruff errors
kosiew
committed
1ea75fd

Commits on Nov 1, 2025

reverted accidental formatting
kosiew
committed
27bc012
Merge branch 'main' into udf-1172
kosiew
committed
f9f1799
fix ruff errors
kosiew
committed
6b40052