Search before asking
Description
Integrate the BulkLoad client SDK with the Flink connector so a bounded
INSERT INTO job can efficiently populate an empty Fluss primary-key table or
partition.
The Flink integration should coordinate one server transaction, build buckets
in parallel, and publish the target only after every bucket has completed.
Scope
- Add an explicit connector option for selecting the BulkLoad path.
- Enable BulkLoad only for bounded Flink jobs and supported primary-key tables.
- Begin one BulkLoad transaction for the physical target.
- Distribute the immutable build context to parallel build tasks.
- Assign every input row to its Fluss bucket.
- Build bucket snapshots in parallel through
openBuildSession.
- Collect one completed result for every target bucket.
- Commit the complete manifest through one coordinated commit path.
- Abort or report a clear failure when the job cannot complete before the
durable Commit decision.
- Keep temporary credentials out of Flink records, committables, checkpoints,
and savepoints.
- Document configuration, supported cases, failure behavior, and operational
limitations.
Willingness to contribute
Search before asking
Description
Integrate the BulkLoad client SDK with the Flink connector so a bounded
INSERT INTOjob can efficiently populate an empty Fluss primary-key table orpartition.
The Flink integration should coordinate one server transaction, build buckets
in parallel, and publish the target only after every bucket has completed.
Scope
openBuildSession.durable Commit decision.
and savepoints.
limitations.
Willingness to contribute