[FLINK-40496][table-planner] CREATE OR ALTER MATERIALIZED TABLE honors query column position

[AHeise]

What is the purpose of the change

CREATE OR ALTER MATERIALIZED TABLE derives its schema delta via MaterializedTableUtils.validateAndExtractColumnChanges, which appends new columns but never diffs column position. Adding a column anywhere but the end of the query projection therefore stores a schema whose column order disagrees with the query. Nothing rejects it at DDL time (validateChanges passes); the mismatch only surfaces later, when the refresh runs a positional INSERT INTO <table> <expandedQuery> and the wrong query column binds to the wrong sink column.

ALTER MATERIALIZED TABLE ... AS already diffs position via buildSchemaTableChanges, so the two query-evolution paths disagree. This PR makes CREATE OR ALTER honor query column position the same way.

Example — mt = (a, b, c, d):

CREATE OR ALTER MATERIALIZED TABLE mt AS SELECT a, b, c, 42 AS mid, d FROM t1;

Expected stored schema: (a, b, c, mid, d) (the query projection order, as ALTER ... AS produces).
Before this change: (a, b, c, d, mid) — mid appended at the end.

Brief change log

• validateAndExtractColumnChanges emits modifyColumnPosition for name-matched columns whose index changed, mirroring buildSchemaTableChanges.
• Old columns are positioned by their rank among the columns that survive into the new schema, so retained non-persisted columns (kept by CREATE OR ALTER but absent from the query projection) do not skew the position comparison.

Verifying this change

This change added tests and can be verified as follows:

• MaterializedTableUtilsTest#testCreateOrAlter — a physical mid-projection insert and a case with a retained virtual-metadata column, asserting the position-aware TableChange list.
• SqlNodeToOperationSqlCreateOrAlterMaterializedTableConverterTest — end-to-end over real CREATE OR ALTER SQL, exercising schema resolution, the position diff, and validateChanges: inserting before the last column stores the query order and is accepted; inserting before multiple trailing columns is rejected up front with a clear error instead of miscompiling at refresh time.
• AlterMaterializedTableAsQueryOperationValidationTest — guard boundary: a reposition anchored after a newly-added column is accepted, one anchored after an existing column is rejected.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)

Generated-by: Claude Code (Claude Opus 4.8)

[flinkbot]

CI report:

• 0831111 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build