[fix](be) Backport Parquet metadata size limit to branch-4.1 (#67631)

[Gabriel39]

What problem does this PR solve?

Related PR: #67631

Backport #67631 to branch-4.1. FileScannerV2 incorrectly uses the Thrift RPC message ceiling to limit Parquet metadata. Add the independent, mutable parquet_metadata_size_limit with a 256 MiB default, while preserving the file-size check and enforcing the limit before metadata allocation and the second read.

Resolved a test insertion conflict by retaining the existing physical-split refinement test and adding the upstream footer tests. The added and removed code matches the upstream patch.

Release note

FileScannerV2 uses the independent parquet_metadata_size_limit configuration, defaulting to 256 MiB.

Check List (For Author)

• Validation: clang-format 16 check and git diff --check passed; full diff self-reviewed.
• Unit tests: preserved the upstream footer tests. A local NewParquetReaderTest.NativeFooter* run was attempted but stopped during a full rebuild; no local test result is claimed for this backport. CI validation is required.
• Header hygiene: the check script is unavailable on the target branch.
• Behavior changed: Yes; valid Parquet metadata above the RPC ceiling can be accepted within the dedicated metadata limit.
• Does this need documentation: No.

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[Gabriel39]

run buildall

[Gabriel39]

/review

[github-actions]

Static review: changes requested.

Critical checkpoints:

• Resource-bound correctness: one P2 inline issue remains. A rejected negative runtime update can be observed by a concurrent footer parse and the new signed-to-size_t cast turns it into an effectively unbounded limit.
• Footer parsing and compatibility: the structural file-size check and pre-allocation/second-read ceiling are otherwise preserved. The default 256 MiB path is not blocked by Thrift's old 100 MiB default, and cached/shared physical-split metadata reuse does not create another allocation path.
• Format-v2 architecture and lifecycle: no layer-boundary, schema-identity, cache-ownership, cleanup, or hot-path performance regression was found in the remaining changes.
• Tests and validation: the added tests cover admission arithmetic and file-size boundaries, but not the rejected runtime-update interleaving. This review was static-only; no builds or tests were run here, and the current CI set skipped build/unit-test jobs.
• User focus: no additional focus was supplied.

Review completion: complete after two bounded rounds. All Round 2 reviewers returned NO_NEW_VALUABLE_FINDINGS, every candidate was accepted or dismissed with evidence, and the live open PR still matched base 6a4e486cac4837187c8fa9562372d0977a37d713 and head 271b3076b8861810bd0a8fc37504bdeb60b1bcc4 immediately before submission.

[hello-stephen]

BE Regression && UT Coverage Report

Increment line coverage 100.00% (3/3) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	74.45% (31459/42253)
Line Coverage	58.70% (350652/597404)
Region Coverage	55.33% (292142/528030)
Branch Coverage	56.24% (131994/234710)

np

[github-actions]

PR approved by anyone and no changes requested.

[github-actions]

PR approved by at least one committer and no changes requested.