Encryption reader should reject key metadata without file_length

[xanderbailey]

The AGS1 block layout has to be derived from the trusted file_length in StandardKeyMetadata rather than a filesystem stat, otherwise an attacker can drop trailing blocks and every remaining block still authenticates. The reader should fail when file_length is missing, or when it disagrees with the bytes that are actually readable, matching iceberg-rust after apache/iceberg-rust#3236.

Part of #3222

[anbv29]

Hi! I’d like to understand the encryption-reader invariant, add a failing test for missing file_length, and propose a safe validation fix. Could a maintainer confirm the intended error behavior?

[xanderbailey]

I’ll try and make this explicit in the coming PRs as we progress with the encryption work

[anbv29]

Hi! I’d be happy to help with this. Would you welcome a contribution that adds regression tests for missing or inconsistent file_length metadata, followed by the reader validation fix? If so, could you point me to the relevant reader code or any in-progress PR that I should coordinate with?

[xanderbailey]

We don’t have the encryption reader yet, this is more of a tracking issue to make sure we don’t forget this specific issue rather than it being an active bug. Once the linked PR merges I’ll start working on the readers and writers for streaming encryption and we should be able to close this one! Happy to tag you in those PRs as they go up!

[anbv29]

That sounds great, please tag me when the reader/writer PRs are up. I’d be glad to review the validation path and help add the regression tests once the relevant code is available. Thanks!