[lake] Fail fast when a lake plugin is shadowed by a jar in lib - #4203
Open
gezapeti wants to merge 5 commits into
Open
[lake] Fail fast when a lake plugin is shadowed by a jar in lib#4203gezapeti wants to merge 5 commits into
gezapeti wants to merge 5 commits into
Conversation
A LakeStoragePlugin can be provided either from the plugins directory, where it is loaded in an isolated class loader together with the dependencies bundled next to it, or from the main classpath. Providing it from both locations cannot work: classes under "org.apache.fluss." are loaded parent-first, so the copy on the main classpath shadows the one in the plugins directory while the dependencies bundled next to the plugin stay invisible. This is what happens when a fluss-lake-<format> jar is copied into <FLUSS_HOME>/lib in addition to plugins/<format>/. Paimon then fails deep inside CatalogContext with a bare "NoClassDefFoundError: org/apache/hadoop/hdfs/HdfsConfiguration", which gives no hint about the actual cause. Detect the duplicate up front and fail with a message naming both locations and the jar to remove. Resolution is unchanged when the plugin comes from only one location, so the classpath-only setups used by the Flink tiering job keep working. This addresses item (1) of the analysis in apache#1314. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019dqbppMMENgRMdJxwv5GuM
Add tests for the plugin layout users actually deploy, where each plugin lives in its own sub directory of <FLUSS_HOME>/plugins and the main classpath is <FLUSS_HOME>/lib. They cover several plugins installed side by side, isolation between them, and the shadowing that breaks a deployment: a class also present on the main classpath is loaded from there instead of from the plugin jar whenever its package is parent-first, leaving the dependencies bundled next to the plugin invisible. A companion test pins the healthy case, where the same layout resolves from the plugin jar. This addresses item (5) of the analysis in apache#1314. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019dqbppMMENgRMdJxwv5GuM
Two docs pages linked to /downloads#filesystem-jars, but the downloads
page had no such section. Add it, listing the filesystem plugin jars of
the latest release and which of them the binary release already ships,
so both links resolve.
Also spell out the choice between the two artifacts serving hdfs://:
fluss-fs-hdfs bundles Hadoop and is what the distribution ships, while
the thin fluss-fs-hadoop expects Hadoop via HADOOP_CLASSPATH. Exactly
one of them should be installed. The filesystem overview still credited
fluss-fs-hadoop for the bundled plugin, which has been fluss-fs-hdfs
since the artifacts were split.
Add a troubleshooting page explaining how plugins are loaded, why a jar
copied into <FLUSS_HOME>/lib shadows the copy under plugins/ and loses
the dependencies bundled next to it, and what each resulting error
means. Warn about it where users hit it: the filesystem overview, the
lakehouse deployment guide, and the downloads page. The lakehouse guide
also now says explicitly that its ${FLINK_HOME}/lib instructions are
about the Flink installation, not the Fluss one.
This addresses items (2) and (6) of the analysis in apache#1314.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019dqbppMMENgRMdJxwv5GuM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Linked issue: close #1314
A
fluss-lake-<format>jar placed in<FLUSS_HOME>/libas well asplugins/<format>/makes theCoordinatorServer fail with
NoClassDefFoundError: org/apache/hadoop/hdfs/HdfsConfigurationfromCatalogContext.<init>.org.apache.fluss.is parent-first, so thelib/copy shadows the plugin and is loaded without thedependencies bundled next to it. This rejects that setup up front instead of failing later.
Items (3) and (4) of the issue were already addressed upstream; this covers (1), (2), (5) and (6).
Brief change log
LakeStoragePluginSetUpresolves the plugins directory and the classpath separately. Finding thesame format in both now fails with a message naming both locations and the jar to remove.
passes a null
PluginManager, is unaffected.PluginDirectoryLayoutTestcovering the deployedplugins/layout.Filesystem JARssection on the downloads page.Detecting duplicates means the classpath SPI is now always scanned, where before it was skipped once
a plugin was found. A broken service entry there can therefore surface a
ServiceConfigurationError.Tests
LakeStorageTest— the duplicate is rejected and classpath-only resolution still works. Verifiedthe new case fails without the production change.
PluginDirectoryLayoutTest— multiple plugins loaded at once, isolation between them, mainclasspath shadowing, and resolution from the plugin jar when not shadowed.
./mvnw verify -pl fluss-commonpasses: 1798 unit and 186 integration tests. Spotless, Checkstyleand RAT pass.
API and Format
No public API or storage format change.
Documentation
maintenance/troubleshooting-plugins.md.Filesystem JARssection on the downloads page. Two pages already linked to/downloads#filesystem-jars, which did not exist. It also separates the bundledfluss-fs-hdfsfrom the thin
fluss-fs-hadoop, which expects Hadoop viaHADOOP_CLASSPATH.fluss-fs-hadoopfor the bundled HDFS plugin, which has beenfluss-fs-hdfssince [filesystems/hdfs] Publish Bundled HDFS Artifact #1522.Generative AI disclosure
Generated-by: Claude Opus 5 (Claude Code) following the guidelines (https://github.com/apache/fluss/blob/main/AGENTS.md)