Skip to content

ATLAS-5408: Fix hook import classpath and logging issues in docker envs - #760

Merged
nixonrodrigues merged 1 commit into
apache:masterfrom
nixonrodrigues:ATLAS-5408
Sep 18, 2026
Merged

nixonrodrigues merged 1 commit into
apache:masterfrom
nixonrodrigues:ATLAS-5408

Conversation

@nixonrodrigues

Copy link
Copy Markdown
Collaborator

HBase: ship commons-configuration 1.x in atlas-hbase-plugin-impl because import-hbase.sh excludes it from HBase lib while Hadoop still needs it when HADOOP_HOME/classpath is absent.

Logging: create /var/log/atlas in atlas-base and mkdir -p ATLAS_LOG_DIR in import-hbase, import-hive, and import-kafka so logback can open import logs.

Kafka: stop putting broker config and log4j jars on the import classpath to avoid broker log4j.properties writing to /server.log; default KAFKA_CONF to $KAFKA_HOME/config.

What changes were proposed in this pull request?

  • Bundle commons-configuration 1.10 in atlas-hbase-plugin-impl (addons/hbase-bridge/pom.xml).
  • Document in import-hbase.sh why 1.x is in the plugin impl.
  • dev-support/atlas-docker/Dockerfile.atlas-base: create /var/log/atlas, root:hadoop, mode 775.
  • import-hbase.sh, import-hive.sh, import-kafka.sh: mkdir -p "${ATLAS_LOG_DIR}" after default is set.
  • Do not add KAFKA_CONF to import classpath; still source kafka-env.sh.
  • Omit log4j-*.jar from Kafka libs on import classpath.
  • Default KAFKA_CONF → $KAFKA_HOME/config.

How was this patch tested?

Test plan
Rebuild: mvn -pl addons/hbase-bridge,addons/kafka-bridge -am package -DskipTests -Drat.skip=true; rebuild docker images with new hook tarballs and atlas-base.
HBase: create a table, run import-hbase.sh with Atlas URL/credentials.
Hive: run import-hive.sh (smoke).
Kafka: run import-kafka.sh against running broker; confirm clean startup banner, no log4j root-file errors.
Confirm log files under /var/log/atlas/.

HBase: ship commons-configuration 1.x in atlas-hbase-plugin-impl because
import-hbase.sh excludes it from HBase lib while Hadoop still needs it when
HADOOP_HOME/classpath is absent.

Logging: create /var/log/atlas in atlas-base and mkdir -p ATLAS_LOG_DIR in
import-hbase, import-hive, and import-kafka so logback can open import logs.

Kafka: stop putting broker config and log4j jars on the import classpath to
avoid broker log4j.properties writing to /server.log; default KAFKA_CONF to
$KAFKA_HOME/config.

Co-authored-by: Cursor <cursoragent@cursor.com>
@nixonrodrigues
nixonrodrigues merged commit 14f4502 into apache:master Sep 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants