From 823392161ce3b1de0ce705be6ad306030c1c77b9 Mon Sep 17 00:00:00 2001 From: Ranjithkumar Ragavan <43761047+RanjithRagavan@users.noreply.github.com> Date: Sun, 30 Aug 2026 01:26:31 -0700 Subject: [PATCH 1/2] Android: compile shared platform-neutral Java sources in place (step 1 of desktop JVM support) Move the platform-neutral Java/Kotlin API sources (Module, Tensor, EValue, DType, LlmModule, AsrModule, training, ...) from extension/android/executorch_android/src/main/java to a new shared source directory extension/java/src/main/java, preserving git history via renames. The Android AAR stays self-contained and behavior-identical: it compiles the shared sources directly via java.srcDirs, so there is no new published artifact, no new transitive dependency, no manifest change, and bytecode remains at Java 11. Verified locally by building release AARs before and after the move: the only classes.jar delta is one new internal class, org.pytorch.executorch.Log. Logging: Tensor.kt previously imported android.util.Log directly. Shared sources now reference an org.pytorch.executorch.Log facade by simple name; each platform artifact compiles exactly one implementation. The Android implementation (new Log.kt in the Android module) delegates to android.util.Log, so logcat tags, priorities, and messages are unchanged. The shared directory deliberately contains no Log definition, avoiding duplicate-class collisions. Buck targets in extension/android/BUCK keep their historical names, ownership, and attributes; their srcs now reference the shared location. This is step 1 of the plan discussed in #20761: land the behavior- preserving shared-source extraction first, then add the desktop JVM jar and native classifier publication on top. --- docs/source/using-executorch-android.md | 2 +- extension/android/BUCK | 39 ++++++++++++------- .../android/executorch_android/build.gradle | 5 ++- .../main/java/org/pytorch/executorch/Log.kt | 39 +++++++++++++++++++ extension/java/README.md | 34 ++++++++++++++++ .../pytorch/executorch/BackendOptionsMap.kt | 0 .../main/java/org/pytorch/executorch/DType.kt | 0 .../java/org/pytorch/executorch/EValue.kt | 0 .../pytorch/executorch/ExecuTorchRuntime.kt | 0 .../executorch/ExecutorchRuntimeException.kt | 0 .../org/pytorch/executorch/MethodMetadata.kt | 0 .../java/org/pytorch/executorch/Module.kt | 0 .../java/org/pytorch/executorch/Tensor.kt | 1 - .../executorch/annotations/Experimental.kt | 0 .../executorch/extension/asr/AsrCallback.kt | 0 .../executorch/extension/asr/AsrModule.kt | 0 .../extension/asr/AsrTranscribeConfig.kt | 0 .../executorch/extension/llm/LlmCallback.kt | 0 .../extension/llm/LlmGenerationConfig.kt | 0 .../executorch/extension/llm/LlmModule.kt | 0 .../extension/llm/LlmModuleConfig.kt | 0 .../org/pytorch/executorch/training/SGD.kt | 0 .../executorch/training/TrainingModule.kt | 0 23 files changed, 102 insertions(+), 18 deletions(-) create mode 100644 extension/android/executorch_android/src/main/java/org/pytorch/executorch/Log.kt create mode 100644 extension/java/README.md rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/DType.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/EValue.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/MethodMetadata.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/Module.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/Tensor.kt (99%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/annotations/Experimental.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/asr/AsrCallback.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/asr/AsrModule.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/asr/AsrTranscribeConfig.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/training/SGD.kt (100%) rename extension/{android/executorch_android => java}/src/main/java/org/pytorch/executorch/training/TrainingModule.kt (100%) diff --git a/docs/source/using-executorch-android.md b/docs/source/using-executorch-android.md index 0c034d99924..4668413916d 100644 --- a/docs/source/using-executorch-android.md +++ b/docs/source/using-executorch-android.md @@ -20,7 +20,7 @@ All ExecuTorch Android libraries are packaged into an Android library (AAR), exe The AAR artifact contains the Java library for users to integrate with their Java/Kotlin application code, as well as the corresponding JNI library (.so file), which is loaded by the Java code during initialization. -- [Java library](https://github.com/pytorch/executorch/tree/main/extension/android/executorch_android/src/main/java/org/pytorch/executorch) +- [Java library](https://github.com/pytorch/executorch/tree/main/extension/java/src/main/java/org/pytorch/executorch) - [Java API Reference (Javadoc)](https://pytorch.org/executorch/main/javadoc/index.html) - JNI contains the JNI binding for the corresponding Java code, and ExecuTorch native library, including - Core ExecuTorch runtime libraries diff --git a/extension/android/BUCK b/extension/android/BUCK index d1672a2a45b..10a927a6a2d 100644 --- a/extension/android/BUCK +++ b/extension/android/BUCK @@ -3,20 +3,29 @@ load("@fbsource//tools/build_defs/android:fb_android_library.bzl", "fb_android_l oncall("executorch") +# NOTE: the platform-neutral sources live in the shared source directory +# extension/java/src/main/java and are referenced here in place, so the +# Android targets keep their historical names, ownership, and attributes +# (required_for_source_only_abi, pure_kotlin, deps) while compiling the +# exact same classes as before the move. Log.kt is the Android-backed +# implementation of the org.pytorch.executorch.Log facade that the shared +# sources reference. + non_fbcode_target(_kind = fb_android_library, name = "executorch", warnings_as_errors = False, required_for_source_only_abi = True, srcs = [ - "executorch_android/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt", - "executorch_android/src/main/java/org/pytorch/executorch/DType.kt", - "executorch_android/src/main/java/org/pytorch/executorch/EValue.kt", - "executorch_android/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt", - "executorch_android/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt", - "executorch_android/src/main/java/org/pytorch/executorch/MethodMetadata.kt", - "executorch_android/src/main/java/org/pytorch/executorch/Module.kt", - "executorch_android/src/main/java/org/pytorch/executorch/Tensor.kt", - "executorch_android/src/main/java/org/pytorch/executorch/annotations/Experimental.kt", + "../java/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt", + "../java/src/main/java/org/pytorch/executorch/DType.kt", + "../java/src/main/java/org/pytorch/executorch/EValue.kt", + "../java/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt", + "../java/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt", + "../java/src/main/java/org/pytorch/executorch/MethodMetadata.kt", + "../java/src/main/java/org/pytorch/executorch/Module.kt", + "../java/src/main/java/org/pytorch/executorch/Tensor.kt", + "../java/src/main/java/org/pytorch/executorch/annotations/Experimental.kt", + "executorch_android/src/main/java/org/pytorch/executorch/Log.kt", ], autoglob = False, language = "KOTLIN", @@ -34,8 +43,8 @@ non_fbcode_target(_kind = fb_android_library, name = "executorch_training", warnings_as_errors = False, srcs = [ - "executorch_android/src/main/java/org/pytorch/executorch/training/SGD.kt", - "executorch_android/src/main/java/org/pytorch/executorch/training/TrainingModule.kt", + "../java/src/main/java/org/pytorch/executorch/training/SGD.kt", + "../java/src/main/java/org/pytorch/executorch/training/TrainingModule.kt", ], autoglob = False, language = "KOTLIN", @@ -50,10 +59,10 @@ non_fbcode_target(_kind = fb_android_library, name = "executorch_llama", warnings_as_errors = False, srcs = [ - "executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt", - "executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt", - "executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt", - "executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt", + "../java/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt", + "../java/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt", + "../java/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt", + "../java/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt", ], autoglob = False, language = "KOTLIN", diff --git a/extension/android/executorch_android/build.gradle b/extension/android/executorch_android/build.gradle index 2dbe0e1fb5f..9ec25315907 100644 --- a/extension/android/executorch_android/build.gradle +++ b/extension/android/executorch_android/build.gradle @@ -16,7 +16,7 @@ plugins { spotless { kotlin { - target '**/*.kt' + target '**/*.kt', '../../java/src/main/java/**/*.kt' ktfmt() } } @@ -44,6 +44,9 @@ android { sourceSets { main { jniLibs.srcDirs = ['../../../cmake-out-android-so/'] + // Platform-neutral API sources shared with the desktop JVM artifact. + // They are compiled directly into this AAR, keeping it self-contained. + java.srcDirs += ['../../java/src/main/java'] } androidTest { resources.srcDirs += ['src/androidTest/resources'] diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/Log.kt b/extension/android/executorch_android/src/main/java/org/pytorch/executorch/Log.kt new file mode 100644 index 00000000000..10110193d28 --- /dev/null +++ b/extension/android/executorch_android/src/main/java/org/pytorch/executorch/Log.kt @@ -0,0 +1,39 @@ +/* + * Copyright (c) Meta Platforms, Inc. and affiliates. + * All rights reserved. + * + * This source code is licensed under the BSD-style license found in the + * LICENSE file in the root directory of this source tree. + */ + +package org.pytorch.executorch + +/** + * Platform logging facade used by the shared sources under extension/java. + * + *

The shared sources reference [Log] by simple name (same package, no import), and each platform + * artifact compiles its own implementation: + *

+ * + * The shared source set itself contains no [Log] definition, so there is exactly one implementation + * per artifact and no duplicate-class collision. + */ +internal object Log { + @JvmStatic fun v(tag: String, msg: String): Int = android.util.Log.v(tag, msg) + + @JvmStatic fun d(tag: String, msg: String): Int = android.util.Log.d(tag, msg) + + @JvmStatic fun i(tag: String, msg: String): Int = android.util.Log.i(tag, msg) + + @JvmStatic fun w(tag: String, msg: String): Int = android.util.Log.w(tag, msg) + + @JvmStatic fun w(tag: String, msg: String, tr: Throwable): Int = android.util.Log.w(tag, msg, tr) + + @JvmStatic fun e(tag: String, msg: String): Int = android.util.Log.e(tag, msg) + + @JvmStatic fun e(tag: String, msg: String, tr: Throwable): Int = android.util.Log.e(tag, msg, tr) +} diff --git a/extension/java/README.md b/extension/java/README.md new file mode 100644 index 00000000000..030d16e6e6f --- /dev/null +++ b/extension/java/README.md @@ -0,0 +1,34 @@ +# ExecuTorch shared Java sources + +This directory holds the **platform-neutral** Java/Kotlin API sources +(`Module`, `Tensor`, `EValue`, `DType`, `LlmModule`, `AsrModule`, training, +…) that are shared by every Java-platform artifact. + +## How the sources are consumed + +These sources are **not** published as their own artifact. Each platform +build compiles them directly into its own, self-contained artifact by +referencing this directory as an additional source root: + +- **Android** (`extension/android/executorch_android`) adds + `../../java/src/main/java` via `java.srcDirs`, so the shipping AAR embeds + the same classes it always has — no new transitive dependency, unchanged + bytecode target (Java 11), unchanged manifest. +- **Desktop JVM** (`extension/jvm`, added separately) compiles the same + sources into its own jar. + +## Contract for sources in this directory + +1. **No platform APIs.** Sources here must not import `android.*`, + `java.awt.*`, or any other platform-specific API. The only permitted + dependencies are the JDK, fbjni, and soloader. +2. **Logging goes through the `org.pytorch.executorch.Log` facade.** The + facade is referenced by simple name (same package, no import) and is + *not* defined in this directory. Each platform artifact compiles exactly + one implementation (Android: backed by `android.util.Log`; desktop: + console-backed), so there is never a duplicate-class collision and each + platform keeps its native logging behavior. +3. **Native library loading stays per-platform.** Shared sources load the + native library through soloader's `NativeLoader`, whose delegate is + configured by the platform runtime, not by a cross-platform mutable + global. diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt b/extension/java/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt rename to extension/java/src/main/java/org/pytorch/executorch/BackendOptionsMap.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/DType.kt b/extension/java/src/main/java/org/pytorch/executorch/DType.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/DType.kt rename to extension/java/src/main/java/org/pytorch/executorch/DType.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/EValue.kt b/extension/java/src/main/java/org/pytorch/executorch/EValue.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/EValue.kt rename to extension/java/src/main/java/org/pytorch/executorch/EValue.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt b/extension/java/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt rename to extension/java/src/main/java/org/pytorch/executorch/ExecuTorchRuntime.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt b/extension/java/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt rename to extension/java/src/main/java/org/pytorch/executorch/ExecutorchRuntimeException.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/MethodMetadata.kt b/extension/java/src/main/java/org/pytorch/executorch/MethodMetadata.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/MethodMetadata.kt rename to extension/java/src/main/java/org/pytorch/executorch/MethodMetadata.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/Module.kt b/extension/java/src/main/java/org/pytorch/executorch/Module.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/Module.kt rename to extension/java/src/main/java/org/pytorch/executorch/Module.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/Tensor.kt b/extension/java/src/main/java/org/pytorch/executorch/Tensor.kt similarity index 99% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/Tensor.kt rename to extension/java/src/main/java/org/pytorch/executorch/Tensor.kt index f2f3ebea214..433667ccae0 100644 --- a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/Tensor.kt +++ b/extension/java/src/main/java/org/pytorch/executorch/Tensor.kt @@ -8,7 +8,6 @@ package org.pytorch.executorch -import android.util.Log import com.facebook.jni.HybridData import com.facebook.jni.annotations.DoNotStrip import java.nio.Buffer diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/annotations/Experimental.kt b/extension/java/src/main/java/org/pytorch/executorch/annotations/Experimental.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/annotations/Experimental.kt rename to extension/java/src/main/java/org/pytorch/executorch/annotations/Experimental.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/asr/AsrCallback.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/asr/AsrCallback.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/asr/AsrCallback.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/asr/AsrCallback.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/asr/AsrModule.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/asr/AsrModule.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/asr/AsrModule.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/asr/AsrModule.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/asr/AsrTranscribeConfig.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/asr/AsrTranscribeConfig.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/asr/AsrTranscribeConfig.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/asr/AsrTranscribeConfig.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmCallback.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmGenerationConfig.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmModule.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt b/extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt rename to extension/java/src/main/java/org/pytorch/executorch/extension/llm/LlmModuleConfig.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/training/SGD.kt b/extension/java/src/main/java/org/pytorch/executorch/training/SGD.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/training/SGD.kt rename to extension/java/src/main/java/org/pytorch/executorch/training/SGD.kt diff --git a/extension/android/executorch_android/src/main/java/org/pytorch/executorch/training/TrainingModule.kt b/extension/java/src/main/java/org/pytorch/executorch/training/TrainingModule.kt similarity index 100% rename from extension/android/executorch_android/src/main/java/org/pytorch/executorch/training/TrainingModule.kt rename to extension/java/src/main/java/org/pytorch/executorch/training/TrainingModule.kt From ecea77c33d22f95276122831dcfabfb662b001df Mon Sep 17 00:00:00 2001 From: Ranjithkumar Ragavan <43761047+RanjithRagavan@users.noreply.github.com> Date: Thu, 3 Sep 2026 22:20:57 -0700 Subject: [PATCH 2/2] docs: note dual-build maintenance rule for shared sources Address review feedback on #22338: Gradle globs the shared source directory while BUCK lists files explicitly, so a new shared source could be picked up by the OSS AAR but silently omitted from the Buck build. Add a MAINTENANCE note to the BUCK header and a pointer section to extension/java/README.md requiring srcs updates in lockstep. --- extension/android/BUCK | 6 ++++++ extension/java/README.md | 8 ++++++++ 2 files changed, 14 insertions(+) diff --git a/extension/android/BUCK b/extension/android/BUCK index 10a927a6a2d..c723d604bf3 100644 --- a/extension/android/BUCK +++ b/extension/android/BUCK @@ -10,6 +10,12 @@ oncall("executorch") # exact same classes as before the move. Log.kt is the Android-backed # implementation of the org.pytorch.executorch.Log facade that the shared # sources reference. +# +# MAINTENANCE: Gradle globs the shared directory (java.srcDirs) while the +# srcs lists below are explicit per-file. When adding a shared source under +# extension/java/src/main/java, add it to the matching target here in the +# same change — otherwise the OSS AAR picks it up while this build silently +# omits it. non_fbcode_target(_kind = fb_android_library, name = "executorch", diff --git a/extension/java/README.md b/extension/java/README.md index 030d16e6e6f..25b030b76ad 100644 --- a/extension/java/README.md +++ b/extension/java/README.md @@ -32,3 +32,11 @@ referencing this directory as an additional source root: native library through soloader's `NativeLoader`, whose delegate is configured by the platform runtime, not by a cross-platform mutable global. + +## Adding a source here + +Gradle (`extension/android/executorch_android/build.gradle`) globs this whole +directory, but `extension/android/BUCK` lists each shared file explicitly. +**Update the matching BUCK target's `srcs` in the same change** whenever you +add or remove a file here — otherwise the OSS AAR picks it up while the Buck +build silently omits it.