fix(python-uv): take into account dependencies on other workspace members - #887
fix(python-uv): take into account dependencies on other workspace members#887noritada wants to merge 6 commits into
Conversation
…bers This commit fixes an issue where an application created as a member of a uv workspace would fail to build if they depended on other workspace members. The following shows the problematic workspace structure and the error message. ``` workspace root ├── lib │ ├── pyproject.toml │ └── src ├── pyproject.toml ├── sam-app │ ├── __init__.py │ ├── app.py │ ├── pyproject.toml │ ├── samconfig.toml │ └── template.yaml └── uv.lock ``` ``` Build Failed Error: PythonUvBuilder:ResolveDependencies - UV package build failed: Failed to build from pyproject.toml: Lock file operation failed: Failed to install dependencies using uv: UV pip install failed: Using CPython 3.13.0 interpreter at: /path/to/workspace/.venv/bin/python3 error: Distribution not found at: file:///path/to/workspace/sam-app/lib ``` Even though `lib` and `sam-app` are in the same directory level in the workspace, the workflow attempts to install `lib` under `sam-app`. In the workflow, `uv export` outputs a list of dependency packages, which is then passed to `uv pip install` for installation. When doing so, [`uv export` outputs relative paths from the workspace root][1]. Therefore, `uv pip install` must be run from the workspace root, not from the application directory. Additionally, dependencies on other packages within the workspace are exported as editable installations (e.g., `-e ./lib`) by default. When this is passed to `uv pip install`, only the `.pth` (path configuration) file for the package will be installed without the package body. To prevent this, the `--no-editable` option needs to be used. [1]: astral-sh/uv#20238
|
Thanks for the fix and the unusually clear write-up, @noritada — and apologies for the long silence on this one. I reproduced this end to end and your diagnosis is correct. Details below, including one thing I'd like changed before merge. Yes, this fixes #892I built the exact workspace from your repro (bare root +
So #892 is closable when this merges. I also confirmed that both halves of the change are load-bearing, which wasn't obvious to me from the diff alone. With only the And your claim about export paths (astral-sh/uv#20238) checks out: exporting from the member dir emits Your approach also looks like the only reliable one here. Because uv writes One change requested: fall back when
|
There was a problem hiding this comment.
Code Review Results
Reviewed: 37cd1d1..b813ed8
Files: 9
Comments: 2
Comments on lines outside the diff:
[aws_lambda_builders/workflows/python_uv/packager.py:372] [BUG] uv export still runs with cwd=project_dir while uv pip install now runs with cwd=workspace_dir, so any relative path shared by the two commands no longer resolves to the same location.
temp_requirements is os.path.join(scratch_dir, "lock_requirements.txt"), and scratch_dir is never absolutized on the way in — PythonUvBuildAction forwards whatever the workflow was given. With a relative scratch dir, --output-file is written under project_dir but -r is looked up under the workspace root, so the install fails with a missing requirements file for exactly the workspace layout this PR fixes. install_requirements already guards --target against this (and its comment notes the incremental-build dependencies dir does arrive relative in practice), so the requirements path deserves the same treatment:
args.extend(["-r", os.path.abspath(requirements_path)])config.cache_dir is derived from scratch_dir too and is passed to uv as --cache-dir from the new cwd, so it has the same exposure.
| python_version=python_version, | ||
| platform="linux", | ||
| cwd=project_dir, | ||
| cwd=workspace_dir, |
There was a problem hiding this comment.
[BUG] Splitting the two UV invocations across different working directories breaks the relative path they share.
temp_requirements is built as os.path.join(scratch_dir, "lock_requirements.txt") and is never absolutized. It is written by uv export --output-file (which still runs with cwd=project_dir) and then read by uv pip install -r (which now runs with cwd=workspace_dir). Before this change both commands shared the same cwd, so a relative scratch_dir resolved consistently. Now, when workspace_dir != project_dir, export writes to project_dir/<scratch_dir>/lock_requirements.txt while install looks for workspace_dir/<scratch_dir>/lock_requirements.txt and fails. config.cache_dir (set to os.path.join(scratch_dir, "uv-cache") in _ensure_cache_dir) has the same problem and would silently create a second cache directory.
scratch_dir is not normalized anywhere in the call chain — builder.py only does os.makedirs(scratch_dir) on it. This codebase already treats these paths as possibly relative: install_requirements deliberately wraps the target in os.path.abspath() (line 142, with a comment about UV's cwd, covered by test_install_requirements_resolves_relative_target_to_absolute), and java_gradle/actions.py:55 applies os.path.abspath to scratch_dir for the same reason.
Anchor the shared path so it is independent of which cwd UV runs in:
temp_requirements = os.path.abspath(os.path.join(scratch_dir, "lock_requirements.txt"))Absolutizing requirements_path inside install_requirements (mirroring the existing --target handling) would also protect the _build_from_requirements path, which passes a caller-supplied manifest path.
Note: this was raised in the previous review round at packager.py:372 and does not appear to have been addressed or explicitly dismissed, so re-raising.
This PR fixes an issue where an application created as a member of a
uv workspace would fail to build if they depended on other workspace
members.
The following shows the problematic workspace structure and the error
message.
Even though
libandsam-appare in the same directory level in theworkspace, the workflow attempts to install
libundersam-app.In the workflow,
uv exportoutputs a list of dependency packages,which is then passed to
uv pip installfor installation. When doingso,
uv exportoutputs relative paths from the workspace root.Therefore,
uv pip installmust be run from the workspace root, notfrom the application directory.
Additionally, dependencies on other packages within the workspace are
exported as editable installations (e.g.,
-e ./lib) by default.When this is passed to
uv pip install, only the.pth(pathconfiguration) file for the package will be installed without the
package body. To prevent this, the
--no-editableoption needs to beused.
Closes #892.
Commands to reproduce the build failure
uv init --bare uv init --lib lib sam init --name sam-app --runtime python3.14 --architecture arm64 \ --dependency-manager pip --package-type Zip \ --app-template hello-world cd sam-app uv init uv add lib@../lib # remove requirements.txt and edit the app # edit template.yaml to configure `BuildMethod: python-uv` and `CodeUri: .` sam build --beta-features