Repository navigation
Conversation
HsingTang
force-pushed
the
hsingtang/databricks-codex
branch
2 times, most recently
from
November 11, 2026 17:36
0adf796 to
7011b5e
Compare
HsingTang
force-pushed
the
hsingtang/databricks-codex
branch
5 times, most recently
from
November 11, 2026 23:30
cd36445 to
ef527b5
Compare
Add examples/codex/databricks: an application-owned gateway and a lifetime-managing Environment context manager that back the native executor with a Databricks sandbox, following the remote-execution pattern in docs/sdk/execution.md (the Databricks counterpart of offline_docker.py). The gateway relays the SDK's WSS executor connection to the sandbox exec-server over the databricks-sdk streaming command API; databricks_sandbox_environment starts the gateway and yields a configured Environment. Ship as an example (not a src/agents module or dependency extra) to match how the branch ships executor gateways. Include a runnable runner, a README, example requirements, and tests for the relay, bearer auth, teardown paths, the context manager, and the SSL_CERT_FILE trust helper. Validated end-to-end against a live sandbox (personal-staging): the exec-server stdio mode, bidirectional streaming over execute_command, and newline framing, plus exec_command / write_stdin / apply_patch, multi-turn with session resume and a real sandbox stop-start, large and single-line output, and the Environment approval / transform_arguments / transform_result hooks with an enforced NetworkPolicy. tests/codex pass (11) and mypy --strict is clean against the preview wheel on a glibc-2.39 host. This also fixes the runner's sandbox-image spec, which set a non-existent SandboxSpec.image instead of SandboxSpec.environment.image_uri. Co-authored-by: Isaac <no-reply@databricks.com>
HsingTang
force-pushed
the
hsingtang/databricks-codex
branch
from
November 12, 2026 05:02
ef527b5 to
1124c2e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Draft integration that backs the native executor (v1.0 /
feat/codex-harness) with a Databricks sandbox, so an agent driven by this SDK runsexec_command/write_stdin/apply_patchinside the sandbox.This PR is opened inside a personal fork (base
feat/codex-harness, headhsingtang/databricks-codex) for Databricks and OpenAI folks to review before anything is proposed upstream. It is not a contribution request toopenai/openai-agents-pythonyet.Integration style
It follows the "remote execution" pattern documented in
docs/sdk/execution.md: an application-owned gateway speaking the exec-server protocol, plus a lifetime-managingEnvironmentasync context manager (the "manage the executor lifetime" pattern). It is the Databricks counterpart ofexamples/offline_docker.py, and is shipped as an example - not asrc/agentsmodule or a dependency extra - to match how the branch ships executor gateways. It deliberately does not use theagents.classicBaseSandboxClientprovider model, which is a separate subsystem and not howexecution.mdconnects a self-hosted executor.Approach
The native
Environmentdials an authenticatedwss://executor and relays the exec-server protocol over it; a Databricks sandbox runs that exec-server in stdio mode, reached through thedatabricks-sdkstreaming command API (WorkspaceClient.sandbox.execute_command). Nothing in the SDK speaks stdio and nothing in the sandbox speaks WebSocket, so a small localhost gateway bridges them - it terminates the WebSocket, authenticates the bearer token at the HTTP handshake, opens the exec-server's stdio, and relays newline-framed messages both ways. Becauseexecute_commandreaches the sandbox outbound through the Databricks control plane, the gateway runs on loopback next to the application and the sandbox needs no inbound ingress. The agent loop runs in-process viaNativeProvider, which is what makes theEnvironmentapproval / transform /NetworkPolicyhooks available.Contents
All under
examples/codex/databricks/:gateway.py-SandboxExecServerGateway(newline-framed relay + self-signed loopback TLS helper).environment.py-databricks_sandbox_environment, the lifetime context manager that starts the gateway and yields a configuredEnvironment.runner.py- a runnable end-to-end example (optionalSANDBOX_CREATE=1provisioning).README.md,requirements.txt.tests/codex/- tests for the relay, bearer auth, teardown paths, the context manager, and theSSL_CERT_FILEtrust-bundle helper, with a fakeCommandProcess.Validated behaviors (Databricks staging, 2026-11-11)
Driven through this gateway over the real
execute_commandtransport. The live-agent tool calls were scripted deterministically with the SDK's own Responses fixture, so each behavior is exercised directly rather than left to the model's discretion:codexis a Databricks "ucode" wrapper over a real codex-cli 0.167.0 at/usr/local/libexec/codex-package/bin/codex. Itsexec-serversupports stdio via the--listenoption (values:ws://IP:PORTdefault,stdio,stdio://);stdiois not a subcommand, which is whyexec-server stdioerrors. The gateway runscodex exec-server --listen stdiowith_UCODE_DISPATCH_ACTIVE=1.agent.session(environment=...)turns drove tool calls through the gateway to the exec-server and back over thedatabricks-sdkexecute_commandstreamingCommandProcess(stdin,stdout.readline,text=False).exec_command, interactive-TTYwrite_stdin, andapply_patch(which added a file in the sandbox).stop_sandbox+start_sandboxcycle returned to service with the workspace preserved.approvalreturningFalseblocked a command's side-effect;transform_argumentsrewrote the command;transform_resultchanged what the model saw; and aNetworkPolicywas enforced by the executor's managed proxy (an allowed domain returned 200, a non-allowed domain was blocked).NativeProvider(base_url=".../ai-gateway/codex/v1", api_key=<PAT>)works with the Databricks PAT from~/.databrickscfg; no OpenAI key.model="gpt-5.5"is served bysystem.ai.gpt-5-5.Static checks:
ruff check/ruff format --checkpass;pytest tests/codex(11 passed) andmypy --strictare clean against the preview wheel on a glibc-2.39 host. Running mypy also caught and fixed a latent bug in the runner's optional provisioning path, which set a non-existentSandboxSpec.imageinstead ofSandboxSpec.environment.image_uri.Validated topologies
codex exec-server --listen stdiosubprocess.databricks-sdkexecute_commandstreamingCommandProcess, with tool side-effects confirmed back in the executor sandbox.Remaining
Internal dependency to resolve before any public release: the gateway reaches the real exec-server via
_UCODE_DISPATCH_ACTIVE=1. On the image,codexis a Databricks "ucode" self-healing wrapper that re-dispatches throughucode codex; that env var is the wrapper's internal recursion sentinel (gated by a Databricks feature flag), not a documented or supported interface, and could change. A supported invocation (e.g. the image exposing an exec-server entrypoint, or the wrapper passingexec-serverthrough) should be confirmed with the Databricks sandbox team before this is publicized or used in production.Open design question (not a blocker): auto-managed trust rewrites the process-global
SSL_CERT_FILE, so only one auto-managed environment runs per process; a per-connection trust lever onEnvironmentwould remove that constraint. The exec-server also offers a native--listen ws://mode with built-in--ws-auththat, with the sandboxport_forwardAPI, could simplify the gateway; the SDK's wss/TLS requirement still needs termination somewhere. The managed Agents API path (sandboxcodex exec-server --remotedialing OpenAI) remains the alternative and is covered by the cookbook.This pull request and its description were written by Isaac.