Reject WebSocket egress upgrades and add E2E HTTPS egress

[Adhita Selvaraj (swiftdiaries)]

Addresses #1017

Summary

• Return 403 for HTTP/1.1 WebSocket upgrades in the Envoy and Agent Gateway egress HTTP paths. The WebSocket fixtures remain as negative tests for ws:// and for wss:// when TLS is terminated by the MITM gateway.
• Update the Agent Gateway egress configuration to use a tunneled CONNECT front door and internal routing.
• Add end-to-end coverage for HTTPS on a nonstandard port, WebSocket denial, TLS trust under MITM, and egress access logs. The tests exercise both gVisor and micro-VM lanes and the Envoy and Agent Gateway configurations in CI.
• Extend the egress demo, origin fixtures, setup scripts, and failure diagnostics needed by these tests.
• Select kvm-clock for amd64 micro-VM guests so restored guests use Cloud Hypervisor's clock correction and accept newly issued TLS certificates.

The baseline egress path passes end-to-end TLS through without reading its HTTP headers. It can distinguish the destination of a wss:// connection but cannot distinguish its encrypted WebSocket handshake from ordinary HTTPS. The secure WebSocket denial test therefore runs only in the MITM lanes. The broader GA requirement to block WSS on passthrough paths remains unresolved.

Validation

• Local Kind runs previously passed targeted MITM trust and networking egress tests under gVisor, including HTTPS. Targeted micro-VM cases also passed in a Linux VM for an earlier version of this branch.
• The rebased commit passed targeted Go tests, the full Go race suite, lint, code generation, formatting, and diff checks locally. Local make verify stopped at Python license verification because the host's default Python is too old for the pinned grpcio dependency.
• Fork CI run 36176090367 passed run-tests, Envoy E2E, AgentGateway E2E, and the aggregate check on this exact rebased commit. It covers baseline and MITM gVisor and micro-VM lanes, including cleartext WebSocket denial, MITM secure WebSocket denial, delayed-restore TLS, and HTTPS on port 8443.

Fork CI validation: swiftdiaries#4

[Adhita Selvaraj (swiftdiaries)]

/assign haiyanmeng

[Adhita Selvaraj (swiftdiaries)]

The test setup part ended up being bigger than I would have liked. Please let me know if there's something that I can trim down or reuse. I'm re-checking for this myself as well.

[haiyanmeng]

Adhita Selvaraj (@swiftdiaries) , thanks for the change!

[haiyanmeng]

Fixes #1017

Let us say Addresses #1017 instead since it does not close all the gaps in #1017.
Fixes #1017 will auto-close the issue after the PR is merged.

[Adhita Selvaraj (swiftdiaries)]

ran the MITM fixture related tests; checked it with gvisor + mitm. I couldn't run the micro-vm tests on my machine.

$ E2E_EGRESS_MITM=1 hack/run-e2e-kind.sh \
  ./internal/e2e/suites/egressmitm -v -args --no-color
...
--- PASS: TestActorEgressMITMTrust (62.04s)
...

$ E2E_EGRESS_MITM=1 hack/run-e2e-kind.sh \
  ./internal/e2e/suites/networking -run '^TestActorEgress' -v -args --no-color
...
--- PASS: TestActorEgressHTTPSNonStandardPort (19.29s)
--- PASS: TestActorEgressWebSocket (10.55s)
--- PASS: TestActorEgressSecureWebSocket (19.98s)
...

[haiyanmeng]

pr-workflow / e2e-test (pull_request)Failing after 16m

Adhita Selvaraj (@swiftdiaries) , PTAL.

[Adhita Selvaraj (swiftdiaries)]

pr-workflow / e2e-test (pull_request)Failing after 16m

Adhita Selvaraj (Adhita Selvaraj (@swiftdiaries)) , PTAL.

Took a look at the failure, it looks like the guest clock doesn't restore the snapshot downtime? I'm not too familiar with the micro-vm test setup and I can't reproduce it :( It looks like a pre-existing failure mode for fresh certs? I'll take some time to look into this and meanwhile, create a separate issue to track, post it on slack to ask for broader help.

[Adhita Selvaraj (swiftdiaries)]

Benjamin Elder (@BenTheElder) could I please borrow your expertise a little here? Trying to debug the guestclock not correcting time when a snapshot is restored. I'm not too sure about the setup with Cloud Hypervisor for the micro-vm test path
This was the closest change I could see: #972

[Adhita Selvaraj (swiftdiaries)]

re: last commit - trying to see if i can set clocksource=kvm-clock in the micro-vm MITM CI codepath to see if it solves for the clock skew

[Adhita Selvaraj (swiftdiaries)]

Tested this on a Linux VM:

Setup:
Kind v1.37.0
Kata 4.0.0
Cloud Hypervisor 53.0

E2E_EGRESS_MITM=1 E2E_SANDBOX_CLASS=microvm \
  hack/run-e2e-kind.sh ./internal/e2e/suites/networking \
  -run '^(TestActorEgressHTTPSNonStandardPort|TestActorEgressWebSocket|TestActorEgressSecureWebSocket)$' \
  -v -args --no-color

--- PASS: TestActorEgressHTTPSNonStandardPort (19.96s)
--- PASS: TestActorEgressWebSocket (12.38s)
--- PASS: TestActorEgressSecureWebSocket (20.58s)
PASS
ok  github.com/agent-substrate/substrate/internal/e2e/suites/networking  53.103s

snapshot, clock, clocksource

golden takeGoldenSnapshotAt: 2026-09-16T05:33:59.957947998Z
host before:                 2026-09-16T05:39:00.283067832Z
guest UTC:                   2026-09-16T05:39:00.320563549Z
host after:                  2026-09-16T05:39:00.388914694Z
current_clocksource:         kvm-clock
available_clocksource:       kvm-clock tsc acpi_pm

Without knowing the clock source in CI, I can't confirm if the last commit is the necessary fix. If it's something else then forcing kvm-clock might break other working tests. So gating this to tests which uses TLS is the one way I could think of to limit blast radius for the change. If we find that this bug occurs in other tests, we can extend coverage and make it standard.

WDYT? haiyanmeng Bowei Du (@bowei)

[Adhita Selvaraj (swiftdiaries)]

rebased and fixed some additional gateway checks in CI
cc haiyanmeng

[haiyanmeng]

Adhita Selvaraj (@swiftdiaries) , https://github.com/agent-substrate/substrate/blob/main/docs/egress-traffic.md specifies the supported egress traffic for GA.

Currently, the Envoy dataplane supports WebSocket HTTP/1.1 Upgrade. To align with the specification, we can update the PR in the following way: keep the test fixture, modify the Envoy configuration to stop supporting WebSocket HTTP/1.1 Upgrade, and have a negative test verifying that the egress gateway denies the WebSocket HTTP/1.1 Upgrade traffic with 403.

WDYT?

cc Bowei Du (@bowei)

[Adhita Selvaraj (swiftdiaries)]

Sounds good! Let me update the PR and the re-run tests in CI locally

[haiyanmeng]

Adhita Selvaraj (@swiftdiaries) , this PR has 19 commits so far. Please squash them into 1-2 commits.

[haiyanmeng]

Eitan Yarmush (@EItanya) , can you take a look at this PR since it modifies the AgentGateway config?

[haiyanmeng]

Adhita Selvaraj (@swiftdiaries) , please update the PR title and description since this PR not only includes e2e tests, it also changes the egress gateway's behavior.

[Adhita Selvaraj (swiftdiaries)]

tested with CI on my fork's main branch with passing checks.
rebased again to fix merge conflict, checked CI again with fork's main
cc Bowei Du (@bowei) haiyanmeng