feat(api-core): add channel orchestration for OpenTelemetry

[chalmerlowe]

This pull request introduces channel orchestration helper functions in google.api_core._observability to support OpenTelemetry (OTel) client interceptors for synchronous gRPC channels (asynchronous gRPC channels are included for comparison).

Problem

Generated client libraries need a centralized, maintainable way to create and instrument gRPC channels with OpenTelemetry tracing when enabled via environment variables or client options.
Because synchronous gRPC (grpc) and asynchronous gRPC (grpc.aio) have fundamentally different channel creation lifecycles (sync channels can be intercepted post-creation via OpenTelemetry's custom applier, whereas async grpc.aio channels are immutable and require interceptors at construction time), handling these differences directly in every generated client or transport creates boilerplate and risk of drift.

Solution

This PR introduces channel creation orchestration helpers in google.api_core._observability:

1. create_channel_with_otel (Sync gRPC):
   • Calls channel_factory(*channel_args, **channel_kwargs) to construct the raw channel.
   • If OpenTelemetry capabilities are enabled and installed, wraps the raw channel using OpenTelemetry's client interceptor.
   • Supports positional *channel_args and keyword-only client_options, allowing clients to pass a lazy factory via functools.partial(create_channel_with_otel, Transport.create_channel, client_options=...) directly into Transport(channel=...).
2. create_async_channel_with_otel (Async gRPC - For Comparison):
   • Injects the OpenTelemetry async client interceptor (aio_client_interceptor) into kwargs['interceptors'] prior to invoking channel_factory(*channel_args, **channel_kwargs).
   • Preserves any existing user-supplied interceptors.
3. _get_otel_interceptor:
   • Internal helper centralizing tracer_provider extraction from ClientOptions and instantiating sync (client_interceptor) or async (aio_client_interceptor) interceptors without duplication.

Testing

• Added comprehensive unit test suite in tests/unit/test_observability.py:
  • Positional argument passthrough (*channel_args).
  • functools.partial binding and lazy execution for both sync and async.
  • Interceptor list injection (handling None, empty list, or pre-existing interceptors).
  • Validated against GOOGLE_SDK_EXPERIMENTAL_PYTHON_TRACING_ENABLED feature gating.
• Verified 100% test pass rate across Nox unit-3.10 and lint sessions.

Notes for Reviewers

• Supporting *channel_args alongside keyword-only client_options enables client.py templates to pass functools.partial as the channel parameter, preserving true lazy channel initialization in Transport.__init__ and eliminating the need to duplicate channel parameters (credentials, scopes, etc.) in Client.__init__.

[gemini-code-assist]

Code Review

This pull request refactors OpenTelemetry channel instrumentation in google/api_core/_observability.py by replacing apply_otel_capabilities_to_channel with dedicated helpers for creating synchronous and asynchronous channels with OTel capabilities (create_channel_with_otel and create_async_channel_with_otel). Unit tests are updated accordingly. The review feedback highlights an inconsistency in interceptor execution order between the sync and async implementations, suggesting that the async OTel interceptor should be prepended rather than appended to the interceptors list to maintain consistent tracing semantics across both environments.

[chalmerlowe]

There is plenty of room for deduplicating some of the inner workings of these tests (using fixtures, reusable functions, etc). Happy to revise these but would prefer to get some initial buy-in on the overall approach in the body of the code before investing in what might end up being premature optimization.

[daniel-sanche]

Can we also accept *channel_args? That would make this easier to pass into the transport init: #18188 (comment)

[chalmerlowe]

Adding *channel_args to create_channel_with_otel (and create_async_channel_with_otel) is a good improvement.

Transports pass self._host positionally to channel_init(self._host, ...). Supporting *channel_args allows us to pass functools.partial(_observability.create_channel_with_otel, Transport.create_channel, client_options=self._client_options) as the channel argument in the client.

This preserves true lazy channel initialization in the transport and eliminates the need for the client to duplicate extracting and passing credentials, scopes, quota_project_id, etc.

[daniel-sanche]

I left some comments in your other PR, but if it's possible to decouple the interceptor more from the channel, that could make thinks a lot easier for composition in the future.

I think the previous apply_otel_capabilities_to_channel would be better suited for this. If we go with option A, the client could do something like

grpc_interceptor = functools.partial(apply_otel_capabilities_to_channel, client_options=options)
interceptor_list = [grpc_interceptor, logging_interceptor]
Transport(interceptors=interceptor_list, ...)