Skip to content

[Bug]: gRPC interceptor swallows exceptions and returns None instead of re-raising #895

Description

@Harshad01

Problem Description

The create_span wrapper in instana/instrumentation/grpcio.py catches exceptions
from gRPC stubs, records them on the trace span, and then falls off the end of
the function returning None. gRPC then tries to unpack the return value as a tuple:

response, call = None   # inside grpc._interceptor._with_call

which raises:

TypeError: cannot unpack non-iterable NoneType object

The real gRPC status code (e.g. PERMISSION_DENIED) is silently absorbed into the
span and never reaches the caller. This makes the error completely undiagnosable
from the application side.

This only happens inside an active Instana span (i.e. inside a traced API request).
Calling the same gRPC method without Instana active gives the correct RpcError.

Instana version: 3.15.0
pymilvus version: 2.6.13 (gRPC client)
Python: 3.11/3.12

Minimal, Complete, Verifiable, Example

# Reproduce: make a gRPC call via pymilvus inside a traced FastAPI request
# (Instana must be active with a live span)

from pymilvus import utility
utility.has_collection("my_collection", using="my_connection")

# Expected: grpc.RpcError with StatusCode.PERMISSION_DENIED
# Actual:   TypeError: cannot unpack non-iterable NoneType object

# Root cause in instana/instrumentation/grpcio.py::create_span:
# The wrapper catches the gRPC exception, records it on the span via
# span.record_exception(exc), but never re-raises — the function returns
# None implicitly.
# gRPC then does:  response, call = None  -> TypeError
#
# Fix: add `raise` after span.record_exception(exc) in create_span

Python Version

python 3.12

Python Modules

instana==3.15.0
grpcio==1.62.x
pymilvus==2.6.13
wrapt==1.x

Python Environment

Running inside a CP4D (Cloud Pak for Data) Kubernetes pod.
Instana is loaded via AUTOWRAPT_BOOTSTRAP=instana environment variable.
Python 3.11 / 3.12 (container image).

Key env vars relevant to this bug:
AUTOWRAPT_BOOTSTRAP=instana
INSTANA_SERVICE_NAME=datasift-api

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions