fix(tracing): instrument FastAPI ASGI layer for inbound traceparent propagation#4972
Open
STHITAPRAJNAS wants to merge 1 commit intogoogle:mainfrom
Open
fix(tracing): instrument FastAPI ASGI layer for inbound traceparent propagation#4972STHITAPRAJNAS wants to merge 1 commit intogoogle:mainfrom
STHITAPRAJNAS wants to merge 1 commit intogoogle:mainfrom
Conversation
…ropagation Fixes google#4767. When get_fast_api_app() is used in production behind an OTel-instrumented caller (e.g. a Next.js service using @opentelemetry/sdk-node), every inbound request carried a W3C traceparent header that ADK silently discarded. The TracerProvider and W3C propagator were already wired up correctly by _setup_telemetry(), but without an ASGI-level hook to extract the header, each request spawned a new trace root instead of continuing the caller's trace. The fix calls FastAPIInstrumentor.instrument_app(app) immediately after the FastAPI instance is created, but only when an OTel export pipeline is actually active (OTLP env vars or otel_to_cloud=True). The instrumentation is applied before the CORS and origin-check middleware are registered so that Starlette's reverse-registration order leaves the security wrappers outermost in the stack. The call is best-effort: if opentelemetry-instrumentation-fastapi is absent a debug-level message is emitted and the server starts normally. The package is added to the otel-gcp extras so users who install google-adk[otel-gcp] get end-to-end distributed tracing out of the box without any manual post-instrumentation step. opentelemetry-instrumentation-fastapi is also added to the test extras so the new unit tests can import it directly. google#4767
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4767.
Root cause
get_fast_api_app()calls_setup_telemetry()which correctly configures aTracerProviderand registers the W3CTraceContextTextMapPropagator. However, theFastAPIapp returned to callers never had the OTel ASGI middleware applied to it. Because thetraceparentheader is only extracted at the ASGI boundary, every inbound HTTP request started a new trace root instead of continuing the distributed trace initiated by the upstream caller.The existing workaround —
FastAPIInstrumentor().instrument_app(app)applied manually afterget_fast_api_app()returns — confirms the plumbing is otherwise correct.What this PR does
After the
FastAPIinstance is created insideAdkWebServer.get_fast_api_app(), callFastAPIInstrumentor.instrument_app(app)when an OTel export pipeline is active (OTEL_EXPORTER_OTLP_*env vars orotel_to_cloud=True):Key design decisions:
ImportErroris caught and logged atDEBUGlevel; the server continues normally without the instrumentation.CORSMiddlewareand_OriginCheckMiddlewareso that Starlette's reverse-registration order leaves the security wrappers outermost, and the OTel middleware extracts headers before the route handler runs.Dependency changes
opentelemetry-instrumentation-fastapiis added to:otel-gcpextras — users who installgoogle-adk[otel-gcp]now get automatic end-to-end tracing without any manual post-instrumentation step.testextras — required by the new unit tests.Tests
Four new tests in
tests/unittests/cli/test_fast_api.py:test_fastapi_instrumented_when_otlp_env_var_setinstrument_appcalled when an OTLP env var is presenttest_fastapi_instrumented_when_otel_to_cloud_enabledinstrument_appcalled whenotel_to_cloud=Truetest_fastapi_not_instrumented_without_otel_configinstrument_appNOT called when OTel is unconfiguredtest_missing_fastapi_instrumentor_does_not_prevent_startup/healthresponds 200 even if the package is absent