Introducing Confident Monitor

AgentCore

Use Confident AI for LLM observability and evals for Amazon AgentCore

Overview

Amazon Bedrock AgentCore is AWS's managed runtime for deploying and scaling AI agents. Confident AI allows you to trace and evaluate agents running on AgentCore in just a few lines of code — each /invocations request shows up in the Observatory as one trace, with the model, tool, and agent spans from whatever framework you run inside it nested underneath, so you can see every turn your deployed agent handles and how it performed.

RuntimeRequirementsSetup
PythonPython 3.10+, bedrock-agentcore (tested with 1.22.0), OTel ASGI instrumentation (tested with 0.63b1)Call init() explicitly before creating the app
TypeScriptNot supported—

Auto-Instrument

  1. Install Dependencies

    Run the following command to install confident-trace with its agentcore extra, along with the AgentCore runtime and your model SDK:

    pip install 'confident-trace[agentcore]' bedrock-agentcore boto3

    The agentcore extra pulls in OpenTelemetry's ASGI middleware, which is what lets confident-trace wrap the AgentCore HTTP server and open a span for each incoming /invocations request. If you are using AgentCore with Strands, also install:

    pip install 'strands-agents[otel]'
  2. Setup Confident AI Key

    Get your Confident AI Project API key and set it as an environment variable. Boto3 uses your normal AWS credential chain.

    export CONFIDENT_API_KEY="<your-confident-api-key>"
    export BEDROCK_MODEL="<bedrock-model-id>"
    # plus AWS_REGION and AWS credentials for Bedrock
  3. Instrument AgentCore

    Call init() once at startup, before constructing BedrockAgentCoreApp. Every request that hits your entrypoint will then produce a trace with the Bedrock converse span underneath the server span.

    main.py
    import asyncio
    import os
    
    import boto3
    from bedrock_agentcore.runtime import BedrockAgentCoreApp
    from confident_trace import init, shutdown
    
    init(
        endpoint="https://otel.confident-ai.com/v1/traces",
        protocol="http/protobuf",
        api_key=os.environ["CONFIDENT_API_KEY"],
    )
    app = BedrockAgentCoreApp()
    client = boto3.client("bedrock-runtime")
    
    
    @app.entrypoint
    async def invoke(payload, context):
        # Offloading to a thread keeps the trace context intact for the sync Boto3 call.
        return await asyncio.to_thread(
            client.converse,
            modelId=os.environ["BEDROCK_MODEL"],
            messages=[{"role": "user", "content": [{"text": payload["prompt"]}]}],
        )
    
    
    if __name__ == "__main__":
        try:
            app.run()
        finally:
            shutdown()
  4. Run your agent

    Start the app locally and send it a request:

    python main.py
    curl -X POST http://localhost:8080/invocations \
      -H "Content-Type: application/json" \
      -d '{"prompt": "Explain OpenTelemetry in one sentence."}'

    Done ✅. Open the Observatory in your Confident AI project to inspect the trace. The same file runs unchanged when deployed to AgentCore.

What Gets Captured

Each AgentCore invocation is exported as a trace that connects the incoming request to the model, agent, and tool operations performed while handling it.

  • Invocation lifecycle — request timing, status, streaming duration, and the operations nested inside the entry point.
  • Conversation identity — the AgentCore session is used as the trace's thread ID, grouping invocations from the same session.
  • Distributed context — incoming W3C trace context is preserved so AgentCore work can remain connected to an upstream trace.
  • Model calls — model details, messages, tool requests and results, finish reasons, and token usage when provided by the enabled model integration.
  • Agent and tool calls — operations emitted by enabled framework integrations remain nested under the invocation.
  • Custom spans — custom application spans created while handling the request retain their place in the trace.

The AgentCore request boundary itself doesn't inspect or record request and response bodies. Content captured by nested integrations follows their respective content policies.

Set Trace Span Properties

Use a trace context to add properties you know before the call starts. It creates no extra span; the trace started by the @app.entrypoint handler inherits the tags, metadata, user ID, and customer ID.

main.py
from confident_trace import init, trace_context

init()

@app.entrypoint
async def invoke(payload, context):
    async with trace_context(
        tags=["support"],
        metadata={"release": "2026-09"},
        user_id=payload.get("user_id"),
        customer_id=payload.get("customer_id"),
    ):
        return await asyncio.to_thread(
            client.converse,
            modelId=os.environ["BEDROCK_MODEL"],
            messages=[{"role": "user", "content": [{"text": payload["prompt"]}]}],
        )

This request-driven example uses user_id and customer_id because the payload supplies only IDs.

See users and customers to set an optional display name alongside each ID, and trace context for every supported trace property and update behavior.

Instrumenting Multi-Turn

You do not need turn() when one AgentCore entry-point call is already one conversational turn—the integration creates that turn's trace automatically. Use turn() when you want to define the boundary yourself, such as grouping two sequential AgentCore calls into one turn. Reuse the same thread ID on later turns to group them into one conversation.

main.py
from confident_trace import init, turn

init()

@app.entrypoint
async def invoke(payload, context):
    async with turn("support-turn", thread_id=context.session_id):
        first = await call_agent(payload["prompt"])
        return await call_agent(f"Summarize this result: {first}")

See threads for thread I/O, turn IDs, and user IDs.

Disable AgentCore Instrumentation

Pass init() a list of integration identifiers to opt in to only those integrations. The identifier for AgentCore is "agentcore" in Python; omit it to disable this integration. An empty list disables all automatic instrumentation:

main.py
from confident_trace import init
init(instrumentations=())
# Use ("agentcore",) to opt in; omit "agentcore" to disable it.

This turns off Confident AI's automatic instrumentation; calls made after initialization are not instrumented by this integration.

Next Steps

Now that your deployed agent is traced, dive deeper into:

Need help instrumenting your application?Connect your model calls and agent workflows to Confident AITalk to an expert

Last updated on

Built byConfident AI