Agent Core

Overview

Amazon AgentCore is AWS's managed runtime for deploying and scaling AI agents. Confident AI allows you to trace and evaluate AgentCore agents — in just a few lines of code.

The integration works via OpenTelemetry: instrument_agentcore() registers a AgentCoreSpanInterceptor and a ContextAwareSpanProcessor on the global TracerProvider. The interceptor translates AWS Bedrock / Strands / Traceloop OTel spans into Confident AI spans, and the processor ships them to Confident AI in real time.

Tracing Quickstart

1. Install Dependencies

   Run the following command to install the required packages:

   pip install -U deepeval opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

   If you are using AgentCore with Strands, also install:

   pip install bedrock-agentcore strands-agents

2. Instrument AgentCore

   Call instrument_agentcore once at startup, before your agent runs. It attaches to the active OpenTelemetry TracerProvider (creating one if needed) and begins forwarding spans to Confident AI automatically.

   import os
   from bedrock_agentcore import BedrockAgentCoreApp
   from strands import Agent
   from deepeval.integrations.agentcore import instrument_agentcore
   
   instrument_agentcore()
   
   app = BedrockAgentCoreApp()
   agent = Agent(model="amazon.nova-lite-v1:0")
   
   @app.entrypoint
   def invoke(payload):
       user_message = payload.get("prompt", "Hello! How can I help you today?")
       result = agent(user_message)
       return {"result": result.message}
   
   if __name__ == "__main__":
       response = invoke({"prompt": "Explain OpenTelemetry in one sentence."})
       print(f"Agent Response: {response['result']}")

3. Run your agent

   Invoke your agent by executing the script:

   python main.py

   You can directly view the traces on Confident AI by clicking on the link in the output printed in the console.

Advanced Usage

Logging threads

Threads are used to group related traces together, and are useful for chat apps, agents, or any multi-turn interactions. You can learn more about threads here. Pass the thread_id to instrument_agentcore.

import os
from bedrock_agentcore import BedrockAgentCoreApp
from strands import Agent
from deepeval.integrations.agentcore import instrument_agentcore

instrument_agentcore(
    thread_id="thread_1",
    user_id="user_1"
)

app = BedrockAgentCoreApp()
agent = Agent(model="amazon.nova-lite-v1:0")

@app.entrypoint
def invoke(payload):
    user_message = payload.get("prompt", "Hello! How can I help you today?")
    result = agent(user_message)
    return {"result": result.message}

Trace attributes

Other trace-level attributes can be passed to instrument_agentcore. All parameters are optional and apply to every trace produced while the instrumentation is active.

import os
from bedrock_agentcore import BedrockAgentCoreApp
from strands import Agent
from deepeval.integrations.agentcore import instrument_agentcore

instrument_agentcore(
    name="Name of Trace",
    tags=["Tag 1", "Tag 2"],
    metadata={"Key": "Value"},
    user_id="user_1",
    thread_id="conversation-abc123",
    environment="production",
)

app = BedrockAgentCoreApp()
agent = Agent(model="amazon.nova-lite-v1:0")

@app.entrypoint
def invoke(payload):
    user_message = payload.get("prompt", "Hello! How can I help you today?")
    result = agent(user_message)
    return {"result": result.message}

Logging prompts

If you are managing prompts on Confident AI and wish to log them, use next_llm_span to associate a Prompt with the next LLM span before invoking your agent.

from bedrock_agentcore import BedrockAgentCoreApp
from strands import Agent
from deepeval.prompt import Prompt
from deepeval.tracing import next_llm_span
from deepeval.integrations.agentcore import instrument_agentcore

instrument_agentcore(environment="production")

app = BedrockAgentCoreApp()
agent = Agent(model="amazon.nova-lite-v1:0")

prompt = Prompt(alias="<prompt-alias>")
prompt.pull(version="00.00.01")

@app.entrypoint
def invoke(payload):
    user_message = payload.get("prompt", "")
    with next_llm_span(prompt=prompt):
        result = agent(user_message)
    return {"result": result.message}

Re-configuring at runtime

instrument_agentcore is idempotent — calling it again on the same TracerProvider updates the trace-level settings in place without stacking additional processors. This lets you reconfigure per-request fields (such as thread_id or user_id) by calling instrument_agentcore again before each invocation.

from deepeval.integrations.agentcore import instrument_agentcore

# First call: sets up the processors
instrument_agentcore(environment="production")

# Subsequent call: updates settings only, no new processors added
instrument_agentcore(environment="production", user_id="user_42", thread_id="conv-999")

Evals Usage

Online evals

You can run online evals on your AgentCore agent, which will run evaluations on all incoming traces on Confident AI's servers. This approach is recommended if your agent is in production.

1. Create metric collection

   Create a metric collection on Confident AI with the metrics you wish to use to evaluate your AgentCore agent.

2. Run evals

   Pass the metric_collection parameter to instrument_agentcore to run online evals at the trace level. For span-level evals, use update_current_span(metric_collection=...) inside your agent code.

   import os
   from bedrock_agentcore import BedrockAgentCoreApp
   from strands import Agent
   from deepeval.integrations.agentcore import instrument_agentcore
   
   instrument_agentcore(
       metric_collection="my-trace-collection",
       environment="production",
   )
   
   app = BedrockAgentCoreApp()
   agent = Agent(model="amazon.nova-lite-v1:0")
   
   @app.entrypoint
   def invoke(payload):
       user_message = payload.get("prompt", "Hello! How can I help you today?")
       result = agent(user_message)
       return {"result": result.message}

You can view evals on Confident AI by clicking on the link in the output printed in the console.