LangChain

Overview

LangChain is a framework for building LLM applications. Confident AI provides a callback handlers in both python and typescript SDKs to trace and evaluate your LangChain applications automatically.

The callback handler captures the following spans from your LangChain application:

• LLM spans — model name, provider, input messages, output content, tool calls made by the model, and token usage
• Tool spans — tool name, input parameters, and output; also aggregated at the trace level as tools_called
• Retriever spans — query input and retrieved document output
• Chain spans — inputs and outputs for top-level chains (used to set trace-level input/output)

Tracing Quickstart

1
pip install -U deepeval langchain langchain-openai

$
export CONFIDENT_API_KEY="<your-confident-api-key>"

1
from langchain_core.tools import tool
2
from langchain_openai import ChatOpenAI
3
from langchain_core.prompts import ChatPromptTemplate
4
from langchain.agents import create_tool_calling_agent, AgentExecutor
5
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
6
7
from deepeval.integrations.langchain import CallbackHandler
8
9
@tool
10
def multiply(a: int, b: int) -> int:
11
    """Returns the product of two numbers"""
12
    return a * b
13
14
llm = ChatOpenAI(model="gpt-4o-mini")
15
16
agent_prompt = ChatPromptTemplate.from_messages(
17
    [
18
        ("system", "You are a helpful assistant that can perform mathematical operations."),
19
        ("human", "{input}"),
20
        MessagesPlaceholder("agent_scratchpad"),
21
    ]
22
)
23
24
agent = create_tool_calling_agent(llm, [multiply], agent_prompt)
25
agent_executor = AgentExecutor(agent=agent, tools=[multiply], verbose=True)
26
27
result = agent_executor.invoke(
28
    {"input": "What is 8 multiplied by 6?"},
29
    config={"callbacks": [CallbackHandler()]},
30
)

1
python main.py

Advanced Features

Set trace attributes

Confident AI’s LLM tracing advanced features provide teams with the ability to set certain attributes for each trace when invoking your LangChain application.

For example, thread_id and user_id are used to group related traces together, and are useful for chat apps, agents, or any multi-turn interactions. You can learn more about threads here.

You can set these attributes in the CallbackHandler when invoking your LangChain application.

1
result = agent_executor.invoke(
2
    {"input": "What is 8 multiplied by 6?"},
3
    config={
4
        "callbacks": [CallbackHandler(thread_id="123")]
5
    },
6
)

Logging prompts

If you are managing prompts on Confident AI and wish to log them, pass your Prompt object to the language model instance’s metadata parameter.

1
from langchain_openai import ChatOpenAI
2
from deepeval.prompt import Prompt
3
4
prompt = Prompt(alias="<prompt-alias>")
5
prompt.pull(version="00.00.01")
6
7
llm = ChatOpenAI(
8
    model="gpt-4o-mini",
9
    metadata={"prompt": prompt}
10
)

Evals Usage

Online evals

If your LangChain application is in production, and you still want to run evaluations on your traces, use online evals. It lets you run evaluations on all incoming traces on Confident AI’s server.

1
from langchain_core.tools import tool
2
from langchain_openai import ChatOpenAI
3
from langchain_core.prompts import ChatPromptTemplate
4
from langchain.agents import create_tool_calling_agent, AgentExecutor
5
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
6
7
from deepeval.integrations.langchain import CallbackHandler
8
9
@tool
10
def multiply(a: int, b: int) -> int:
11
    """Returns the product of two numbers"""
12
    return a * b
13
14
llm = ChatOpenAI(model="gpt-4o-mini")
15
agent_prompt = ChatPromptTemplate.from_messages([("system", "You are a helpful assistant that can perform mathematical operations."), ("human", "{input}"), MessagesPlaceholder("agent_scratchpad")])
16
agent = create_tool_calling_agent(llm, [multiply], agent_prompt)
17
agent_executor = AgentExecutor(agent=agent, tools=[multiply], verbose=True)
18
19
result = agent_executor.invoke(
20
    {"input": "What is 8 multiplied by 6?"},
21
    config={
22
        "callbacks": [
23
          CallbackHandler(metric_collection="<metric_collection_name>")
24
        ]
25
    },
26
)

View on Confident AI

You can view the evals on Confident AI by clicking on the link in the output printed in the console.