Spring AI

Overview

Spring AI is the Spring framework for building AI applications in Java. Its observability is built on Micrometer Observations, and Spring Boot bridges those observations to OpenTelemetry. Point Spring Boot's OTLP exporter at Confident AI and every ChatClient call, advisor, model call, tool call, and vector store query shows up in the Observatory as a typed trace.

Auto-Instrument

1. Install Dependencies

   Add Spring Boot's OpenTelemetry support next to your Spring AI model starter. Versions are managed by the Spring Boot BOM.

   Spring Boot 4Spring Boot 3

   <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-opentelemetry</artifactId>
   </dependency>

2. Set Your API Keys

   Get your Confident AI Project API key and set it as an environment variable, along with the provider key your app uses:

   export CONFIDENT_API_KEY="<your-confident-api-key>"
   export OPENAI_API_KEY="<your-openai-key>"

3. Configure the Exporter

   Send traces to Confident AI over OTLP/HTTP with your API key in the x-confident-api-key header:

   Spring Boot 4Spring Boot 3

   spring:
     application:
       name: weather-assistant
     ai:
       tools:
         observations:
           include-content: true
   management:
     tracing:
       sampling:
         probability: 1.0
     opentelemetry:
       tracing:
         export:
           otlp:
             endpoint: https://otel.confident-ai.com/v1/traces
             headers:
               x-confident-api-key: ${CONFIDENT_API_KEY}
     otlp:
       metrics:
         export:
           enabled: false

   • management.tracing.sampling.probability defaults to 0.1, so Spring Boot drops 9 in 10 traces unless you raise it.
   • spring.ai.tools.observations.include-content adds tool arguments and results to tool spans.
   • spring.application.name becomes the trace's service.name.
   • On Spring Boot 4, the OpenTelemetry starter also exports metrics to localhost:4318 by default. Confident AI only accepts traces, so turn metrics export off unless you send metrics somewhere else.

4. Add Messages to Spans

   Add this configuration class to your application. Spring Boot applies every ObservationFilter bean automatically. The filter writes each model call's input and output messages, the ChatClient call's input and output, and retrieved documents as standard gen_ai.* span attributes:

   package com.example;
   
   import io.micrometer.common.KeyValue;
   import io.micrometer.observation.Observation;
   import io.micrometer.observation.ObservationFilter;
   import java.util.ArrayList;
   import java.util.LinkedHashMap;
   import java.util.List;
   import java.util.Map;
   import org.springframework.ai.chat.client.ChatClientResponse;
   import org.springframework.ai.chat.client.observation.ChatClientObservationContext;
   import org.springframework.ai.chat.messages.AssistantMessage;
   import org.springframework.ai.chat.messages.Message;
   import org.springframework.ai.chat.messages.ToolResponseMessage;
   import org.springframework.ai.chat.model.ChatResponse;
   import org.springframework.ai.chat.model.Generation;
   import org.springframework.ai.chat.observation.ChatModelObservationContext;
   import org.springframework.ai.document.Document;
   import org.springframework.ai.util.json.JsonParser;
   import org.springframework.ai.vectorstore.observation.VectorStoreObservationContext;
   import org.springframework.context.annotation.Bean;
   import org.springframework.context.annotation.Configuration;
   
   @Configuration
   public class ConfidentObservationConfig {
   
       @Bean
       ObservationFilter confidentContentFilter() {
           return context -> {
               if (context instanceof ChatModelObservationContext chat) {
                   put(context, "gen_ai.input.messages", messages(chat.getRequest().getInstructions()));
                   put(context, "gen_ai.output.messages", outputMessages(chat.getResponse()));
               }
               if (context instanceof ChatClientObservationContext client) {
                   put(context, "gen_ai.input.messages", messages(client.getRequest().prompt().getInstructions()));
                   ChatClientResponse response = client.getResponse();
                   put(context, "gen_ai.output.messages", outputMessages(response == null ? null : response.chatResponse()));
                   client.getRequest().context().forEach((key, value) -> {
                       if (key.startsWith("confident.trace.") && value != null) {
                           context.addHighCardinalityKeyValue(KeyValue.of(key, value.toString()));
                       }
                   });
               }
               if (context instanceof VectorStoreObservationContext store) {
                   put(context, "gen_ai.retrieval.documents", documents(store.getQueryResponse()));
               }
               return context;
           };
       }
   
       private static void put(Observation.Context context, String key, List<?> value) {
           if (value != null && !value.isEmpty()) {
               context.addHighCardinalityKeyValue(KeyValue.of(key, JsonParser.toJson(value)));
           }
       }
   
       private static List<Map<String, Object>> messages(List<Message> messages) {
           return messages.stream().map(ConfidentObservationConfig::message).toList();
       }
   
       private static List<Map<String, Object>> outputMessages(ChatResponse response) {
           if (response == null) {
               return null;
           }
           List<Map<String, Object>> output = new ArrayList<>();
           for (Generation generation : response.getResults()) {
               Map<String, Object> message = message(generation.getOutput());
               String finishReason = generation.getMetadata().getFinishReason();
               if (finishReason != null) {
                   message.put("finish_reason", finishReason.toLowerCase());
               }
               output.add(message);
           }
           return output;
       }
   
       private static Map<String, Object> message(Message message) {
           List<Map<String, Object>> parts = new ArrayList<>();
           if (message.getText() != null && !message.getText().isEmpty()) {
               parts.add(part("text", "content", message.getText()));
           }
           if (message instanceof AssistantMessage assistant) {
               for (AssistantMessage.ToolCall call : assistant.getToolCalls()) {
                   Map<String, Object> part = part("tool_call", "name", call.name());
                   part.put("id", call.id());
                   part.put("arguments", json(call.arguments()));
                   parts.add(part);
               }
           }
           if (message instanceof ToolResponseMessage tool) {
               for (ToolResponseMessage.ToolResponse response : tool.getResponses()) {
                   Map<String, Object> part = part("tool_call_response", "response", json(response.responseData()));
                   part.put("id", response.id());
                   parts.add(part);
               }
           }
           Map<String, Object> result = new LinkedHashMap<>();
           result.put("role", message.getMessageType().getValue());
           result.put("parts", parts);
           return result;
       }
   
       private static Map<String, Object> part(String type, String key, Object value) {
           Map<String, Object> part = new LinkedHashMap<>();
           part.put("type", type);
           part.put(key, value);
           return part;
       }
   
       private static Object json(String value) {
           try {
               return JsonParser.fromJson(value, Object.class);
           }
           catch (RuntimeException e) {
               return value;
           }
       }
   
       private static List<Map<String, Object>> documents(List<Document> documents) {
           if (documents == null) {
               return null;
           }
           return documents.stream().map(document -> {
               Map<String, Object> result = new LinkedHashMap<>();
               result.put("id", document.getId());
               result.put("content", document.getText());
               result.put("score", document.getScore());
               return result;
           }).toList();
       }
   }

5. Run Your Application

   Use ChatClient as you normally would. No other code changes are needed:

   @RestController
   class WeatherController {
   
       private final ChatClient chatClient;
   
       WeatherController(ChatClient.Builder builder) {
           this.chatClient = builder.build();
       }
   
       @GetMapping("/weather")
       String weather(@RequestParam String city) {
           return chatClient.prompt()
               .user("What's the weather in " + city + "?")
               .call()
               .content();
       }
   }

   Done ✅. Start your app, send a request, and open the Observatory in your Confident AI project to inspect the trace.

What Gets Captured

Spring AI's spans keep their hierarchy, so you can follow a request from the ChatClient call through its advisors, model calls, and tool calls.

Messages and retrieved documents come from the filter in step 4. Tool arguments and results need spring.ai.tools.observations.include-content. Everything else comes from Spring AI's own spans.

Set Trace Properties

Pass trace properties as ChatClient advisor parameters. The filter copies any parameter whose key starts with confident.trace. onto the trace:

String answer = chatClient.prompt()
    .user("What's the weather in Paris?")
    .advisors(advisor -> advisor
        .param("confident.trace.user_id", "user-42")
        .param("confident.trace.customer_id", "customer-7"))
    .call()
    .content();

See users and customers for what each ID unlocks in Confident AI.

Instrumenting Multi-Turn

Spring AI's conversation ID becomes the trace's thread ID, so every turn of a chat memory conversation lands in the same thread:

String answer = chatClient.prompt()
    .user(message)
    .advisors(MessageChatMemoryAdvisor.builder(chatMemory).build())
    .advisors(advisor -> advisor.param(ChatMemory.CONVERSATION_ID, "chat-42"))
    .call()
    .content();

Without chat memory, set ChatMemory.CONVERSATION_ID the same way, or pass confident.trace.thread_id as an advisor parameter.

Next Steps

Now that your Spring AI app is traced, dive deeper into:

Last updated on October 13, 2026