Handler Mode

Handler Mode lets you connect an AI app that has no HTTP endpoint to Confident AI. Instead of pointing an AI Connection at a URL, you implement a small handler function that the Confident Agent runs inside your own environment. Confident AI sends inputs down the agent's existing outbound tunnel, your function produces the output locally, and the agent relays it back.

When to Use Handler Mode

Reach for Handler Mode when you want to evaluate or red team an app that isn't exposed as a callable service:

• The pipeline is an internal script, notebook, or library with no API in front of it.
• Standing up and hosting an endpoint (even internally) is more work than you want for an evaluation.
• Security or compliance requires that inputs are processed in place, with nothing inbound and no code hosted by Confident AI.

If your app already has an HTTP endpoint, use a standard AI Connection — or, for a private endpoint behind a firewall, the Confident Agent in its default forwarding mode.

How It Works

The agent connects outbound over WebSocket Secure (WSS) exactly as in forwarding mode. The only difference is what happens when a request arrives: instead of forwarding it to an internal URL, the agent calls your handler function and relays whatever it returns.

sequenceDiagram
    participant P as Your Pipeline (imported)
    participant H as handler()
    participant A as Confident Agent
    participant C as Confident AI

    A->>C: Connect outbound (WSS/443)
    Note over A,C: WebSocket tunnel established

    C->>A: Send evaluation input
    A->>H: handler(request)
    H->>P: Call your app in-process
    P-->>H: Output
    H-->>A: return output
    A-->>C: Relay output back
    Note over C: Evaluation continues

Your handler is imported once when the agent starts, so expensive setup — loading a model, creating clients — happens a single time. The function is then called once per input, over the same long-lived connection.

Requirements

• Outbound internet access on port 443 (WSS) from the machine running the agent
• A runtime where your pipeline is importable (its dependencies installed)
• No inbound ports need to be opened

Quick Start

A handler runs in the same runtime as the app it calls, so pick the tab for the language your pipeline is written in. Python and TypeScript load your handler from a file; Go, Java, and Rust register the handler in code as a library; any other language connects through a small local endpoint and the agent's forwarding mode.

The Handler

A handler is any function registered with the SDK — @handler in Python, handler(...) in TypeScript and Rust, confidentagent.Handler(...) in Go, Decorator.handler(...) in Java. The agent discovers it at startup; the function name is yours. Exactly one handler per agent process. (For languages that use the HTTP fallback, this section doesn't apply — your endpoint defines the contract instead.)

Request

Your handler receives one Request object. The contract carries the same data in every native SDK, but field names and types follow each language's conventions. Read only the fields you need; unused fields are safe to ignore.

Response

Return a Response containing your app's output. The contract carries the same data in every native SDK, with language-specific field names and types. A plain string remains supported as shorthand for output-only handlers.

from confident_agent import handler, Request, Response

@handler
def rag_handler(request: Request) -> Response:
    result = pipeline(request.input)
    return Response(
        output=result.answer,
        retrieval_context=result.chunks,
        tools_called=result.tools,
    )

import { handler, type Request, type Response } from "confident-agent";

handler((request: Request): Response => {
  const result = pipeline(request.input);
  return {
    output: result.answer,
    retrievalContext: result.chunks,
    toolsCalled: result.tools,
  };
});

confidentagent.Handler(func(request confidentagent.Request) (any, error) {
    result := pipeline(request.Input)
    return confidentagent.Response{
        Output:           result.Answer,
        RetrievalContext: result.Chunks,
        ToolsCalled:      result.Tools,
    }, nil
})

handler((Request request) -> {
    var result = pipeline(request.input);
    var response = new Response(result.answer);
    response.retrievalContext = result.chunks;
    response.toolsCalled = result.tools;
    return response;
});

handler(|request: Request| async move {
    let result = pipeline(&request.input);
    Ok(Response {
        output: result.answer,
        retrieval_context: Some(result.chunks),
        tools_called: Some(result.tools),
        state: None,
    })
});

Handling Errors

If your handler raises or returns an error, the agent reports the failure for that single input and the run continues — one bad input won't abort the whole evaluation. The error message is surfaced on the test case so you can debug it.

from confident_agent import handler, Request, Response

@handler
def summarize_handler(request: Request) -> Response:
    if not request.input:
        raise ValueError("empty input")
    return Response(
        output=summarize(request.input, request.context),
    )

import { handler, type Request, type Response } from "confident-agent";

handler((request: Request): Response => {
  if (!request.input) {
    throw new Error("empty input");
  }
  return {
    output: summarize(request.input, request.context),
  };
});

confidentagent.Handler(func(request confidentagent.Request) (any, error) {
    if request.Input == "" {
        return nil, errors.New("empty input")
    }
    return confidentagent.Response{
        Output: summarize(request.Input, request.Context),
    }, nil
})

handler((Request request) -> {
    if (request.input.isEmpty()) {
        throw new IllegalArgumentException("empty input");
    }
    return new Response(
        summarize(request.input, request.context)
    );
});

handler(|request: Request| async move {
    if request.input.is_empty() {
        return Err(HandlerError::new("empty input"));
    }
    Ok(Response {
        output: summarize(&request.input, &request.context),
        ..Response::default()
    })
});

Configuration

In Python and TypeScript, the --handler CLI flag and the CONFIDENT_HANDLER environment variable are equivalent; use whichever fits how you run the agent. In Go, Java, and Rust the handler is registered in code, so neither applies.

Testing Your Handler

Click Ping on the connection. Confident AI sends one sample input through the tunnel, runs your handler, and shows what it returned — a green result confirms the handler is discovered, runs, and produces an output. If the handler isn't found, raises, or returns nothing, the error is shown so you can fix it before running a full evaluation or assessment.

✅ Done. Your AI app is connected with no endpoint — ready to run evaluations and red teaming.

Multi-Turn

For multi-turn evaluations, request.turns carries the conversation so far and request.state carries anything you returned from the previous turn. Return a Response with an updated state to thread information forward. See Multi-Turn State for the full model.

Next Steps

Last updated on August 3, 2026