Proxy mode

Change your base_url, get monitoring. Universal across providers.

Proxy mode is the default way to connect an agent to rendfly. When it ships, the change is one line: point your provider SDK’s base_url at https://api.rendfly.com/{provider} and we observe everything that flows through — system messages, conversations, tool calls, and all.

Why proxy mode is the default

Most production agents already talk to a provider over HTTPS. Proxy mode sits in that path and mirrors the traffic to rendfly’s eval pipeline without changing anything about your agent’s behavior.

It works across every provider that’s OpenAI-compatible — OpenAI, Anthropic, Gemini, Groq, Mistral, vLLM, and Ollama. Streaming responses are mirrored as they arrive, not buffered, so your agent’s latency profile stays intact. Tool calls and function call round-trips pass through transparently. Vendor-specific request headers are preserved end-to-end.

The infrastructure overhead is real but small: rendfly targets ≤30ms added at p99. The eval pipeline is async — judging runs after the response is forwarded, so your agent never waits on us.

The illustrative one-line change

Once it ships, here’s what the change looks like for the three most common setups. None of these endpoints are live today — they’re illustrative of what the one-line migration will actually look like.

OpenAI Python SDK:

from openai import OpenAI

client = OpenAI(
    api_key="sk-...",
    base_url="https://api.rendfly.com/openai/v1"  # was: https://api.openai.com/v1
)

Anthropic Python SDK:

import anthropic

client = anthropic.Anthropic(
    api_key="sk-ant-...",
    base_url="https://api.rendfly.com/anthropic"  # was: https://api.anthropic.com
)

cURL:

# was: https://api.openai.com/v1/chat/completions
curl https://api.rendfly.com/openai/v1/chat/completions 
  -H "Authorization: Bearer sk-..." 
  -H "X-Rendfly-Key: pk_live_..." 
  -d '{ "model": "gpt-4o", "messages": [...] }'

What you’ll need

Three things before proxy mode can start observing your agent:

  1. A rendfly project. Created from the dashboard — each project maps to one agent and tracks its own rules and alerts independently.
  2. Your existing provider API key. Rendfly forwards requests to the upstream provider on your behalf. Your key leaves your environment only to reach the provider; rendfly never stores it in plaintext.
  3. A rendfly project key. Issued per project, prefixed pk_live_.... You’ll send this as the X-Rendfly-Key header (or via Authorization, once the header scheme is finalized). This is what ties incoming traffic to the right project.

Limits and edge cases

A few behaviors worth knowing before you connect:

  • Streaming. Server-sent event streams are forwarded chunk by chunk as they arrive. rendfly reconstructs the full message for eval in the background after the last chunk.
  • Tool calls. Multi-turn tool/function call conversations are tracked across turns and judged as a unit. The round-trip to your tool server is not proxied — only the LLM calls.
  • Provider-specific headers. Headers like anthropic-version, x-request-id, and OpenAI’s openai-organization pass through unmodified in both directions.
  • Error responses. If the upstream provider returns a 4xx or 5xx, rendfly forwards it verbatim. The failed request is logged but not judged.
  • What is rendfly — how the eval pipeline works once conversations start flowing.
  • LLM-as-judge — what happens to a conversation after rendfly captures it.
Updated 2026-05-09