API mode (OpenAI Stored Completions)

Zero code changes — flip a setting in OpenAI and paste a read-only key.

API mode is the simplest possible path if you’re already on OpenAI: flip a setting in your OpenAI organization dashboard, create a read-only admin key, paste it into rendfly. No code changes, no proxy in your request path, no SDK to install.

The flow

Three steps, all in dashboards:

  1. Enable Stored Completions on your OpenAI organization. OpenAI’s Stored Completions feature retains conversation records server-side and makes them queryable via API. You turn this on in the OpenAI platform settings. The toggle is at the organization level, so you’ll need owner or admin access.

  2. Create a read-only admin API key. In the OpenAI dashboard, generate an API key with read-only permission scoped to Stored Completions. This key lets rendfly pull conversation records and nothing else — it can’t create API calls, can’t read your billing, and can’t touch other resources. Read-only is a hard requirement; rendfly will reject keys with write permissions at connection time.

  3. Paste it into rendfly’s project settings. Open your project in the rendfly dashboard, go to “Connect provider”, choose API mode, and paste the key. Rendfly will validate access immediately and start pulling conversations on a rolling basis.

Once connected, rendfly’s eval pipeline receives the same conversations it would see through proxy mode. Rule extraction, LLM-as-judge scoring, and drift detection all work identically regardless of ingestion path.

What we read, what we don’t

When rendfly connects to your OpenAI account in API mode, it reads:

  • Conversation content (messages, roles, tool calls) from Stored Completions records
  • Conversation metadata: model, timestamp, token counts, finish reason

Rendfly never touches:

  • Your standard (non-stored) API call history
  • Billing information or usage reports
  • Other OpenAI API keys in your organization
  • Model fine-tunes, files, assistants, or any other API resource

The connection is read-only by design and enforced at the key level. If OpenAI ever adds a more granular permission scope for Stored Completions specifically, rendfly will update to require only that scope.

When to use API mode vs proxy mode

API mode is the right choice when:

  • You’re already on OpenAI and have Stored Completions enabled (or are willing to enable it).
  • You don’t want to touch your agent’s code or deployment at all.
  • A third party manages the agent’s infrastructure and changing the base_url would require a coordination round.

Proxy mode is the right choice when:

  • Your agent uses multiple providers (Anthropic, Groq, Mistral, etc.) in addition to or instead of OpenAI.
  • You’ve decided against Stored Completions for data retention or legal reasons — conversations would sit in OpenAI’s systems longer than your policy allows.
  • You want the same ingestion path regardless of which provider you’re using, so switching providers later doesn’t change your monitoring setup.

Both modes feed the same eval pipeline. You can start with API mode today (if you’re already on OpenAI) and migrate to proxy mode later without losing any historical data.

  • Proxy mode — the alternative for multi-provider setups or when you want zero data held at OpenAI.
  • What is rendfly — how rendfly uses the conversations it captures.
Updated 2026-05-09