OpenAI

OpenAI provides GPT models for chat, reasoning, audio, and image generation. UGENT integrates with OpenAI through the native openai provider type, and also supports Azure OpenAI, the Responses API, and the Realtime API.

Overview

Capability	Models	API
Chat / reasoning	gpt-6-sol, gpt-6-luna, gpt-6.1-sol, gpt-6-astra	Responses API
Audio chat (hears a recording)	gpt-audio-1.5	Chat Completions
Image generation	DALL-E, GPT Image	Images API
Text-to-speech	gpt-4o-mini-tts (default)	Audio Speech API
Speech-to-text	gpt-transcribe (default)	Audio Transcriptions API
Realtime voice	gpt-realtime-2.1	Realtime API (WebSocket)
Built-in search	Web search	Responses / Chat Completions

The GPT-6 models at a glance:

Model	Context	Max output	Reasoning off?	Notes
gpt-6-sol	1.05M	128K	Yes (none)
gpt-6-luna	1.05M	128K	Yes (none)	Lowest price
gpt-6.1-sol	1.05M	128K	No (lowest is low)	Newer Sol; tools only on the Responses API
gpt-6-astra	1.05M	128K	No (lowest is low)	Highest price; tools only on the Responses API

All four reason by default, at effort medium, and refuse temperature, so UGENT leaves it out. GPT-5.6 and later follow the same rules; see GPT-5.6 and later.

Quick Start

[llm]
default_instance = "gpt"

[llm.instances.gpt]
type = "openai"
api_key = "$OPENAI_API_KEY"
default_model = "gpt-6-sol"
context_window = 1050000
max_tokens = 128000

Store your API key securely. Copy it to your clipboard, then pipe it in, so it never appears in your shell history:

macOS

pbpaste | ugent secret add @openai_api_key --provider openai --kind api-key --stdin

Linux X11

xclip -selection clipboard -o | ugent secret add @openai_api_key --provider openai --kind api-key --stdin

Vault & Secrets has the Wayland and Windows forms. Then reference the key with its vault handle:

[llm.instances.gpt]
api_key_ref = "@openai_api_key"

Chat Completions vs Responses API

OpenAI exposes two HTTP dialects. UGENT selects between them with api_dialect:

Value	Behavior
auto (default)	Native openai uses the Responses API for every model except its audio chat models, which use Chat Completions. openai-compatible uses the Responses API for GPT-5.6 and later (except on classic Azure addressing) and Chat Completions for everything else
chat_completions (alias chat)	Force the Chat Completions API (/chat/completions)
responses	Force the Responses API (/responses)

[llm.instances.gpt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-6-sol"
api_dialect = "responses"        # or "chat_completions" or "auto"

The audio chat models are gpt-audio*, gpt-4o-audio-preview, gpt-4o-mini-audio-preview and any id with an audio token. They go to Chat Completions because only it carries a recording to them; an explicit api_dialect still wins. Transcription and speech models (whisper, *-transcribe, *-tts) never go to the Responses API, whatever api_dialect says.

GPT-5.6 and later

These models reason at effort medium when a request names none, and Chat Completions carries their function tools only at effort none. GPT-6.1 Sol and GPT-6 Astra have no none, so on Chat Completions they cannot call tools at all. UGENT therefore:

• sends them to the Responses API on an openai-compatible instance whose api_dialect is unset, recognising gateway ids (us.openai.gpt-6-sol) and a deployment alias through its catalog_model. Classic Azure addressing stays on Chat Completions (see Azure OpenAI);
• on Chat Completions, whether forced or classic Azure, sends their tool turns at effort none, with a log line, and refuses a GPT-6.1 Sol or GPT-6 Astra tool turn before the request with an error naming the fix: api_dialect = "responses" with a base URL and api_version that serve /responses.

Extra Request Fields (extra_body)

Any Responses API or Chat Completions field that UGENT does not model directly can be passed through extra_body. Keys are merged into the top-level request body:

[llm.instances.gpt.reasoning]
mode = "enabled"                 # extra_body requires a reasoning block
effort = "medium"

[llm.instances.gpt.reasoning.provider.openai_compatible]
extra_body = { parallel_tool_calls = false, tool_choice = "auto", service_tier = "flex", instructions = "Be concise." }

extra_body needs a [reasoning] block, and the block's mode still applies: on GPT-5.6 and later mode = "disabled" sends the lowest effort the model accepts (see Reasoning), so write the reasoning you want alongside extra_body. UGENT sends extra_body as written. On GPT-5 and later, OpenAI accepts top_p, like temperature, only at effort none, so keep sampling fields out of extra_body on a reasoning request.

WARNING

Do not use extra_body for fields that have dedicated settings (temperature, max_output_tokens, store, previous_response_id, reasoning). The dedicated setting is authoritative.

Tools

UGENT sends tools in OpenAI's strict mode where their schema allows it. A tool whose schema uses a keyword strict mode refuses (allOf, oneOf, not, if/then/else, dependentRequired, dependentSchemas and similar) is sent non-strict instead, so one such tool, from an MCP server say, does not fail the whole request. On native OpenAI's Chat Completions an optional property of a strict tool is sent as anyOf with null, the form that endpoint accepts.

Reasoning

[llm.instances.gpt.reasoning]
mode = "enabled"                 # disabled | enabled | adaptive
effort = "high"                  # minimal | low | medium | high | xhigh | max

[llm.instances.gpt.reasoning.provider.openai]
summary = "auto"                 # ask for a reasoning summary

• No [reasoning] block. GPT-5.6 and later get effort medium with a reasoning summary, so the steps show. OpenAI generates summaries only for verified organizations: when an endpoint refuses a request carrying the summary UGENT added, UGENT sends it again without one, stops asking on that instance (one warning) and reasoning continues.
• With a block, what you write is sent: a summary you set is always sent, and a block without one asks for none.
• mode = "disabled" on GPT-5.6 and later sends the least effort the model accepts: none, or low on GPT-6.1 Sol and GPT-6 Astra, which cannot switch reasoning off (a warning says so). Leaving the field out would not turn it off.
• mode = "adaptive" is not native on OpenAI and runs as enabled, with a warning.
• effort is sent as the nearest value the model accepts: xhigh from GPT-5.2 on and max from GPT-5.6 on as written, and minimal, which models from GPT-5.1 on refuse, as low.

Reasoning summaries stream as progress notes. The TUI shows them as a one-line status, never in the answer.

Reasoning across turns

On the Responses API, a reasoning model's reasoning is replayed to it on later requests of the conversation. For that, UGENT asks for encrypted reasoning and sends store: false, so OpenAI does not store the response. Setting store = true turns this off: OpenAI stores the response and keeps the reasoning itself, and no encrypted reasoning is asked for. store = false is sent as written, and replay goes on.

[llm.instances.gpt.cache]
store = true                     # OpenAI keeps the response; no encrypted reasoning is asked for

Reasoning is replayed only to the model and upstream that produced it: the same base URL, key and routing headers. A conversation moved to another deployment, another gateway route or another key goes on without the earlier reasoning. If an endpoint refuses replayed reasoning, the request is sent once more without it; a second refusal within ten minutes suspends replay for that model for thirty.

Built-in Web Search

Ground responses with real-time web search without a separate tool:

[llm.instances.gpt.builtin_search]
enabled = true

[llm.instances.gpt.builtin_search.openai]
search_context_size = "medium"   # low | medium | high

When enabled, OpenAI's built-in search is active during generation. Toggle it at runtime with /builtin-search on gpt or /builtin-search off gpt.

Audio Chat Models

gpt-audio-1.5 hears a voice message itself, on Chat Completions; every other native OpenAI model hears only its transcript, because the Responses API takes no audio. A recording reaches it as MP3 or WAV, the only formats OpenAI's input_audio takes (a WAV labelled audio/x-wav, audio/wave or audio/vnd.wave included); a recording in any other format is represented by its transcript.

Two things make the audio models a poor default for agent work. They refuse a request that carries no audio, so a typed message in a conversation on one fails; offer them for voice only. And with the agent's long instructions their answers are less reliable than a text model's. For most deployments, transcribe with gpt-transcribe and answer with a text model. The whole voice path is in Voice.

Text-to-Speech (TTS)

[llm.instances.gpt.media.tts]
enabled = true
model = "gpt-4o-mini-tts"        # the default
voice = "alloy"                  # the default
format = "wav"                   # the default

Speech goes to /audio/speech. An openai-compatible instance with this block speaks through its own /audio/speech, so a local speech server works the same way. Azure OpenAI is not supported here: its deployments are addressed differently. An instance with [media.tts] is what reads answers aloud; see Voice.

OpenAI retires its text-to-speech models, gpt-4o-mini-tts included, on 2027-01-06. For read aloud on a deployment that will outlive them, prefer Gemini (see Voice).

Speech-to-Text (STT)

[llm.instances.gpt.media.stt]
enabled = true
model = "gpt-transcribe"         # the default

Transcription goes to the instance's own /audio/transcriptions with the instance's own key, so a key held in the vault resolves as it does for chat. An openai-compatible instance with an explicit [media.stt] block transcribes through its own endpoint too; Azure OpenAI is not supported. gpt-transcribe replaced the gpt-4o transcribe models, which OpenAI shuts down on 2027-02-26, and OpenAI bills it per minute of audio. Voice messages are transcribed before routing; see Voice.

Image Generation

[llm.instances.gpt.media.generation.image]
enabled = true
model = "gpt-image-1"            # or dall-e-3
default_size = "1024x1024"
response_format = "url"
max_images_per_request = 1

Native Audio (Realtime API)

The Realtime API holds a spoken conversation over a WebSocket. UGENT speaks the GA Realtime API (wss://api.openai.com/v1/realtime, no beta header) and uses it in two places: the TUI's /live command, and live calls from the web workspace.

[llm.instances.openai-rt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-6-luna"

[llm.instances.openai-rt.media.live]
enabled = true
model = "gpt-realtime-2.1"
voice = "marin"

Key	Default	Description
enabled	false	Offer this instance for live calls
model	gpt-realtime-2.1 for calls; the instance's default_model for /live	The Realtime model (on Azure, the deployment name)
voice	marin for calls, alloy for /live	The voice replies are spoken in
input_audio	true	/live: capture the microphone (needs the live-audio build)
output_audio	true	/live: play spoken replies

Both sides are transcribed into the conversation, the person's with gpt-transcribe, and turns are taken by semantic voice activity detection. /live accepts only a model whose name contains realtime or live.

Live calls from the web workspace are served by native OpenAI instances only, not Azure; the TUI's /live also works with Azure OpenAI. How a call hands work to the agent is in Voice, and the command itself in Slash Commands.

Azure OpenAI

Azure OpenAI is configured as an openai-compatible instance. UGENT auto-detects Azure hosts (*.openai.azure.com, *.services.ai.azure.com) and applies Azure's REST conventions:

• Auth header defaults to api-key (not Authorization: Bearer)
• Chat Completions is deployment-scoped
• Responses API is resource-scoped

For GPT-5.6 and later, use the Responses API (the v1 surface below, or api_dialect = "responses"): classic deployment addressing stays on Chat Completions, where their tool turns run without reasoning and GPT-6.1 Sol and GPT-6 Astra cannot call tools.

Chat Completions (Classic Deployment)

[llm.instances.azure]
type = "openai-compatible"
api_key = "$AZURE_OPENAI_API_KEY"
base_url = "https://YOUR-RESOURCE.openai.azure.com"
deployment = "your-deployment-name"
api_version = "2024-10-21"
default_model = "gpt-5.4-mini"

Responses API on Azure

[llm.instances.azure_responses]
type = "openai-compatible"
api_key = "$AZURE_OPENAI_API_KEY"
base_url = "https://YOUR-RESOURCE.openai.azure.com"
deployment = "gpt-6-sol"
api_version = "2025-04-01-preview"
default_model = "gpt-6-sol"
api_dialect = "responses"

Azure v1 Surface (Next-Gen)

The v1 surface needs no api-version and no deployment path segment. GPT-5.6 and later go to the Responses API there without api_dialect:

[llm.instances.azure_v1]
type = "openai-compatible"
api_key = "$AZURE_OPENAI_API_KEY"
base_url = "https://YOUR-RESOURCE.openai.azure.com/openai/v1"
deployment = "gpt-6-sol"
default_model = "gpt-6-sol"

Microsoft Entra ID (Azure AD) Authentication

Instead of a static API key, authenticate with an auto-refreshed Entra ID bearer token. When azure_auth is set, no api_key is required.

Service Principal (client_secret):

[llm.instances.azure_entra]
type = "openai-compatible"
base_url = "https://YOUR-RESOURCE.openai.azure.com"
deployment = "gpt-5.4-mini"
api_version = "2024-10-21"
default_model = "gpt-5.4-mini"

[llm.instances.azure_entra.azure_auth]
method = "client_secret"
tenant_id = "$AZURE_TENANT_ID"
client_id = "$AZURE_CLIENT_ID"
client_secret = "@azure_client_secret"
# scope = "https://cognitiveservices.azure.com/.default"

Managed Identity (for UGENT running inside Azure):

[llm.instances.azure_entra.azure_auth]
method = "managed_identity"
# client_id = "$AZURE_USER_ASSIGNED_CLIENT_ID"   # user-assigned only

Azure CLI (local development):

[llm.instances.azure_entra.azure_auth]
method = "azure_cli"

tenant_id, client_id, and client_secret are resolved like any other secret (@handle, $ENV_VAR, or inline).

Through a Gateway

A gateway that fronts OpenAI's models (or serves them from Amazon Bedrock) is an openai-compatible instance. Its model ids find their catalog entry by themselves, so us.openai.gpt-6-sol is priced and sized as gpt-6-sol and goes to the Responses API; see Gateway model ids. Bedrock refuses temperature on GPT-6 at any value, and UGENT leaves it out for GPT-5 and later behind any wrapper. For a model the catalog does not know, set disable_temperature = true.

[llm.instances.gateway]
type = "openai-compatible"
base_url = "https://gateway.example.com/v1"
api_key_ref = "@gateway_key"
default_model = "us.openai.gpt-6-sol"

If your gateway does not serve /responses, set api_dialect = "chat_completions", with the limits described in GPT-5.6 and later.

Complete Configuration Example

[llm]
default_instance = "gpt"

[llm.instances.gpt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-6-sol"
context_window = 1050000
max_tokens = 128000
api_dialect = "auto"

[llm.instances.gpt.reasoning]
mode = "enabled"
effort = "high"

[llm.instances.gpt.reasoning.provider.openai]
summary = "auto"

[llm.instances.gpt.builtin_search]
enabled = true

[llm.instances.gpt.builtin_search.openai]
search_context_size = "medium"

# TTS
[llm.instances.gpt.media.tts]
enabled = true
model = "gpt-4o-mini-tts"
voice = "alloy"

# STT
[llm.instances.gpt.media.stt]
enabled = true
model = "gpt-transcribe"

# Image generation
[llm.instances.gpt.media.generation.image]
enabled = true
model = "gpt-image-1"
default_size = "1024x1024"

Using OpenAI with Other Providers

[llm]
default_instance = "gpt"

[llm.instances.gpt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-6-sol"

[llm.instances.gemini]
type = "google"
api_key_ref = "@google_api_key"
default_model = "gemini-3.8-flash"

[llm.instances.azure]
type = "openai-compatible"
api_key_ref = "@azure_api_key"
base_url = "https://my-resource.openai.azure.com/openai/v1"
deployment = "gpt-6-luna"
default_model = "gpt-6-luna"

Switch with /providers use gpt, /providers use gemini, or /providers use azure, or send one message elsewhere with /with gemini <prompt>.