OpenAI

OpenAI provides GPT models for chat, reasoning, audio, and image generation. UGENT integrates with OpenAI through the native openai provider type, and also supports Azure OpenAI, the Responses API, and the Realtime API.

Overview

Capability	Models	API
Chat / reasoning	gpt-5.5, gpt-5.4-mini	Chat Completions or Responses
Image generation	DALL-E, GPT Image	Images API
Text-to-speech	gpt-4o-mini-tts, tts-1	Audio Speech API
Speech-to-text	whisper-1, gpt-4o-transcribe	Audio Transcriptions API
Native audio	gpt-4o-realtime-preview	Realtime API (WebSocket)
Built-in search	Web search	Chat Completions / Responses

Quick Start

[llm]
default_instance = "gpt"

[llm.instances.gpt]
type = "openai"
api_key = "$OPENAI_API_KEY"
default_model = "gpt-5.5"
context_window = 272000
max_tokens = 100000
temperature = 0.3

Store your API key securely:

echo -n "sk-your-openai-api-key" | ugent secret add --stdin openai_api_key

[llm.instances.gpt]
api_key_ref = "@openai_api_key"

Chat Completions vs Responses API

OpenAI exposes two HTTP dialects. UGENT selects between them with api_dialect:

Value	Behavior
auto (default)	Native openai uses the Responses API; all others use Chat Completions
chat_completions	Force the Chat Completions API (/chat/completions)
responses	Force the Responses API (/responses)

[llm.instances.gpt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-5.5"
api_dialect = "responses"        # or "chat_completions" or "auto"

Audio and transcription models always use Chat Completions regardless of api_dialect, because they do not support the Responses endpoint.

Extra Request Fields (extra_body)

Any Responses API or Chat Completions field that UGENT does not model directly can be passed through extra_body. Keys are merged into the top-level request body:

[llm.instances.gpt.reasoning]
mode = "disabled"                # extra_body requires a reasoning block

[llm.instances.gpt.reasoning.provider.openai_compatible]
extra_body = { parallel_tool_calls = false, tool_choice = "auto", service_tier = "flex", top_p = 0.9, instructions = "Be concise." }

WARNING

Do not use extra_body for fields that have dedicated settings (temperature, max_output_tokens, store, previous_response_id, reasoning). The dedicated setting is authoritative.

Reasoning

[llm.instances.gpt.reasoning]
mode = "enabled"                 # disabled | enabled | adaptive
effort = "high"                  # minimal | low | medium | high | xhigh | max

• mode = "adaptive" — UGENT activates reasoning based on task complexity.
• effort = "high" or "max" — deeper reasoning chains, more tokens.

Built-in Web Search

Ground responses with real-time web search without a separate tool:

[llm.instances.gpt.search]
enabled = true

[llm.instances.gpt.search.openai]
search_context_size = "medium"   # low | medium | high

When enabled, OpenAI's built-in search is active during generation.

Text-to-Speech (TTS)

[llm.instances.gpt.media.tts]
enabled = true
model = "gpt-4o-mini-tts"        # or tts-1 for faster, lower quality
default_voice = "alloy"          # alloy, echo, fable, onyx, nova, shimmer

Speech-to-Text (STT)

[llm.instances.gpt.media.stt]
enabled = true
model = "gpt-4o-transcribe"      # or whisper-1

Image Generation

[llm.instances.gpt.media.generation.image]
enabled = true
model = "gpt-image-1"            # or dall-e-3
default_size = "1024x1024"
response_format = "url"
max_images_per_request = 1

Native Audio (Realtime API)

The OpenAI Realtime API enables low-latency voice conversations over WebSocket. UGENT supports it through the native audio configuration:

[llm.instances.gpt.media.audio_transcription]
enabled = true
model = "gpt-4o-transcribe"

[llm.instances.gpt.media.tts]
enabled = true
model = "gpt-4o-mini-tts"
default_voice = "alloy"

Use the voice mode in the TUI or web interface to start a real-time audio session. UGENT manages the WebSocket lifecycle automatically.

Azure OpenAI

Azure OpenAI is configured as an openai-compatible instance. UGENT auto-detects Azure hosts (*.openai.azure.com, *.services.ai.azure.com) and applies Azure's REST conventions:

• Auth header defaults to api-key (not Authorization: Bearer)
• Chat Completions is deployment-scoped
• Responses API is resource-scoped

Chat Completions (Classic Deployment)

[llm.instances.azure]
type = "openai-compatible"
api_key = "$AZURE_OPENAI_API_KEY"
base_url = "https://YOUR-RESOURCE.openai.azure.com"
deployment = "your-deployment-name"
api_version = "2024-10-21"
default_model = "gpt-4o"

Responses API on Azure

[llm.instances.azure_responses]
type = "openai-compatible"
api_key = "$AZURE_OPENAI_API_KEY"
base_url = "https://YOUR-RESOURCE.openai.azure.com"
deployment = "gpt-4.1"
api_version = "2025-04-01-preview"
default_model = "gpt-4.1"
api_dialect = "responses"

Azure v1 Surface (Next-Gen)

The v1 surface needs no api-version and no deployment path segment:

[llm.instances.azure_v1]
type = "openai-compatible"
api_key = "$AZURE_OPENAI_API_KEY"
base_url = "https://YOUR-RESOURCE.openai.azure.com/openai/v1"
deployment = "gpt-4.1"
default_model = "gpt-4.1"
api_dialect = "responses"

Microsoft Entra ID (Azure AD) Authentication

Instead of a static API key, authenticate with an auto-refreshed Entra ID bearer token. When azure_auth is set, no api_key is required.

Service Principal (client_secret):

[llm.instances.azure_entra]
type = "openai-compatible"
base_url = "https://YOUR-RESOURCE.openai.azure.com"
deployment = "gpt-4o"
api_version = "2024-10-21"
default_model = "gpt-4o"

[llm.instances.azure_entra.azure_auth]
method = "client_secret"
tenant_id = "$AZURE_TENANT_ID"
client_id = "$AZURE_CLIENT_ID"
client_secret = "@azure_client_secret"
# scope = "https://cognitiveservices.azure.com/.default"

Managed Identity (for UGENT running inside Azure):

[llm.instances.azure_entra.azure_auth]
method = "managed_identity"
# client_id = "$AZURE_USER_ASSIGNED_CLIENT_ID"   # user-assigned only

Azure CLI (local development):

[llm.instances.azure_entra.azure_auth]
method = "azure_cli"

tenant_id, client_id, and client_secret are resolved like any other secret (@handle, $ENV_VAR, or inline).

Complete Configuration Example

[llm]
default_instance = "gpt"

[llm.instances.gpt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-5.5"
context_window = 272000
max_tokens = 100000
temperature = 0.3
api_dialect = "auto"

[llm.instances.gpt.reasoning]
mode = "enabled"
effort = "high"

[llm.instances.gpt.search]
enabled = true

[llm.instances.gpt.search.openai]
search_context_size = "medium"

# TTS
[llm.instances.gpt.media.tts]
enabled = true
model = "gpt-4o-mini-tts"
default_voice = "alloy"

# STT
[llm.instances.gpt.media.stt]
enabled = true
model = "gpt-4o-transcribe"

# Image generation
[llm.instances.gpt.media.generation.image]
enabled = true
model = "gpt-image-1"
default_size = "1024x1024"

Using OpenAI with Other Providers

[llm]
default_instance = "gpt"

[llm.instances.gpt]
type = "openai"
api_key_ref = "@openai_api_key"
default_model = "gpt-5.5"

[llm.instances.gemini]
type = "google"
api_key_ref = "@google_api_key"
default_model = "gemini-3.5-pro"

[llm.instances.azure]
type = "openai-compatible"
api_key_ref = "@azure_api_key"
base_url = "https://my-resource.openai.azure.com"
deployment = "gpt-4o"
api_version = "2024-10-21"
default_model = "gpt-4o"

Switch with /model gpt, /model gemini, or /model azure.