Models and Voices

Reference for the developer-only system model and voice catalog pages, organization-managed models, and the organization-scoped catalog used by staff tooling.

• Routes:
• `/models`
• `/voices`
• Sidebar menu metadata marks both as developer-only (`isDeveloper: true`).
• The pages render `AccessDenied` for non-developer users.
• Frontend `isDeveloper` is populated by mapping backend `is_admin` to frontend `is_developer` in `AuthContext`.
• Backend CRUD endpoints are under `/api/admin/*` and protected by `adminMiddleware`.
• Authenticated organization members use `GET /api/organizations/{organization_id}/ai-models` for the available model catalog; `GET /api/init/ai-voices` remains the read-only voice catalog. The former `GET /api/init/models` model endpoint is no longer available.

• UI supports list/search, detail view, and create/edit/delete.
• The detail pane now also shows provider-enriched metadata when available, including context window, pricing, modalities, and supported generation parameters.
• Current dialog fields:
• display name
• model identifier
• provider
• `is_multimodal`
• `is_realtime`
• `is_tts`
• `is_stt`
• `Input Cost`
• `Output Cost`
• `Cached Input Cost`
• Provider options shown in the dialog:
• `OpenAI`
• `OpenRouter`
• `Google`
• `DeepSeek`
• `Anthropic`
• `Codex`
• `Custom`
• Pricing fields are optional numeric USD values stored per 1M tokens.
• If `Cached Input Cost` is left empty, runtime cost calculation falls back to the normal input cost.
• Backend model records also support a `voices` array, but the current models dialog does not expose that field.
• Models are selected by an alias. Aliases are unique within an organization; an organization-owned model with the same alias as a system model replaces that model for that organization. Model resolution uses the alias rather than the provider/model pair.
• Current provider-specific runtime notes:
• `Codex` models use an organization-level Codex connection rather than a typed provider API key field.
• Codex supports both streamed agent execution and callers that expect a completed JSON generation result, including structured workflow LLM calls. For completed-result calls, the adapter streams from the upstream Responses endpoint, collects completed output items, and returns the JSON response shape expected by the AI SDK.
• Codex Responses requests disable upstream response storage.
• When `web.search` is selected for an agent, Codex models receive the native OpenAI web-search tool.
• `Custom` models are treated as OpenAI-compatible models at the runtime layer.
• For an organization-owned Custom model with its own endpoint, runtime uses that model's saved URL and token ahead of provider-level or system credentials. Otherwise an active organization Custom connection supplies the key and Base URL; if unavailable, runtime falls back to the system Custom provider configuration. Calls using the organization's own model credentials use the organization proxy and do not spend Alloy tokens.
• Custom model identifiers are normalized with a `custom/` prefix in the catalog and that prefix is removed before the model ID is sent to the OpenAI-compatible endpoint.
• When an agent run includes `web.search`, Custom model chat requests ask the configured OpenAI-compatible gateway for web search using its `web` plugin. Runs without that tool do not request the plugin; the gateway must support that plugin for search to work.
• Before custom-model generation or streaming, all system messages are merged into one leading system message with blank-line separators; non-system conversation messages keep their order. This supports self-hosted OpenAI-compatible chat templates that reject multiple or non-leading system messages.
• When a reasoning effort is requested, Custom models receive it as an OpenAI-compatible `reasoning: { effort }` provider option. The runtime does not gate this on the model registry, because the registry cannot vouch for arbitrary OpenAI-compatible endpoints.

Runtime STT behavior

• Inbound Telegram, Slack, and MS Teams audio or voice attachments are transcribed with the AI teammate `stt_model` before the message is sent to the agent.
• Runtime STT accepts only models marked `is_stt`.
• Current runtime STT providers are:
• `OpenAI`
• `OpenRouter`
• `Custom`
• `Google`
• `OpenAI`, `OpenRouter`, and `Custom` use OpenAI-compatible audio transcription endpoints.
• Custom STT sends requests through the same organization or system Base URL resolved for the Custom provider.
• `Google` uses a multimodal language model with the audio file as input.
• If no STT model is configured, the model is not found, the model is not marked `is_stt`, the provider is unsupported, the API key is unavailable, or transcription fails, the agent receives a system note that the voice message could not be transcribed.

Admin API

• `GET /api/admin/ai-models`
• `GET /api/admin/ai-models/metadata`
• `GET /api/admin/ai-models/{id}`
• `POST /api/admin/ai-models`
• `PUT /api/admin/ai-models/{id}`
• `DELETE /api/admin/ai-models/{id}`
• Metadata lookup requires:
• `provider`
• `model`
• Current backend metadata lookup is implemented for `OpenRouter` models and returns resolved model id, name, pricing, and capability flags when found.

Organization-owned Custom models (`/org-settings/ai-models`)

• Organization owners and admins can use Organization Settings > AI Models to add, edit, or delete models served by their own OpenAI-compatible endpoints. The list shows each model's name, alias, URL, and a `Multimodal` badge when applicable. A duplicate alias overrides the Alloy system model with that alias for this organization; removing the override restores the system model if one exists, or leaves references unresolved if none does.
• The Add model and Edit model dialogs accept `Name`, `Alias`, `Model` (the endpoint's model identifier), `API URL` (HTTP or HTTPS), `API token`, `Accepts images`, and optional `Input price`, `Output price`, and `Cached input price` in USD per 1M tokens. Prices are for usage-cost statistics. The saved API token is never displayed; leave the token field blank during editing to keep it. Alias and model identifiers start with a letter or digit and can contain letters, digits, `.`, `_`, `-`, `/`, `:`, `@`, or `=`. The backend stores Custom model identifiers with a `custom/` prefix, which the editor hides and the runtime removes when calling the endpoint.
• Authenticated organization members: `GET /api/organizations/{organization_id}/ai-models` lists system and organization models available to them, one per alias. Admin-only management routes at the same base: `GET /own` lists the organization's own models; `POST` creates; `PATCH /{ai_model_id}` edits; `DELETE /{ai_model_id}` removes an owned model. Create requires name, alias, model identifier, API URL, and token. Update omits the token to preserve it. Responses expose `has_api_token`, not the secret token. These routes do not change the developer-only system model catalog.

Authenticated catalog API

• `GET /api/organizations/{organization_id}/ai-models`
• Returned fields include:
• `id`
• `alias`
• `model`
• `name`
• `provider`
• `is_connected` for provider availability in the current organization
• `is_multimodal`
• `is_realtime`
• `is_tts`
• `is_stt`
• `voices`
• Frontend consumers use this endpoint to populate chat/TTS/STT selectors outside the admin pages.
• For Codex models, `is_connected` is `false` when the current organization has no active Codex provider connection; other providers currently return `true`.

• UI supports list/search, detail view, and create/edit/delete.
• Current dialog fields:
• provider
• voice identifier
• gender
• Provider options shown in the dialog:
• `OpenAI`
• `OpenRouter`
• `Google`
• Backend/provider model supports more than the current dialog exposes:
• backend validation/model enum accepts `OpenAI`, `OpenRouter`, `Google`, `DeepSeek`, and `Anthropic`
• the current create/edit dialog only offers `OpenAI`, `OpenRouter`, and `Google`
• the list page can still display voices from any stored provider
• Voice uniqueness is `(provider, voice)`.

Admin API

• `GET /api/admin/ai-voices`
• `GET /api/admin/ai-voices/{id}`
• `POST /api/admin/ai-voices`
• `PUT /api/admin/ai-voices/{id}`
• `DELETE /api/admin/ai-voices/{id}`

Authenticated catalog API

• `GET /api/init/ai-voices`
• Optional query filters:
• `provider`
• `gender`
• `search`
• AI teammate voice selectors filter this list by the currently selected TTS model provider.

• Chat `model` must not be a realtime, TTS, or STT model.
• `tts_model` must be TTS-capable or realtime-capable.
• `stt_model` must be STT-capable.
• `voice` cannot be set without a `tts_model`.
• When `tts_model` is set, `voice` is required and must exist for that TTS model's provider.
• Current frontend filtering matches the backend validation:
• chat model dropdown excludes `is_realtime`, `is_tts`, and `is_stt`
• TTS dropdown includes `is_tts` or `is_realtime`
• STT dropdown includes `is_stt`

• Developer-only system settings in `/defaults` can define:
• `AI Model Fallback 1`
• `AI Model Fallback 2`
• Runtime execution retries those fallback models in order before the run is marked failed.
• Each individual streamed model call has a five-minute wall-clock timeout. The timeout covers waiting for and consuming the model stream, excludes tool execution, and is classified as a non-terminal model failure so fallback models can be attempted.
• A completed run step that recovered through a fallback model is recorded with `fallback_used` and shown in execution logs as a `Fallback used` badge; the summarized error of the failed attempt is persisted as `last_error` on the step log.
• Primary-model and fallback-model retry diagnostics are written through the shared Mastra logger.
• Provider failure diagnostics are summarized for logs. They keep high-level fields such as provider error name, message, status, retryability, URL, and provider code/message, while raw request and response payload fields are redacted.