StepAudio 3 Realtime

StepFun's WebSocket Realtime service provides text/audio input and output, custom Function Calling, and server VAD. Gateway continues to own backend agents and tool execution.

Configuration

Edit the user configuration file shown by qwenaudio config:

QWEN_AUDIO_REALTIME_PROVIDER=stepfun
STEPFUN_API_KEY=your-stepfun-key

Optional setting	Default	Description
STEPFUN_REALTIME_URL	wss://api.stepfun.com/v1/realtime	WebSocket endpoint
STEPFUN_REALTIME_MODEL	stepaudio-3-realtime-preview	Currently supported model profile
STEPFUN_REALTIME_VOICE	Empty	Service default, or a voice ID supported by the model

Credentials and voices are independent of DashScope. Unknown model IDs fail before connecting; new models require a capability profile. Preview availability and replacement follow the official model documentation.

In Desktop, select StepFun under Voice Frontend, enter the key, and apply. Restart a terminal Gateway; for an installed service, run qwenaudio gateway restart. config show and config set --realtime-model use the active provider's model catalog. Remote clients use their Gateway's configuration.

Integration boundary

• Only Gateway-managed type: "function" tools are registered. StepFun's built-in type: "web_search" and type: "retrieval" tools are not enabled. A custom Gateway function named web_search still executes through Gateway.
• Input/output use mono 24 kHz PCM16, negotiated through Gateway. The current profile does not support image or video transport.
• The published API does not fully specify transient response instructions. Announcements and delivery instructions use ordinary conversation items before response.create; these remain in conversation history. Automatic corrections requiring transient instructions stay disabled.
• Thinking deltas only signal response activity. The adapter normalizes cancellation into a common terminal event.

Current validation limits

Live-service checks cover voice input/audio output, custom function calls, tool-result delivery, and continuing after cancellation. This does not establish interaction quality in every environment: desktop speaker tests still showed frequent interruptions and the model verbally asking for permission without a tool call. Such speech neither creates a backend task nor grants authorization. This integration adds no automatic authorization, forced tool calls, or automatic replay rules.

For interrupted playback, correlate Gateway logs by responseId / turnId: realtime.provider.speech_started, realtime.provider.speech_stopped, realtime.response.done, and realtime.playback.started/ended/cancelled. These events contain no raw audio or API keys. Restart Gateway after updating to load the new code.

References: Realtime API and developer guide. Local protocol tests cover connection, tool output, announcements, and cancellation. Cloud voices, model behavior, and latency require a valid key to verify.