Extending qwen-audio-agent
The runtime is generic; every scenario-specific behavior enters through a declared seam. This page maps the seams and points to the guide for each.
Frontend Tools: MCP, OpenAPI, Profiles
Add chatbot tools without touching the voice path: connect MCP servers through the frontend MCP client, expose selected REST operations from an OpenAPI 3.x document, or bundle persona plus tool configuration as a versioned Frontend Profile.
→ Frontend MCP client · Frontend OpenAPI adapter · Frontend Profiles
Voice Frontend: Custom Realtime Provider
Swap the realtime speech model for another cloud service or your own stack by implementing the provider contract and registering it in the provider registry.
Knowledge: Retrieval Provider
The Gateway defines a small retrieval boundary instead of shipping a RAG stack — connect the knowledge system you already operate.
→ Knowledge Retrieval Provider
Backend: Connect a New Agent
Four paths put a backend behind the protocol-neutral BackendPort: the zero-code generic ACP entry, a remote A2A agent, a custom adapter via the Backend Adapter SDK, or a first-class backend with one-click install.
→ Connecting a New Backend · Backend Adapter SDK · A2A Backend Adapter
Persona and Memory
The assistant's name, personality, and voice live in ASSISTANT.md; durable user facts live in USER.md / MEMORY.md. All are plain Markdown in the config directory, editable while the gateway keeps its constrained write path.
→ Assistant Profile and User Preferences · Long-Term Memory
Client: Build Your Own
The Gateway speaks typed events over a single WebSocket. Build a custom client against the client protocol, or embed the assistant into a host page through the stability contract — the same channel the desktop orb, TUI, and WebUI use. examples/custom-conversation-client/ is a minimal starting point.
→ Gateway Client Protocol · Gateway Contract
Desktop Appearance
The desktop orb renders replaceable pet skins: a pet.json manifest plus a spritesheet.