Skip to main content
Best Practices

Integrate the real-time voice conversation model

Ask the Agent tool to create and run a voice client and connect to the Token Plan real-time voice conversation model.

Supported models

ModelCapability
qwen-audio-3.0-realtime-plusReal-time voice conversation
For more models, see real-time voice conversation models supported by Token Plan.

Prerequisites

  • A Token Plan subscription. For the Personal Edition, get the sk-sp- API key from My Subscription (Personal). For the Team Edition, assign a seat in Member Management, then get the API key.
  • Python 3.9 or later, a microphone, and speakers.

Step 1: Send the integration instruction to the Agent tool

Expand the integration instruction below and send it to the Agent tool you use. It reads the configured Token Plan key, installs dependencies, creates qwen_realtime_voice.py, and runs it. You do not need to configure the key manually or write Python.
Set up the Token Plan real-time voice conversation model qwen-audio-3.0-realtime-plus in the current project:

1. Use the Token Plan API key already configured in the current Agent tool (prefer TOKEN_PLAN_API_KEY, then ANTHROPIC_AUTH_TOKEN). If neither is available, ask me to configure a Token Plan key in the Agent tool first. Do not display the key or write it to files or logs.
2. Install dependencies: pip install "websockets>=15" sounddevice certifi
3. Create qwen_realtime_voice.py with the following content:

```python
#!/usr/bin/env python3
import asyncio
import base64
import json
import os
import queue
import ssl
import threading

import certifi
import sounddevice as sd
import websockets

API_KEY = os.environ.get("TOKEN_PLAN_API_KEY") or os.environ.get("ANTHROPIC_AUTH_TOKEN")
if not API_KEY:
  raise SystemExit("Token Plan API Key not found. Configure it in the Agent tool first.")
WS_URL = "wss://token-plan.maas.qwencloudapi.com/api-ws/v1/realtime?model=qwen-audio-3.0-realtime-plus"

play_queue = queue.Queue()


def play_audio():
  with sd.RawOutputStream(samplerate=24000, channels=1, dtype="int16") as stream:
    while True:
      chunk = play_queue.get()
      if chunk is None:
        break
      stream.write(chunk)


async def receive(ws):
  transcript = []
  async for message in ws:
    event = json.loads(message)
    event_type = event["type"]
    if event_type == "conversation.item.input_audio_transcription.completed":
      print(f"\n[You] {event.get('transcript', '')}", flush=True)
    elif event_type == "response.audio.delta":
      play_queue.put(base64.b64decode(event["delta"]))
    elif event_type == "response.audio_transcript.delta":
      transcript.append(event.get("delta", ""))
    elif event_type == "response.audio_transcript.done":
      print(f"[AI] {''.join(transcript)}", flush=True)
      transcript.clear()
    elif event_type == "error":
      print(f"[Error] {event}", flush=True)


async def send_mic(ws):
  loop = asyncio.get_running_loop()
  audio_queue = asyncio.Queue()

  def callback(indata, frames, time_info, status):
    loop.call_soon_threadsafe(audio_queue.put_nowait, bytes(indata))

  with sd.RawInputStream(
    samplerate=16000,
    channels=1,
    dtype="int16",
    blocksize=1600,
    callback=callback,
  ):
    print("Start speaking. Press Ctrl+C to stop.", flush=True)
    while True:
      chunk = await audio_queue.get()
      await ws.send(
        json.dumps(
          {
            "type": "input_audio_buffer.append",
            "audio": base64.b64encode(chunk).decode(),
          }
        )
      )


async def main():
  context = ssl.create_default_context(cafile=certifi.where())
  async with websockets.connect(
    WS_URL,
    ssl=context,
    additional_headers={"Authorization": f"Bearer {API_KEY}"},
    max_size=None,
  ) as ws:
    await ws.recv()
    await ws.send(
      json.dumps(
        {
          "type": "session.update",
          "session": {
            "modalities": ["audio", "text"],
            "voice": "longanqian",
            "instructions": "You are a voice assistant. Keep answers concise and natural.",
            "input_audio_format": "pcm",
            "output_audio_format": "pcm",
            "turn_detection": {
              "type": "server_vad",
              "threshold": 0.5,
              "silence_duration_ms": 600,
            },
          },
        }
      )
    )
    threading.Thread(target=play_audio, daemon=True).start()
    await asyncio.gather(receive(ws), send_mic(ws))


if __name__ == "__main__":
  try:
    asyncio.run(main())
  except KeyboardInterrupt:
    pass
  finally:
    play_queue.put(None)
```

4. Use the Bash tool to run python3 qwen_realtime_voice.py with a timeout of 600000 milliseconds.
5. After the script starts, tell the user to start speaking and press Ctrl+C to stop.
On the first run, allow the app running the Agent tool (such as Cursor or Terminal) to access the microphone in system settings. Otherwise, the script receives no audio. Use headphones to prevent the microphone from capturing the model's audio and interrupting the response.

Step 2: Start the voice conversation

After the Agent tool starts the script, speak when you see the start message. The terminal shows the transcript and the model's reply, and the speakers play the model's audio. Press Ctrl+C to end the session.
The Agent tool installs dependencies, creates the script, and starts the session. Audio data travels between the local client and Token Plan and does not pass through the Agent tool's text model.

Save as a tool extension

To use the instruction again, ask the Agent tool to save it as a Skill, Slash Command, or Agent configuration for your current tool, then start it through that tool's trigger. Common tools and their extension mechanisms and paths:
ToolExtension mechanismConfiguration file path
Claude CodeSlash Command.claude/commands/voice-chat.md
CodexSkill~/.codex/skills/token-plan-voice/SKILL.md
Qwen CodeSkill~/.qwen/skills/voice-chat/SKILL.md
OpenCodeAgent.opencode/agents/voice-chat.md
OpenClawSkill~/.openclaw/workspace/skills/token-plan-voice/SKILL.md
Hermes AgentSkill~/.hermes/skills/media/voice-chat/SKILL.md
QoderSkill~/.qoder/skills/voice-chat/SKILL.md
For the front matter format of Skills and Agents, see Integrate multimodal generation models.

Limits

  • With external speakers, the model's audio may be captured by the microphone and interrupt the response. Use headphones.
  • Bash calls from the Agent tool have a timeout. For long conversations, run the script as a standalone process or wrap it in an MCP server.
  • The current solution handles voice input and output only.