Qwen-Audio-Realtime Client Events

Client events are sent by the client to the server to configure the session, send audio, manage the conversation, and control inference. User guide: Real-time voice conversation (Qwen-Audio-Realtime). For the event interaction timeline, see WebSocket API.

session.update

After establishing the connection, send this event to update the session's default configuration. Include only the fields you want to change; omitted fields keep their current value. The server validates the parameters on receipt — if they are invalid, it returns an error; if valid, it applies the changes and returns the full configuration.

turn_detection can only be changed before the first audio is sent (IDLE state).

Field	Type	Required	Description
type	string	Required	Event type, fixed as `session.update`.
session	object	Optional	Session configuration.
session.modalities	array	Optional	The model's output modalities. Options: `["text"]` (text output only); `["audio", "text"]` (default, outputs both text and audio).
session.voice	string	Optional	TTS voice name. Default: `longanqian`. Can only be set in the first `session.update`; ignored in subsequent calls. Supports system voices (`longanqian`, `longanlingxin`, `longanlingxi`, `longanxiaoxin`, `longanlufeng`) and voice-cloning voices (a `voice_id` created via the Voice Cloning API).
session.enable_speech_emotion	boolean	Optional	Whether to enable emotion enhancement. When enabled, the response voice exhibits more pronounced emotional variations. Default: `true`. Options: `true`, `false`.
session.instructions	string	Optional	System instructions that set the model's persona, response style, and behavior preferences. Applies to the entire session.
session.input_audio_format	string	Optional	Input audio format. Currently only `pcm` (16 kHz, 16-bit, mono) is supported, and it is the default. Can only be changed before the first audio is sent (IDLE state).
session.output_audio_format	string	Optional	Output audio format. Currently only `pcm` (24 kHz, 16-bit, mono) is supported, and it is the default.
session.max_history_turns	integer	Optional	The maximum number of historical QA turns allowed per request. Range: 1-50. Default: 20.
session.tools	array	Optional	A list of function calling tool definitions. Once configured, the model can decide on its own whether to call a tool based on user input. Each tool includes `type` (fixed as `function`), `function.name` (required), `function.description` (optional), and `function.parameters` (optional, containing `type`, `properties`, `required`).
session.turn_detection	object\|null	Optional	Turn detection configuration. Set to `null` to switch to push-to-talk mode (manually submit audio and manually trigger inference). If this field is omitted, the system enables VAD with default parameters.
session.turn_detection.type	string	Optional	VAD type. `server_vad` (default): detects the start and end of speech based on acoustic features and automatically triggers inference. `smart_turn`: intelligent turn detection that combines acoustic perception with semantic understanding — non-semantic sounds (such as "um" or "uh") do not trigger a conversation turn or interrupt model playback.
session.turn_detection.threshold	float	Optional	VAD sensitivity. Effective only in server_vad mode (has no effect in smart_turn mode). Lower values are more sensitive. Range: [-1.0, 1.0]. Default: 0.5.
session.turn_detection.silence_duration_ms	integer	Optional	The minimum silence duration (in milliseconds) required after speech ends before triggering a model response. Effective only in server_vad mode (has no effect in smart_turn mode). Range: [200, 6000]. Default: 800. For conversational scenarios, 400-800 is recommended.
session.turn_detection.voiceprint_audio_urls	array	Optional	Effective only in smart_turn mode. A list of publicly accessible URLs for the target speaker's pre-recorded audio, used for speaker enhancement. Supports up to 5 URLs. Required audio format: 16 kHz PCM or WAV. Can only be configured in the first `session.update` event; subsequent occurrences of this field are ignored.

Basic configuration example:

{
    "type": "session.update",
    "session": {
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "longanqian",
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "silence_duration_ms": 800
        }
    }
}

Function calling example:

{
    "type": "session.update",
    "session": {
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "longanqian",
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get the weather for a specified city",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "city": {
                                "type": "string",
                                "title": "City"
                            }
                        },
                        "required": ["city"]
                    }
                }
            }
        ],
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "silence_duration_ms": 800
        }
    }
}

Voiceprint enrollment (voiceprint_audio_urls) example:

{
    "type": "session.update",
    "session": {
        "turn_detection": {
            "type": "smart_turn",
            "voiceprint_audio_urls": [
                "https://example.com/speaker1.pcm",
                "https://example.com/speaker2.wav"
            ]
        }
    }
}

input_audio_buffer.append

Appends audio data to the input buffer. Send continuously at a high frequency (for example, one frame every 20-40 ms). This event has no server acknowledgment.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `input_audio_buffer.append`.
audio	string	Required	Base64-encoded audio data.

{
    "type": "input_audio_buffer.append",
    "audio": "<base64-encoded audio data>"
}

input_audio_buffer.commit

push-to-talk mode only. Commits the buffered audio as a user message. Committing does not automatically trigger inference — send response.create to trigger it manually. This event is ignored in server_vad / smart_turn mode.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `input_audio_buffer.commit`.

{
    "type": "input_audio_buffer.commit"
}

input_audio_buffer.clear

push-to-talk mode only. Clears any uncommitted audio in the buffer. Ignored in server_vad / smart_turn mode. The server responds with an input_audio_buffer.cleared event.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `input_audio_buffer.clear`.

{
    "type": "input_audio_buffer.clear"
}

conversation.item.create

Manually inserts a conversation item into the conversation context. Use this to inject historical context, add text information, or write back the result of a function call.

If item.id already exists in the conversation, the server returns an error and rejects the creation.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `conversation.item.create`.
previous_item_id	string	Optional	The item after which to insert the new item. If omitted, the item is appended to the end of the conversation.
item	object	Required	The conversation item to create.
item.id	string	Optional	The unique identifier of the conversation item. If omitted, the server generates one automatically. If the specified ID already exists in the conversation, the server returns an error.
item.type	string	Required	The conversation item type. `message`: a regular conversation message; `function_call`: a function call request (typically generated by the server, but the client can also use it to add historical context); `function_call_output`: a tool execution result (after the client receives a `function_call`, it executes the tool and writes back the result using this type).
item.role	string	Required for `message` type	The message role. Options: `system`, `user`, `assistant`.
item.content	array	Required for `message` type	The list of message content. Each element includes a `type` and its corresponding data field. `system` supports `input_text` (required field `text`); `user` supports `input_text` (required field `text`) and `input_audio` (required field `audio`, Base64-encoded); `assistant` supports `output_text` (required field `text`).
item.call_id	string	Required for `function_call` / `function_call_output` types	The unique identifier of the function call, used to associate the request with its result.
item.name	string	Required for `function_call` type	The name of the function to call.
item.arguments	string	Required for `function_call` type	The function call arguments, as a JSON string.
item.output	string	Required for `function_call_output` type	The tool execution result, as a JSON string.

Inject a user text message:

{
    "type": "conversation.item.create",
    "previous_item_id": "item_xxx",
    "item": {
        "id": "my_item_001",
        "type": "message",
        "role": "user",
        "content": [
            {
                "type": "input_text",
                "text": "Please summarize our last conversation"
            }
        ]
    }
}

Write back a function calling result:

{
    "type": "conversation.item.create",
    "item": {
        "type": "function_call_output",
        "call_id": "call_xxx",
        "output": "{\"temperature\":18,\"condition\":\"sunny\"}"
    }
}

conversation.item.retrieve

Retrieves a conversation item stored on the server. For audio-type content, the returned item includes only the transcript (transcript), not the original audio data.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `conversation.item.retrieve`.
item_id	string	Required	The ID of the conversation item to retrieve. The server returns the result via a `conversation.item.retrieved` event.

{
    "type": "conversation.item.retrieve",
    "item_id": "item_xxx"
}

conversation.item.delete

Deletes the specified conversation item from the conversation context. The server confirms with a conversation.item.deleted event.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `conversation.item.delete`.
item_id	string	Required	The ID of the conversation item to delete.

{
    "type": "conversation.item.delete",
    "item_id": "item_xxx"
}

response.create

Explicitly triggers a round of model inference. Behavior by mode:

push-to-talk mode: Must be called manually, after committing the buffered audio via input_audio_buffer.commit or writing a function_call_output. Not allowed while a response is already being generated.
server_vad mode: Usually triggered automatically by the server. The client can also call it manually when no response is currently being generated; not allowed while a response is already being generated.
smart_turn mode: Allowed while waiting for the user's next input. Not allowed within an active turn (from input_audio_buffer.speech_started to response.done).

The response field is optional and overrides the session's default configuration for this round of inference. In function calling scenarios, the client also uses this event to trigger the second round of inference after writing a function_call_output.

In server_vad / smart_turn mode, a manually triggered response can still be interrupted by new speech.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `response.create`.
response	object	Optional	Overrides the session's default configuration for this round of inference. If omitted, the current session configuration is used.
response.modalities	array	Optional	Overrides the output modalities for this round. Options are the same as `modalities` in `session.update`.
response.voice	string	Optional	Overrides the TTS voice for this round.

{
    "type": "response.create",
    "response": {
        "modalities": ["audio", "text"]
    }
}

response.cancel

Cancels the response currently being generated. Any text already produced is written to the item chain, and the server returns response.done with status=cancelled. Returns an error if no inference is currently in progress.

Field	Type	Required	Description
type	string	Required	Event type, fixed as `response.cancel`.

{
    "type": "response.cancel"
}

​session.update

​input_audio_buffer.append

​input_audio_buffer.commit

​input_audio_buffer.clear

​conversation.item.create

​conversation.item.retrieve

​conversation.item.delete

​response.create

​response.cancel

session.update

input_audio_buffer.append

input_audio_buffer.commit

input_audio_buffer.clear

conversation.item.create

conversation.item.retrieve

conversation.item.delete

response.create

response.cancel