Skip to main content
Real-time voice conversation

Qwen-Audio client events

Client event reference for the Qwen-Audio Realtime API.

User guide: Real-time voice conversation (Qwen-Audio-Realtime). For the event interaction sequence, see WebSocket access.
This page is the event reference for Qwen-Audio-Realtime. Event definitions are based on WebSocket access. AOQ and WebRTC use their respective data channels to send and receive model events, and media tracks to transmit media. First read WebSocket access, WebRTC access, or AOQ access.
For WebSocket SDK usage, see Android SDK, iOS SDK, and HarmonyOS SDK. To select a model and protocol, see Overview.

session.update

Description: After a connection is established, send this event to update the default session configuration. Include only the fields you want to change; omitted fields retain their current values. If any parameter is invalid, the server returns an error. If all parameters are valid, the server applies the changes and returns the full configuration.
turn_detection can only be modified before the first audio is sent (IDLE state).
The examples below apply to qwen-audio-3.1-realtime-plus and qwen-audio-3.1-realtime-flash and explicitly select the system voice longanqian_v3.1.
{
    "type": "session.update",
    "session": {
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "longanqian_v3.1",
        "speech_rate": 1.2,
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "silence_duration_ms": 800
        }
    }
}
Language configuration (qwen-audio-3.1-realtime-plus / qwen-audio-3.1-realtime-flash) After connecting to qwen-audio-3.1-realtime-plus or qwen-audio-3.1-realtime-flash, you can send the following session.update event to set the source language of the input audio to Chinese and the target language of the output audio to English.
{
    "type": "session.update",
    "session": {
        "input_audio_transcription": {
            "language": "zh"
        },
        "output_audio": {
            "language": "en"
        }
    }
}
Function Calling:
{
    "type": "session.update",
    "session": {
        "modalities": [
            "text",
            "audio"
        ],
        "voice": "longanqian_v3.1",
        "tools": [
            {
                "type": "function",
                "function": {
                    "name": "get_weather",
                    "description": "Get weather for a specified city",
                    "parameters": {
                        "type": "object",
                        "properties": {
                            "city": {
                                "type": "string",
                                "title": "City"
                            }
                        },
                        "required": ["city"]
                    }
                }
            }
        ],
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "silence_duration_ms": 800
        }
    }
}
Voiceprint registration (voiceprint_audio_urls):
{
    "type": "session.update",
    "session": {
        "turn_detection": {
            "type": "smart_turn_v2",
            "voiceprint_audio_urls": [
                "https://example.com/speaker1.pcm",
                "https://example.com/speaker2.wav"
            ]
        }
    }
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: session.update.
sessionobject(optional) Session configuration.
session.modalitiesarray(optional) Output modalities for the model. Valid values:
  • ["text"]
Text only.
  • ["audio", "text"] (default)
Both text and audio.
session.voicestring(optional) TTS voice name. The default is longanqian_v3.1 for 3.1 Plus/Flash and longanqian for 3.0 Plus/Flash. System voices and cloned voices are supported. Can only be set in the first session.update; ignored in subsequent calls.
  • System voices for 3.1 Plus/Flash: Supports longanqian_v3.1 and the union of the voice lists for 3.1 TTS Plus and Flash. For the complete range, see Voice configuration.
  • System voices for 3.0 Plus/Flash: Available values: longanqian, longanlingxin, longanlingxi, longanxiaoxin, longanlufeng.
  • Cloned voices: Create a voice through the Voice cloning API, and set this parameter to the returned voice_id. For details, see Voice configuration.
After connecting to qwen-audio-3.1-realtime-plus or qwen-audio-3.1-realtime-flash, set the voice in the first session.update, for example:
session.speech_ratefloat(optional) Speaking rate of model audio output. Range: [0.5, 2.0]. Default: 1.0. Lower values slow the speech; higher values speed it up. Supported by qwen-audio-3.1-realtime-plus, qwen-audio-3.1-realtime-flash, qwen-audio-3.0-realtime-plus, qwen-audio-3.0-realtime-flash.
session.enable_speech_emotionboolean(optional) Specifies whether to enable speech emotion enhancement. When enabled, the response voice exhibits more pronounced emotional variations. Default: true. Valid values: true and false.
session.instructionsstring(optional) System instructions that define the model's role, response style, and behavioral preferences. Applies to the entire session.
session.input_audio_formatstring(optional) Input audio format. Currently only pcm (16 kHz, 16-bit, mono) is supported and is the default. Can only be modified before the first audio is sent (IDLE state).
session.output_audio_formatstring(optional) Output audio format. Currently only pcm (24 kHz, 16-bit, mono) is supported and is the default.
session.input_audio_transcriptionobject(optional) Input audio transcription configuration. Applies to qwen-audio-3.1-realtime-plus and qwen-audio-3.1-realtime-flash.
session.input_audio_transcription.languagestring(optional) The source language of the input audio. Valid values:
  • zh: Chinese
  • en: English
  • ja: Japanese
  • ko: Korean
  • vi: Vietnamese
  • th: Thai
  • id: Indonesian
  • ar: Arabic
  • fr: French
  • de: German
  • es: Spanish
  • pt: Portuguese
  • ru: Russian
  • it: Italian
session.output_audioobject(optional) Output audio configuration. Applies to qwen-audio-3.1-realtime-plus and qwen-audio-3.1-realtime-flash.
session.output_audio.languagestring(optional) The target language of the output audio. This parameter applies to speech synthesis (TTS). The model makes a best effort to produce speech in the target language, but this is not guaranteed, and the language of the response text is not changed. To generate text in the target language, state the requirement explicitly in the prompt, for example by specifying the reply language in instructions. Valid values:
  • zh: Chinese
  • en: English
  • fr: French
  • de: German
  • ja: Japanese
  • ko: Korean
  • ru: Russian
  • pt: Portuguese
  • th: Thai
  • id: Indonesian
  • vi: Vietnamese
  • es: Spanish
  • it: Italian
  • ar: Arabic
session.max_history_turnsinteger(optional) Maximum number of conversation turns (question-answer pairs) included in a single request. Valid values: 1 to 50. Default: 20.
session.enable_searchboolean(optional) Applies to the qwen-audio-3.1-realtime-plus, qwen-audio-3.1-realtime-flash, qwen-audio-3.0-realtime-plus, and qwen-audio-3.0-realtime-flash models. Specifies whether to enable web search. Default: false. When enabled, the model decides whether to search the web based on the real-time query. You can set enable_search to true and configure Function Calling tools through tools in the same session.
session.search_optionsobject(optional) The web search configuration. This parameter takes effect only when enable_search is set to true.
session.search_options.enable_sourceboolean(optional) Specifies whether to return the sources of search results. Set this parameter to true to return the sources.
session.toolsarray(optional) Tool definitions for Function Calling. Once configured, the model decides whether to call a tool based on user input.
session.tools.typestring(required) Fixed value: function.
session.tools.function.namestring(required) Name of the tool function.
session.tools.function.descriptionstring(optional) Description of the tool function. The model uses this to determine whether to call the tool.
session.tools.function.parametersobject(optional) Description of the tool function's input parameters. The model uses this to extract the required parameters. Omit this field if the function takes no parameters.
session.tools.function.parameters.typestring(required) Fixed value: object.
session.tools.function.parameters.propertiesobject(optional) Describes each parameter's name, data type, and description.
session.tools.function.parameters.requiredarray(optional) Specifies which parameters are required.
session.turn_detectionobject|null(optional) The turn detection fields described below apply to qwen-audio-3.1-realtime-plus, qwen-audio-3.1-realtime-flash, qwen-audio-3.0-realtime-plus, and qwen-audio-3.0-realtime-flash. Turn detection configuration. To switch to push-to-talk mode, set this field to null. In push-to-talk mode, audio must be committed manually and inference triggered manually. If this field isn't provided, VAD is enabled with its default parameters.
session.turn_detection.typestring(optional) VAD type. Valid values:
  • server_vad (default): Detects speech onset and offset based on acoustic features and automatically triggers inference.
  • smart_turn: An intelligent turn detection mode that combines acoustic perception and semantic understanding to determine turn boundaries. Filler sounds such as "um" or "uh" don't trigger a new turn or interrupt model playback.
  • smart_turn_v2 (recommended): Builds on smart_turn to better understand conversation context and interaction intent in multi-speaker settings. Reduces unintended responses or interruptions caused by bystanders or unrelated interjections.
session.turn_detection.thresholdfloat(optional) VAD sensitivity. Applies to server_vad, smart_turn, and smart_turn_v2. Lower values increase VAD sensitivity, making it easier to detect faint sounds (including background noise) as speech. Higher values decrease sensitivity, requiring clearer and louder speech to trigger detection. Range: [-1.0, 1.0]. Default: 0.5.
session.turn_detection.silence_duration_msinteger(optional) Minimum silence duration (in milliseconds) after speech ends before triggering a model response. Only effective in server_vad mode (ignored in smart_turn / smart_turn_v2 modes). Lower values produce faster responses but may cause false triggers during brief pauses. Range: [200, 6000]. Default: 800. Recommended range for conversations: 400-800.
session.turn_detection.voiceprint_audio_urlsarray(optional) Only effective in smart_turn / smart_turn_v2 modes. A list of publicly accessible URLs pointing to pre-recorded audio samples from the target user, used for speaker enhancement. Once registered, the model locks onto the target speaker during duplex conversations, effectively ignoring other voices and background noise. Maximum 5 URLs. Audio format requirements: 16kHz PCM or WAV. Tip: This parameter can only be configured in the first session.update event. It is ignored in subsequent session.update events.
{
    "type": "session.update",
    "session": {
        "voice": "longanqian_v3.1"
    }
}

input_audio_buffer.append

Description: For WebSocket access, appends audio data to the input buffer. Send data continuously at short intervals, such as one frame every 20~40 ms. The server does not acknowledge this event.
{
    "type": "input_audio_buffer.append",
    "audio": "<Base64-encoded audio data>"
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: input_audio_buffer.append.
audiostring(required) Base64-encoded audio data.

input_audio_buffer.commit

Description: Push-to-talk mode only. Commits the buffered audio as a user message. This doesn't automatically trigger inference. Send response.create to trigger inference manually. This event is ignored in server_vad, smart_turn, and smart_turn_v2 modes.
{
    "type": "input_audio_buffer.commit"
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: input_audio_buffer.commit.

input_audio_buffer.clear

Description: For WebSocket access, this event takes effect only in push-to-talk mode. It clears uncommitted audio from the buffer. This event is ignored in server_vad / smart_turn / smart_turn_v2 modes. The server responds with an input_audio_buffer.cleared event.
{
    "type": "input_audio_buffer.clear"
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: input_audio_buffer.clear.

conversation.item.create

Description: Inserts a conversation item into the conversation context. Use this event to inject historical context or add text content, or to return Function Calling results.
If item.id already exists in the conversation, an error is returned and the item isn't created.
Inject a user text message:
{
    "type": "conversation.item.create",
    "previous_item_id": "item_xxx",
    "item": {
        "id": "my_item_001",
        "type": "message",
        "role": "user",
        "content": [
            {
                "type": "input_text",
                "text": "Please summarize our last conversation"
            }
        ]
    }
}
Return a Function Calling result:
{
    "type": "conversation.item.create",
    "item": {
        "type": "function_call_output",
        "call_id": "call_xxx",
        "output": "{\"temperature\":18,\"condition\":\"sunny\"}"
    }
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: conversation.item.create.
previous_item_idstring(optional) Specifies the conversation item after which the new item is inserted. If not provided, the item is appended to the end of the conversation.
itemobject(required) The conversation item to create.
item.idstring(optional) Unique identifier for the conversation item. If not provided, the server generates one automatically. An error is returned if the specified ID already exists in the conversation.
item.typestring(required) Conversation item type. Valid values:
  • message: A regular conversation message.
  • function_call: A function call request. Typically generated by the server, but clients can also use it to inject historical context.
  • function_call_output: A tool execution result. After receiving a function_call, the client executes the tool and returns the result using this type.
item.rolestring(required for message type) Message role. Valid values: system, user, assistant.
item.contentarray(required for message type) List of message content elements. Each element contains a type field and corresponding data fields.
item.content.systeminput_text: System message. Required field: text. user
  • input_text: User text input. Required field: text.
  • input_audio: User audio input. Required field: audio (Base64-encoded).
item.content.assistantoutput_text: Assistant text output. Required field: text.
item.call_idstring(required for function_call / function_call_output type) Unique identifier for the function call, used to correlate requests and results.
item.namestring(required for function_call type) Name of the function to call.
item.argumentsstring(required for function_call type) Function call parameters in JSON string format.
item.outputstring(required for function_call_output type) Tool execution result in JSON string format.

conversation.item.retrieve

Description: Retrieves a conversation item stored on the server. Audio-type content in the response contains only the transcript (transcript), not the original audio data.
{
    "type": "conversation.item.retrieve",
    "item_id": "item_xxx"
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: conversation.item.retrieve.
item_idstring(required) ID of the conversation item to retrieve. The server returns the result in a conversation.item.retrieved event.

conversation.item.delete

Description: Deletes a conversation item from the conversation context. The server confirms the deletion with a conversation.item.deleted event.
{
    "type": "conversation.item.delete",
    "item_id": "item_xxx"
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: conversation.item.delete.
item_idstring(required) ID of the conversation item to delete.

response.create

Description: Triggers model inference. Behavior varies by mode:
  • Push-to-talk mode: Must be called manually. Commit the buffered audio with input_audio_buffer.commit first, or return a function_call_output result before triggering. Can't be called while a response is being generated.
  • server_vad mode: Typically triggered automatically by the server. Clients can also call it manually when no response is being generated. Can't be called while a response is being generated.
  • smart_turn / smart_turn_v2 modes: Can be called while waiting for the next user turn. Can't be called during an active turn (between input_audio_buffer.speech_started and response.done).
The optional response field overrides the session defaults for the current inference round. In Function Calling scenarios, after the client returns a function_call_output, this event triggers the second inference round.
In server_vad, smart_turn, and smart_turn_v2 modes, manually triggered inferences can still be interrupted by new speech.
{
    "type": "response.create",
    "response": {
        "modalities": ["audio", "text"]
    }
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: response.create.
responseobject(optional) Overrides the session defaults for the current inference round. If not provided, the current session configuration is used.
response.modalitiesarray(optional) Overrides the output modalities for the current round. Valid values are the same as session.update modalities.
response.voicestring(optional) Overrides the TTS voice for the current round.

response.cancel

Description: Cancels the current inference. Any text generated so far is saved to the item list. The server then returns a response.done event with status=cancelled. An error is returned if no inference is in progress.
{
    "type": "response.cancel"
}
ParameterTypeDescription
typestring(required) Event type. Fixed value: response.cancel.