Real-time voice conversation
Server events for Qwen-Audio Realtime API
Server event reference for the Qwen-Audio Realtime API. All server events include the event_id (auto-generated by the server) and type (event type) fields.
User guide : Real-time voice conversation (Qwen-Audio-Realtime) . For the event interaction sequence, see WebSocket access .
This page is the event reference for Qwen-Audio-Realtime. Event definitions are based on WebSocket access. AOQ and WebRTC use their respective data channels to send and receive model events, and media tracks to transmit media. First read WebSocket access , WebRTC access , or AOQ access .
For WebSocket SDK usage, see Android SDK , iOS SDK , and HarmonyOS SDK . To select a model and protocol, see Overview .
error
Description : Returned when a request fails or a service error occurs. Client errors (invalid_request_error) keep the connection open. Server errors (server_error) terminate the connection.
{
"event_id" : "event_xxx" ,
"type" : "error" ,
"error" : {
"type" : "invalid_request_error" ,
"code" : "invalid_value" ,
"message" : "Cannot create response while another response is in progress." ,
"param" : "response.create"
}
}
Parameter Type Description event_id string Unique identifier for this event. type string Event type. Always error. error object Error details. error.type string Error type, such as invalid_request_error (client error) or server_error (server error). error.code string Error code. error.message string Error message. error.param string The parameter associated with the error.
session.created
Description : The first event sent by the server after the connection is established. Contains the default session configuration.
{
"event_id" : "event_KiKZC2zrhNsKFPZ5cTpyA" ,
"type" : "session.created" ,
"session" : {
"object" : "realtime.session" ,
"model" : "qwen-audio-3.1-realtime-plus" ,
"modalities" : [ "text" , "audio" ],
"voice" : "longanqian_v3.1" ,
"input_audio_transcription" : {
"model" : "fun-asr"
},
"turn_detection" : {
"type" : "server_vad" ,
"threshold" : 0.5 ,
"silence_duration_ms" : 800
},
"id" : "sess_A1LbG2D63WELBSawRbpq8"
}
}
Parameter Type Description type string Event type. Always session.created. session object Session configuration. session.object string Always realtime.session. session.model string Model name. session.modalities array Output modalities of the model. session.voice string Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice. session.input_audio_transcription object Audio transcription configuration. session.input_audio_transcription.model string Transcription model, such as fun-asr. session.turn_detection object Turn detection (VAD) configuration. session.id string Unique session identifier.
session.updated
Description : Returned after a session.update request is processed successfully. Contains the full updated session configuration. If the request fails, an error event is returned instead.
{
"event_id" : "event_FMG6kiHbILCGiqXFPA98e" ,
"type" : "session.updated" ,
"session" : {
"id" : "sess_A1LbG2D63WELBSawRbpq8" ,
"object" : "realtime.session" ,
"model" : "pre-fun-realtime-audiochat-beta-3" ,
"modalities" : [ "text" , "audio" ],
"voice" : "longanqian" ,
"input_audio_transcription" : {
"model" : "fun-asr"
},
"turn_detection" : {
"type" : "smart_turn" ,
"threshold" : 0.1 ,
"silence_duration_ms" : 900
}
}
}
Parameter Type Description type string Event type. Always session.updated. session object Full session configuration after the update. Shares the same structure as the session object in session.created.
Description : Sent when Voice Activity Detection (VAD) detects the start of speech in server_vad / smart_turn / smart_turn_v2 modes.
{
"event_id" : "event_xxx" ,
"type" : "input_audio_buffer.speech_started" ,
"audio_start_ms" : 1200 ,
"item_id" : "item_xxx"
}
Parameter Type Description type string Event type. Always input_audio_buffer.speech_started. audio_start_ms integer Timestamp in milliseconds when speech started. item_id string ID of the item that will be created when this speech segment is committed.
Description : Sent when VAD detects the end of speech in server_vad / smart_turn / smart_turn_v2 modes.
{
"event_id" : "event_xxx" ,
"type" : "input_audio_buffer.speech_stopped" ,
"audio_end_ms" : 3400 ,
"item_id" : "item_xxx" ,
"reason" : "turn_invalid"
}
Parameter Type Description type string Event type. Always input_audio_buffer.speech_stopped. audio_end_ms integer Timestamp in milliseconds when speech ended. item_id string ID of the user message item to be created. reason string Returned only in smart_turn / smart_turn_v2 modes. When set to turn_invalid, indicates the current turn was classified as invalid (no semantic content) and inference will not be triggered. This field is absent for valid turns.
Description : The audio buffer has been committed as a user message, either through a push-to-talk commit or automatic VAD commit.
{
"event_id" : "event_xxx" ,
"type" : "input_audio_buffer.committed" ,
"previous_item_id" : "item_xxx" ,
"item_id" : "item_xxx"
}
Parameter Type Description type string Event type. Always input_audio_buffer.committed. previous_item_id string ID of the previous conversation item. item_id string ID of the created user message item.
Description : The audio buffer has been cleared (push-to-talk mode only).
{
"event_id" : "event_xxx" ,
"type" : "input_audio_buffer.cleared"
}
Parameter Type Description type string Event type. Always input_audio_buffer.cleared.
conversation.item.created
Description : A new conversation item was created. Triggered when user audio is committed, the client manually creates an item, or an assistant response begins.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.created" ,
"previous_item_id" : "item_xxx" ,
"item" : {
"id" : "item_xxx" ,
"object" : "realtime.item" ,
"type" : "message" ,
"status" : "in_progress" ,
"role" : "assistant" ,
"content" : []
}
}
Parameter Type Description type string Event type. Always conversation.item.created. item object The created conversation item. item.id string Unique identifier for the conversation item. item.object string Always realtime.item. item.type string Item type: message (standard message) or function_call (function call). item.status string Item status, such as in_progress or completed. item.role string Message role, such as user or assistant. Only present for message type items. item.content array Message content list. Only present for message type items.
conversation.item.deleted
Description : A conversation item was deleted. Returned as a confirmation after the client sends a conversation.item.delete event.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.deleted" ,
"item_id" : "item_xxx"
}
Parameter Type Description type string Event type. Always conversation.item.deleted. item_id string ID of the deleted conversation item.
conversation.item.retrieved
Description : A conversation item was retrieved successfully. Returned after the client sends a conversation.item.retrieve event. For audio-type content, only the transcript text is included; raw audio data isn't returned.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.retrieved" ,
"item" : {
"id" : "item_xxx" ,
"object" : "realtime.item" ,
"type" : "message" ,
"role" : "user" ,
"content" : [
{
"type" : "input_audio" ,
"transcript" : "Hello"
}
]
}
}
Parameter Type Description type string Event type. Always conversation.item.retrieved. item object The retrieved conversation item. item.content array Message content list. For audio-type content, only the transcript text is included; raw audio data isn't returned.
Description : Incremental Automatic Speech Recognition (ASR) transcription result, streamed during speech recognition. Includes emotion and language detection information.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.input_audio_transcription.delta" ,
"item_id" : "item_xxx" ,
"content_index" : 0 ,
"text" : "Hello" ,
"stash" : "world"
}
Parameter Type Description type string Event type. Always conversation.item.input_audio_transcription.delta. item_id string ID of the associated conversation item. content_index integer Index of the content part. text string Finalized transcription text. stash string Tentative text that hasn't been finalized yet.
Description : Final ASR transcription result. The transcribed text is written to the transcript field of the corresponding item.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.input_audio_transcription.completed" ,
"item_id" : "item_xxx" ,
"content_index" : 0 ,
"transcript" : "Hello world"
}
Parameter Type Description type string Event type. Always conversation.item.input_audio_transcription.completed. transcript string The complete transcription text.
Description : ASR transcription failed.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.input_audio_transcription.failed" ,
"item_id" : "item_xxx" ,
"content_index" : 0 ,
"error" : {
"type" : "transcription_error" ,
"code" : "transcription_failed" ,
"message" : "ASR transcription failed"
}
}
Parameter Type Description type string Event type. Always conversation.item.input_audio_transcription.failed. error.type string Error type, such as transcription_error. error.code string Error code, such as transcription_failed.
conversation.item.ambient_audio_transcription.delta
Description : smart_turn / smart_turn_v2 modes only. Incremental transcription result for ambient audio. When VAD detects voice activity but semantic analysis classifies the turn as invalid (for example, background noise or filler words like "hmm" or "uh"), the ASR result is forwarded to the client as an ambient event. This event isn't linked to any conversation item. The item_id is a standalone temporary ID.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.ambient_audio_transcription.delta" ,
"item_id" : "item_xxx" ,
"content_index" : 0 ,
"text" : "Hmm" ,
"stash" : ""
}
Parameter Type Description type string Event type. Always conversation.item.ambient_audio_transcription.delta. item_id string A standalone temporary ID, not associated with any conversation item.
conversation.item.ambient_audio_transcription.completed
Description : smart_turn / smart_turn_v2 modes only. Final transcription result for ambient audio. Paired with the corresponding delta event to mark the end of an ambient audio transcription segment. This result isn't written to the conversation context.
{
"event_id" : "event_xxx" ,
"type" : "conversation.item.ambient_audio_transcription.completed" ,
"item_id" : "item_xxx" ,
"content_index" : 0 ,
"transcript" : "Hmm" ,
}
Parameter Type Description type string Event type. Always conversation.item.ambient_audio_transcription.completed. item_id string A standalone temporary ID, not associated with the conversation context.
response.created
Description : Sent when a model inference round starts.
{
"event_id" : "event_xxx" ,
"type" : "response.created" ,
"response" : {
"id" : "resp_xxx" ,
"object" : "realtime.response" ,
"status" : "in_progress" ,
"modalities" : [ "text" , "audio" ],
"voice" : "longanqian" ,
"output" : []
}
}
Parameter Type Description type string Event type. Always response.created. response object The response object. response.id string Unique identifier for the response. response.object string Always realtime.response. response.status string Response status, such as in_progress. response.modalities array Output modalities of the model. response.voice string Voice used for audio generation. Either a system voice name or the voice_id of a cloned voice. response.output array Output items of the response. Initially an empty array.
response.output_item.added
Description : A new output item was added to the response. The output item type is message for standard replies, or function_call for Function Calling.
{
"event_id" : "event_xxx" ,
"type" : "response.output_item.added" ,
"response_id" : "resp_xxx" ,
"output_index" : 0 ,
"item" : {
"id" : "item_xxx" ,
"object" : "realtime.item" ,
"type" : "message" ,
"status" : "in_progress" ,
"role" : "assistant" ,
"content" : []
}
}
Function Calling output item example: When the output item is a function call, the response.output_item.added / conversation.item.created / response.output_item.done events contain the following item structure:
{
"id" : "item_xxx" ,
"object" : "realtime.item" ,
"type" : "function_call" ,
"status" : "completed" ,
"call_id" : "call_xxx" ,
"name" : "get_weather" ,
"arguments" : "{ \" city \" : \" Hangzhou \" }"
}
A single response can include multiple function_call items and may also contain standard message output alongside function_call output. The Function Calling portion isn't sent to TTS for audio playback.
Parameter Type Description type string Event type. Always response.output_item.added. response_id string Associated response ID. output_index integer Index of the output item within the response. item object The added output item. item.id string Unique identifier for the output item. item.type string Output item type: message (standard message) or function_call (function call). item.status string Output item status, such as in_progress. item.role string Message role. Always assistant. Only present for message type items.
response.content_part.added
Description : A new content part was added to an output item.
{
"event_id" : "event_xxx" ,
"type" : "response.content_part.added" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"part" : {
"type" : "audio" ,
"text" : ""
}
}
Parameter Type Description type string Event type. Always response.content_part.added. item_id string Associated output item ID. content_index integer Index of the content part within the output item. part object The added content part. part.type string Content type, such as audio or text. part.text string Text content. Initially an empty string.
response.text.delta
Description : Text delta event in text-only mode. Streams text fragments incrementally.
{
"event_id" : "event_xxx" ,
"type" : "response.text.delta" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"delta" : "Hello"
}
Parameter Type Description type string Event type. Always response.text.delta. delta string Incremental text fragment.
response.text.done
Description : Text output complete event in text-only mode.
{
"event_id" : "event_xxx" ,
"type" : "response.text.done" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"text" : "Hello, how can I help you?"
}
Parameter Type Description type string Event type. Always response.text.done. text string The complete text output.
response.audio_transcript.delta
Description : Transcript delta event in audio mode. Streams transcript fragments incrementally.
{
"event_id" : "event_xxx" ,
"type" : "response.audio_transcript.delta" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"delta" : "Hello"
}
Parameter Type Description type string Event type. Always response.audio_transcript.delta. delta string Incremental transcript fragment.
response.audio_transcript.done
Description : Transcript output complete event in audio mode.
{
"event_id" : "event_xxx" ,
"type" : "response.audio_transcript.done" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"transcript" : "Hello, how can I help you?"
}
Parameter Type Description type string Event type. Always response.audio_transcript.done. transcript string The complete transcript text.
response.audio.delta
Description : For WebSocket access, this is an incremental audio data event in audio mode. The delta field contains Base64-encoded PCM audio data.
{
"event_id" : "event_xxx" ,
"type" : "response.audio.delta" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"delta" : "<base64-encoded audio data>"
}
Parameter Type Description type string Event type. Always response.audio.delta. delta string Base64-encoded PCM audio data fragment.
response.audio.done
Description : Audio output complete event in audio mode. Contains no audio data.
{
"event_id" : "event_xxx" ,
"type" : "response.audio.done" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0
}
Parameter Type Description type string Event type. Always response.audio.done.
response.content_part.done
Description : A content part within an output item has completed.
{
"event_id" : "event_xxx" ,
"type" : "response.content_part.done" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"content_index" : 0 ,
"part" : {
"type" : "audio" ,
"text" : "Hello, how can I help you?"
}
}
Parameter Type Description type string Event type. Always response.content_part.done. part object The completed content part. part.text string Text content or audio transcript text.
response.output_item.done
Description : An output item within the response has completed.
{
"event_id" : "event_xxx" ,
"type" : "response.output_item.done" ,
"response_id" : "resp_xxx" ,
"output_index" : 0 ,
"item" : {
"id" : "item_xxx" ,
"object" : "realtime.item" ,
"type" : "message" ,
"status" : "completed" ,
"role" : "assistant" ,
"content" : [
{
"type" : "text" ,
"text" : "Hello, how can I help you?"
}
]
}
}
Parameter Type Description type string Event type. Always response.output_item.done. item object The completed output item with full details. item.status string Output item status, such as completed.
response.function_call_arguments.delta
Description : Function Calling argument delta. When the model decides to invoke a tool, the server first sends response.output_item.added (with item.type=function_call) and conversation.item.created, then streams argument fragments through this event.
{
"event_id" : "event_xxx" ,
"type" : "response.function_call_arguments.delta" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"call_id" : "call_xxx" ,
"delta" : "{ \" city"
}
Parameter Type Description type string Event type. Always response.function_call_arguments.delta. call_id string Unique identifier for the function call. delta string Incremental fragment of the function call arguments (JSON string fragment).
response.function_call_arguments.done
Description : Sent when Function Calling argument output is complete. After receiving this event, the client should: execute the corresponding tool, write the result to the conversation via conversation.item.create with a function_call_output item, and then send response.create to trigger a follow-up inference round.
{
"event_id" : "event_xxx" ,
"type" : "response.function_call_arguments.done" ,
"response_id" : "resp_xxx" ,
"item_id" : "item_xxx" ,
"output_index" : 0 ,
"call_id" : "call_xxx" ,
"name" : "get_weather" ,
"arguments" : "{ \" city \" : \" Hangzhou \" }"
}
Parameter Type Description type string Event type. Always response.function_call_arguments.done. name string Name of the invoked function. arguments string Complete function call arguments (JSON string).
response.done
Description : An inference round is complete. status indicates the reason for completion.
Completed normally
Interrupted and cancelled
{
"event_id" : "event_xxx" ,
"type" : "response.done" ,
"response" : {
"id" : "resp_xxx" ,
"object" : "realtime.response" ,
"status" : "completed" ,
"modalities" : [ "text" , "audio" ],
"voice" : "longanqian" ,
"output" : [
{
"id" : "item_xxx" ,
"object" : "realtime.item" ,
"type" : "message" ,
"status" : "completed" ,
"role" : "assistant" ,
"content" : [
{
"type" : "audio" ,
"transcript" : "Hello, how can I help you?"
}
]
}
]
}
}
Parameter Type Description type string Event type. Always response.done. response object The complete response object. response.id string Unique identifier of the response. response.status string Completion status of the response. Valid values: completed: Completed normally.cancelled: Interrupted and cancelled. status_details.reason is turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation).failed: LLM or TTS error. response.status_details object Status details. Present only when the status is cancelled or failed. response.status_details.type string Status type, such as cancelled. response.status_details.reason string Cancellation reason: turn_detected (VAD interruption) or client_cancelled (client-initiated cancellation). response.output array List of output items in the response, containing complete item objects.
voiceprint_audio_list.in_progress
Description : The voiceprint registration process is in progress asynchronously.
{
"event_id" : "event_PaEcN7CCrlhE8q4MM2yND" ,
"type" : "voiceprint_audio_list.in_progress" ,
"item_id" : "vp_Y12cA986j1KZ9O9YmAXOA"
}
Parameter Type Description type string Event type. Always voiceprint_audio_list.in_progress. item_id string Unique identifier for the voiceprint registration task.
voiceprint_audio_list.completed
Description : The voiceprint registration process has completed successfully.
{
"event_id" : "event_PaEcN7CCrlhE8q4MM2yND" ,
"type" : "voiceprint_audio_list.completed" ,
"item_id" : "vp_Y12cA986j1KZ9O9YmAXOA"
}
Parameter Type Description type string Event type. Always voiceprint_audio_list.completed.
voiceprint_audio_list.failed
Description : The voiceprint registration failed. Normal conversation calls are not blocked.
{
"event_id" : "event_PaEcN7CCrlhE8q4MM2yND" ,
"type" : "voiceprint_audio_list.failed" ,
"item_id" : "vp_Y12cA986j1KZ9O9YmAXOA" ,
"reason" : ""
}
Parameter Type Description type string Event type. Always voiceprint_audio_list.failed. reason string Description of the failure reason.
Qwen-Audio-Realtime Voice Configuration
After creating a cloned voice, use it in a Qwen-Audio-Realtime session and consult the corresponding HTTP API.