This topic describes server events for the Qwen real-time speech and audiovisual translation API, including event types, parameters, and examples.
Reference: Real-time audio and video translation - Qwen
error
Error message returned by the server.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always error. |
| error | object | Detailed information about the error. |
| error.type | string | The error type. |
| error.code | string | The error code. |
| error.message | string | The error message. |
| error.param | string | The parameter that is related to the error, such as session.modalities. |
session.created
When a client connects, the server returns this event first with the default session configurations.
Applies to qwen3.8-livetranslate-flash-realtime.
qwen3.5-livetranslate-flash-realtime:
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always session.created. |
| session | object | The session configuration. |
| session.id | string | The unique identifier for the session. |
| session.object | string | The value is always realtime.session. |
| session.model | string | The model in use. |
| session.input_modalities | array | Applies to qwen3.8-livetranslate-flash-realtime. Input modalities, such as ["audio"]. |
| session.output_modalities | array | Applies to qwen3.8-livetranslate-flash-realtime. Output modalities: ["text"] or ["text", "audio"]. |
| session.audio | object | Applies to qwen3.8-livetranslate-flash-realtime. Audio input and output configuration. |
| session.audio.input.format | object | Input format. type defaults to pcm, and sample_rate defaults to 16000 (Hz). |
| session.audio.input.turn_detection | object | Turn detection configuration. Defaults: type is speaker_detection, threshold is 0.5, and silence_duration_ms is 1000. |
| session.audio.output.format | object | Output format. type defaults to pcm, and sample_rate defaults to 24000 (Hz). |
| session.audio.output.voice | string | Output voice. Defaults to Tina. |
| session.modalities | array | Applies to models such as qwen3.5-livetranslate-flash-realtime. The output modality settings for the model. |
| session.voice | string | Applies to models such as qwen3.5-livetranslate-flash-realtime. The voice for the audio generated by the model. |
| session.input_audio_format | string | Applies to models such as qwen3.5-livetranslate-flash-realtime. The format of the input audio. Defaults to pcm. |
| session.sample_rate | integer | Applies to models such as qwen3.5-livetranslate-flash-realtime. The sample rate of the input audio, in Hz. |
| session.output_audio_format | string | Applies to models such as qwen3.5-livetranslate-flash-realtime. The format of the output audio. Defaults to pcm. |
| session.turn_detection | object | Applies to models such as qwen3.5-livetranslate-flash-realtime. The VAD (Voice Activity Detection) configuration. |
| session.translation | object | (Optional) The translation configuration. |
| session.translation.language | string | (Optional) The target language for translation. |
| session.translation.corpus | object | (Optional) Custom terminology configuration for improving translation accuracy of domain-specific terms. |
| session.translation.corpus.phrases | object | (Optional) A mapping of source-language terms to target-language translations. For details, see Supported languages. |
session.updated
Returned after a successful session.update request. If the request fails, an error event is returned instead.
Applies to qwen3.8-livetranslate-flash-realtime.
qwen3.5-livetranslate-flash-realtime:
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always session.updated. |
| session | object | The session configuration. |
| session.id | string | The unique identifier for the session. |
| session.object | string | The value is always realtime.session. |
| session.model | string | The model in use. |
| session.input_modalities | array | Applies to qwen3.8-livetranslate-flash-realtime. Input modalities, such as ["audio"]. |
| session.output_modalities | array | Applies to qwen3.8-livetranslate-flash-realtime. Output modalities: ["text"] or ["text", "audio"]. |
| session.audio | object | Applies to qwen3.8-livetranslate-flash-realtime. Audio input and output configuration. |
| session.audio.input.format | object | Input format. type defaults to pcm, and sample_rate defaults to 16000 (Hz). |
| session.audio.input.turn_detection | object | Turn detection configuration. Defaults: type is speaker_detection, threshold is 0.5, and silence_duration_ms is 1000. |
| session.audio.output.format | object | Output format. type defaults to pcm, and sample_rate defaults to 24000 (Hz). |
| session.audio.output.voice | string | Output voice. Defaults to Tina. |
| session.modalities | array | Applies to models such as qwen3.5-livetranslate-flash-realtime. The output modality settings for the model. |
| session.voice | string | Applies to models such as qwen3.5-livetranslate-flash-realtime. The voice for the audio generated by the model. |
| session.sample_rate | integer | (Optional) Applies to models such as qwen3.5-livetranslate-flash-realtime. The sample rate of the input audio, in Hz. |
| session.input_audio_format | string | Applies to models such as qwen3.5-livetranslate-flash-realtime. The format of the input audio. The value is always pcm. |
| session.output_audio_format | string | Applies to models such as qwen3.5-livetranslate-flash-realtime. The format of the output audio. The value is always pcm. |
| session.input_audio_transcription | object | Applies to models such as qwen3.5-livetranslate-flash-realtime. The input audio transcription configuration. Returned only when the input_audio_transcription.model parameter is configured in the session. |
| session.input_audio_transcription.model | string | The speech recognition model. |
| session.input_audio_transcription.language | string | The configured speech recognition language. |
| session.turn_detection | object | Applies to models such as qwen3.5-livetranslate-flash-realtime. The VAD (Voice Activity Detection) configuration. This field is not returned in Manual mode (when the client sets this parameter to null in session.update). |
| session.turn_detection.type | string | The VAD type. The value is always server_vad. |
| session.turn_detection.threshold | float | The VAD detection sensitivity. |
| session.turn_detection.prefix_padding_ms | integer | The duration of audio (in milliseconds) retained before speech starts, to avoid losing the beginning of speech. |
| session.turn_detection.silence_duration_ms | integer | The minimum duration of silence (in milliseconds) required after speech ends to determine that speech has concluded. |
| session.turn_detection.create_response | boolean | Whether to automatically trigger a translation response after VAD detects that speech has ended. |
| session.turn_detection.interrupt_response | boolean | Whether to interrupt the current translation response being generated when VAD detects a new speech turn. |
| session.translation | object | (Optional) The translation configuration. |
| session.translation.language | string | (Optional) The target language for translation. |
| session.translation.corpus | object | (Optional) Custom terminology configuration for improving translation accuracy of domain-specific terms. |
| session.translation.corpus.phrases | object | (Optional) A mapping of source-language terms to target-language translations. |
| session.enable_voice_clone | boolean | Whether to enable voice cloning. |
| session.voice_clone_options | object | Voice cloning control parameters. Returned only when enable_voice_clone is true. |
| session.voice_clone_options.frequency | string | The voice cloning frequency. |
session.finished
Session is finished and all audio translations are complete.
This event is returned after the client sends session.finish. After receiving this event, the client must finish local playback of any remaining translated audio before actively disconnecting, if playback is needed.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always session.finished. |
response.created
Returned when the server generates a new model response.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always response.created. |
| response | object | The response object. |
| response.id | string | The unique identifier for the response. |
| response.conversation_id | string | The unique identifier for the current session. |
| response.object | string | The object type. For this event, the value is always realtime.response. |
| response.status | string | The response status. Valid values:
|
| response.modalities | array | Response modality. |
| response.voice | string | The voice of the generated audio. |
| response.output_audio_format | string | The format of the output audio. |
| response.output | array | This event is currently empty. |
response.done
Returned after response generation is complete. Contains all output items except raw audio data.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always response.done. |
| response | object | The response object. |
| response.id | string | The unique identifier for the response. |
| response.conversation_id | string | The unique identifier for the current session. |
| response.object | string | The object type. For this event, the value is always realtime.response. |
| response.status | string | The status of the response. |
| response.modalities | array | The modality of the response. |
| response.voice | string | The voice used for the audio generated by the model. |
| response.output_audio_format | string | The format of the output audio. |
| response.output | object | The output of the response. |
| response.output.id | string | The unique identifier for the response output. |
| response.output.type | string | The type of the output item. The value is currently always message. |
| response.output.object | string | The object type of the output item. The value is currently always realtime.item. |
| response.output.status | string | The status of the output item. |
| response.output.role | string | The role of the output item. |
| response.output.content | array | The content of the output item. |
| response.output.content.type | string | The type of the output content. The value is text for plain text output and audio when the output includes audio. |
| response.output.content.text | string | The text content of the output. |
| response.output.content.transcript | string | The text transcription of the audio content. |
| response.usage | object | Cumulative token usage for the session through the current response. Each response.done reports the current cumulative values. To total the session usage, use the last usage report rather than adding the reports together. |
| response.usage.total_tokens | integer | Total input and output tokens: the sum of input_tokens and output_tokens. |
| response.usage.input_tokens | integer | Input tokens. |
| response.usage.output_tokens | integer | Output tokens. |
| response.usage.input_tokens_details | object | Input token breakdown. |
| response.usage.input_tokens_details.text_tokens | integer | Text tokens in the model’s fixed system prompt. Not billed. |
| response.usage.input_tokens_details.audio_tokens | integer | Audio input tokens. |
| response.usage.input_tokens_details.video_tokens | integer | Image input tokens, billed as image input. |
| response.usage.output_tokens_details | object | Output token breakdown. |
| response.usage.output_tokens_details.text_tokens | integer | Text output tokens. |
| response.usage.output_tokens_details.audio_tokens | integer | Audio output tokens. |
response.text.delta
Applies to qwen3.8-livetranslate-flash-realtime. Returns a new translation fragment for text-only output.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | Unique event identifier. |
| type | string | response.text.delta. |
| response_id | string | Response ID. |
| item_id | string | Message ID. |
| output_index | integer | Output item index. |
| content_index | integer | Content part index. |
| delta | string | New text fragment. Append fragments in arrival order without replacing previously received text. |
response.text.text
Applies to models such as qwen3.5-livetranslate-flash-realtime. For qwen3.8-livetranslate-flash-realtime, handle response.text.delta instead.
Returned for text-only output when the model generates text incrementally.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | A unique identifier for the event. |
| type | string | The type of the event. The value is always response.text.text. |
| text | string | The incremental text that is returned. |
| response_id | string | The response ID. |
| item_id | string | A unique identifier for the message item. |
| output_index | integer | Currently, the value is always 0. |
| content_index | integer | Currently, the value is always 0. |
| stash | string | Temporary text generated by the model. Concatenate stash with the current text to form an interim result. The system updates text and stash continuously using response.text.text events until it receives a response.text.done event. The final complete text is then available in the text field. |
response.text.done
Returned when text generation finishes for text-only output, or if the response is interrupted, incomplete, or canceled.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always response.text.done. |
| response_id | string | The unique identifier for the response. |
| item_id | string | The unique identifier for the message item. |
| output_index | integer | The value is currently always 0. |
| content_index | integer | The value is currently always 0. |
| text | string | The complete text output from the model. |
response.audio.delta
For WebSocket access, the server returns this event when the output modalities include audio and the model incrementally generates new audio data.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | A unique identifier for the event. |
| type | string | The event type. The value is always response.audio.delta. |
| response_id | string | A unique identifier for the response. |
| item_id | string | A unique identifier for the message item. |
| output_index | integer | The value is always 0. |
| content_index | integer | The value is always 0. |
| delta | string | The incremental audio data that is output by the model. The data is Base64-encoded. |
response.audio.done
Returned when audio generation is complete. Also returned if the response is interrupted, incomplete, or canceled. Does not contain complete audio data.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. This is always response.audio.done. |
| response_id | string | The unique identifier for the response. |
| item_id | string | The unique identifier for the message item. |
| output_index | integer | The value is always 0. |
| content_index | integer | The value is always 0. |
input_audio_buffer.speech_started
Returned when the server-side VAD (Voice Activity Detection) detects that the user has started speaking.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always input_audio_buffer.speech_started. |
| audio_start_ms | integer | The time point (in milliseconds) when speech was detected to start, as an offset from the beginning of the audio stream. |
| item_id | string | The unique identifier for the associated message item. |
| speaker_id | integer | Applies to qwen3.8-livetranslate-flash-realtime. The speaker identifier returned when speaker_detection is enabled. Use this event's item_id to associate the speaker with the transcription, and the translation item's previous_item_id to associate it with the translation. |
input_audio_buffer.speech_stopped
Returned when the server-side VAD detects that the user has stopped speaking, indicating the end of the current speech input turn. Because the translation response is generated synchronously with the streaming speech input, the actual translation response may have already started during the speech input process, without needing to wait for this event.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always input_audio_buffer.speech_stopped. |
| audio_end_ms | integer | The time point (in milliseconds) when speech was detected to end, as an offset from the beginning of the audio stream. |
| item_id | string | The unique identifier for the associated message item. |
input_audio_buffer.committed
In Manual mode (turn_detection is null), after the client sends an input_audio_buffer.commit event, the server returns this event as confirmation and automatically starts generating the translation response.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always input_audio_buffer.committed. |
input_audio_buffer.cleared
After the client sends an input_audio_buffer.clear event, the server returns this event as confirmation that the uncommitted audio data in the buffer has been cleared.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always input_audio_buffer.cleared. |
conversation.item.created
Returned when a new message item is created in the conversation. This event is triggered in the following scenarios:
- When the server starts generating a translation response, it creates a corresponding assistant message item (at this point,
contentis an empty array, and the content is progressively populated with the streaming response). - In Manual mode, after the client sends an
input_audio_buffer.commitevent, the server additionally creates a message item corresponding to the user's input audio (contentcontains{"type": "input_audio"}).
item.id of the speech recognition result matches the previous_item_id in the translation result event. Use these values to associate and display the source text with its translation.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always conversation.item.created. |
| previous_item_id | string | The unique identifier of the previous message item. For a translation result event, this value matches the item.id of the speech recognition result for the same VAD segment. |
| item | object | The message item information. |
| item.id | string | The unique identifier for the message item. |
| item.type | string | The value is always message. |
| item.object | string | The value is always realtime.item. |
| item.status | string | The status of the message item. |
| item.role | string | The role of the message. Valid values: assistant or user. |
| item.content | array | The content of the message. When a response is first created, this is an empty array that is progressively populated with the streaming response. For user message items created by commit in Manual mode, it contains {"type": "input_audio"}. |
conversation.item.input_audio_transcription.delta
Applies to qwen3.8-livetranslate-flash-realtime. Returns source transcription increments for input audio.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | Unique event identifier. |
| type | string | conversation.item.input_audio_transcription.delta. |
| item_id | string | Message ID. |
| content_index | integer | Content part index. |
| delta | string | New text fragment. Append fragments in arrival order without replacing previously received text. |
conversation.item.input_audio_transcription.text
Applies to models such as qwen3.5-livetranslate-flash-realtime. For qwen3.8-livetranslate-flash-realtime, handle conversation.item.input_audio_transcription.delta instead.
When input_audio_transcription.model is configured, the server streams speech recognition results in the original source language.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always conversation.item.input_audio_transcription.text. |
| item_id | string | The unique identifier for the message item. |
| content_index | integer | The value is currently always 0. |
| text | string | The confirmed recognition text. |
| stash | string | The recognition text that is pending confirmation. This text may be corrected by subsequent events. |
| language | string | The detected source language. |
| emotion | string | The detected emotion of the recognized audio. Supported emotions are as follows:
|
conversation.item.input_audio_transcription.completed
When input_audio_transcription.model is configured, returns the final recognition result after speech recognition completes.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. This is always conversation.item.input_audio_transcription.completed. |
| item_id | string | The unique identifier for the message item. |
| content_index | integer | This is currently always 0. |
| transcript | string | The complete speech recognition result in the original source language. |
| language | string | The detected source language. |
| emotion | string | The detected emotion of the recognized audio. Supported emotions:
|
conversation.item.input_audio_transcription.failed
Returned when audio input was received but speech recognition failed. This event is handled separately from other error events, making it easier for the client to identify the specific associated item.
| Parameter | Type | Description |
|---|---|---|
| type | string | The event type. The value is always conversation.item.input_audio_transcription.failed. |
| item_id | string | The ID of the associated conversation item. |
| content_index | integer | The index of the content part that contains the audio. |
| error.code | string | The error code. |
| error.message | string | The error message. |
response.audio_transcript.delta
Applies to qwen3.8-livetranslate-flash-realtime. Returns a new translation fragment for text and audio output.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | Unique event identifier. |
| type | string | response.audio_transcript.delta. |
| response_id | string | Response ID. |
| item_id | string | Message ID. |
| output_index | integer | Output item index. |
| content_index | integer | Content part index. |
| delta | string | New text fragment. Append fragments in arrival order without replacing previously received text. |
response.audio_transcript.text
Applies to models such as qwen3.5-livetranslate-flash-realtime. For qwen3.8-livetranslate-flash-realtime, handle response.audio_transcript.delta instead.
Returned for audio output to display real-time translation.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The type of the event. The value is always response.audio_transcript.text. |
| response_id | string | The unique identifier for the response. |
| item_id | string | The unique identifier for the message item. |
| output_index | integer | Currently, the value is always 0. |
| content_index | integer | Currently, the value is always 0. |
| text | string | The confirmed translation text segment. |
| stash | string | Temporary translation text, concatenated with text to form interim results. The system updates text and stash continuously via response.audio_transcript.text events until a response.audio_transcript.done event is received. Then retrieve the final translation from transcript. |
response.audio_transcript.done
Returned when audio output is enabled and text generation finishes.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. This is always response.audio_transcript.done. |
| response_id | string | The unique identifier for the response. |
| item_id | string | The unique identifier for the message item. |
| output_index | integer | This is currently always 0. |
| content_index | integer | This is currently always 0. |
| transcript | string | The complete text. |
response.output_item.added
Returned when a new output item is added during response generation.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always response.output_item.added. |
| response_id | string | The unique identifier for the response. |
| output_index | integer | The value is currently always 0. |
| item | object | Information about the output item. |
| item.id | string | The unique identifier for the output item. |
| item.type | string | The value is always message. |
| item.object | string | The value is always realtime.item. |
| item.status | string | The status of the output item. |
| item.role | string | The role of the message. |
| item.content | string | The content of the message. |
response.output_item.done
Returned when an output item is completed.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. The value is always response.output_item.done. |
| response_id | string | The unique identifier for the response. |
| output_index | integer | The value is currently always 0. |
| item | object | Information about the output item. |
| item.id | string | The unique identifier for the output item. |
| item.object | string | The value is always realtime.item. |
| item.type | string | The value is always message. |
| item.status | string | The status of the output item. |
| item.role | string | The role of the message sender. |
| item.content | string | The content of the message. |
response.content_part.added
Returned when a new content part is added.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique ID of the event. |
| type | string | The type of the event. The value is always response.content_part.added. |
| response_id | string | The unique ID of the response. |
| item_id | string | The unique ID of the message item. |
| output_index | integer | The value is always 0. |
| content_index | integer | The value is always 0. |
| part | object | Outputs item information. |
| part.type | string | The type of the content part. |
| part.text | string | The text of the content part. |
response.content_part.done
Returned when a content part is completed.
| Parameter | Type | Description |
|---|---|---|
| event_id | string | The unique identifier for this event. |
| type | string | The event type. This is always response.content_part.done. |
| response_id | string | The unique identifier for the response. |
| item_id | string | The unique identifier for the message item. |
| output_index | integer | The value is always 0. |
| content_index | integer | The value is always 0. |
| part | object | Information about the content part. |
| part.type | string | The type of the content part. |
| part.text | string | The text of the content part. |