Skip to main content
QwenCloud home page
Docs
Realtime

LiveTranslate server events

WebSocket server reference

Server-side events for the qwen3.5-livetranslate-flash-realtime API.

error

An error from the server.
Example
{
  "event_id": "event_RoUu4T8yExPMI37GKwaOC",
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "invalid_value",
    "message": "Invalid modalities: ['audio']. Supported combinations are: ['text'] and ['audio', 'text'].",
    "param": "session.modalities"
  }
}
​
string
body
Event ID.
​
string
body
Always error.
​
object
body
Error details.

session.created

Sent when you connect. Contains the default session configuration.
Example
{
  "event_id": "event_QxBGpjBDmDDQQWDtrqBKB",
  "type": "session.created",
  "session": {
    "id": "sess_OozZ1vtbPt2muDflHODIH",
    "object": "realtime.session",
    "model": "qwen3.5-livetranslate-flash-realtime",
    "modalities": [
      "text",
      "audio"
    ],
    "voice": "Cherry",
    "input_audio_format": "pcm",
    "sample_rate": 16000,
    "output_audio_format": "pcm",
    "turn_detection": {
      "type": "server_vad",
      "threshold": 0.2,
      "silence_duration_ms": 1000
    },
    "translation": {
      "language": "en",
      "corpus": {
        "phrases": {
          "人工智能": "Artificial Intelligence",
          "机器学习": "Machine Learning"
        }
      }
    }
  }
}
​
string
body
Event ID.
​
string
body
Always session.created.
​
object
body
Session configuration.

session.updated

Sent after a successful session.update request. On error, the server returns an error event instead.
Example
{
  "event_id": "event_QxBGpjBDmDDQQWDtrqBKB",
  "type": "session.updated",
  "session": {
    "id": "sess_OozZ1vtbPt2muDflHODIH",
    "object": "realtime.session",
    "model": "qwen3.5-livetranslate-flash-realtime",
    "modalities": [
      "text",
      "audio"
    ],
    "voice": "Ethan",
    "sample_rate": 16000,
    "input_audio_format": "pcm",
    "output_audio_format": "pcm",
    "input_audio_transcription": {
      "model": "qwen3-asr-flash-realtime",
      "language": "zh"
    },
    "turn_detection": {
      "type": "server_vad",
      "threshold": 0.2,
      "prefix_padding_ms": 300,
      "silence_duration_ms": 1000,
      "create_response": true,
      "interrupt_response": true
    },
    "translation": {
      "language": "en",
      "corpus": {
        "phrases": {
          "人工智能": "Artificial Intelligence",
          "机器学习": "Machine Learning"
        }
      }
    },
    "enable_voice_clone": true,
    "voice_clone_options": {
      "frequency": "once"
    }
  }
}
​
string
body
Event ID.
​
string
body
Always session.updated.
​
object
body
Session configuration.

session.finished

Sent after all translations are complete. The server sends this only after you send session.finish. You can disconnect after you receive it.
Example
{
  "event_id": "event_xxx",
  "type": "session.finished"
}
​
string
body
Event ID.
​
string
body
Always session.finished.

response.created

Sent when the server starts a new response.
Example
{
  "event_id": "event_L8hHVI5jYis6BzAjnPWJh",
  "type": "response.created",
  "response": {
    "id": "resp_P79OOMs8LnrXVpiIHUCKR",
    "object": "realtime.response",
    "conversation_id": "conv_UFClXtYkRkFXrs48y8pmK",
    "status": "in_progress",
    "modalities": [
      "text",
      "audio"
    ],
    "voice": "Cherry",
    "output_audio_format": "pcm16",
    "output": []
  }
}
​
string
body
Event ID.
​
string
body
Always response.created.
​
object
body
Response object.

response.done

Sent when response generation finishes. The response object includes all output items except raw audio data.
Example
{
  "event_id": "event_CNea8oXNipVanSg2VIzkO",
  "type": "response.done",
  "response": {
    "id": "resp_TfhYTqej692vsGA2jNEtH",
    "object": "realtime.response",
    "conversation_id": "conv_ZtyLfKVm8XqLwYRlsuDih",
    "status": "completed",
    "modalities": [
      "text",
      "audio"
    ],
    "voice": "Cherry",
    "output_audio_format": "pcm16",
    "output": [
      {
        "id": "item_MKtkMwN9RtcyE9eJShyWy",
        "object": "realtime.item",
        "type": "message",
        "status": "completed",
        "role": "assistant",
        "content": [
          {
            "type": "audio",
            "transcript": "Hello? "
          }
        ]
      }
    ],
    "usage": {
      "total_tokens": 56,
      "input_tokens": 47,
      "output_tokens": 9,
      "input_tokens_details": {
        "text_tokens": 20,
        "audio_tokens": 27
      },
      "output_tokens_details": {
        "text_tokens": 2,
        "audio_tokens": 7
      }
    }
  }
}
​
string
body
Event ID.
​
string
body
Always response.done.
​
object
body
Response object.

response.text.text

Sent when the model generates text incrementally in text-only mode.
Example
{
  "event_id": "event_B1lIeyOXR7qJMEExbqtTG",
  "type": "response.text.text",
  "response_id": "resp_B1lIdtjF4Noqpn5NOjznj",
  "item_id": "item_B1lIdJsAJlJiFs8ztWpJt",
  "output_index": 0,
  "content_index": 0,
  "text": "How are"
}
​
string
body
Event ID.
​
string
body
Always response.text.text.
​
string
body
Incremental text chunk.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.

response.text.done

Sent when text-only output is complete.
Also sent if the response is interrupted, incomplete, or canceled.
Example
{
  "event_id": "event_B1lIeE2Nac33zn5V7h2mm",
  "type": "response.text.done",
  "response_id": "resp_B1lIdtjF4Noqpn5NOjznj",
  "item_id": "item_B1lIdJsAJlJiFs8ztWpJt",
  "output_index": 0,
  "content_index": 0,
  "text": "How can I assist you today?"
}
​
string
body
Event ID.
​
string
body
Always response.text.done.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.
​
string
body
Complete text output.

response.audio.delta

Sent when the model generates audio data incrementally.
Example
{
  "event_id": "event_B1osWMZBtrEQbiIwW0qHQ",
  "type": "response.audio.delta",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "item_id": "item_OFaPGtzfWCPyGzxnuEX9i",
  "output_index": 0,
  "content_index": 0,
  "delta": "UklGRnoGAABXQVZFZm10IBAAAAAB..."
}
​
string
body
Event ID.
​
string
body
Always response.audio.delta.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.
​
string
body
Base64-encoded audio chunk.

response.audio.done

Sent when audio generation is complete.
Also sent if the response is interrupted, incomplete, or canceled. This event does not contain the full audio data.
Example
{
  "event_id": "event_B1osWMWoDRYyITDyNYcBu",
  "type": "response.audio.done",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "item_id": "item_OFaPGtzfWCPyGzxnuEX9i",
  "output_index": 0,
  "content_index": 0
}
​
string
body
Event ID.
​
string
body
Always response.audio.done.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.

input_audio_buffer.speech_started

Returned when the server-side VAD (Voice Activity Detection) detects that the user has started speaking.
Example
{
  "event_id": "event_xxx",
  "type": "input_audio_buffer.speech_started",
  "audio_start_ms": 568,
  "item_id": "item_xxx"
}
​
string
body
The unique identifier for this event.
​
string
body
Always input_audio_buffer.speech_started.
​
integer
body
The time point (in milliseconds) when speech was detected to start, as an offset from the beginning of the audio stream.
​
string
body
The unique identifier for the associated message item.

input_audio_buffer.speech_stopped

Returned when the server-side VAD detects that the user has stopped speaking, indicating the end of the current speech input turn. Because the translation response is generated synchronously with the streaming speech input, the actual translation response may have already started during the speech input process, without needing to wait for this event.
Example
{
  "event_id": "event_xxx",
  "type": "input_audio_buffer.speech_stopped",
  "audio_end_ms": 3900,
  "item_id": "item_xxx"
}
​
string
body
The unique identifier for this event.
​
string
body
Always input_audio_buffer.speech_stopped.
​
integer
body
The time point (in milliseconds) when speech was detected to end, as an offset from the beginning of the audio stream.
​
string
body
The unique identifier for the associated message item.

input_audio_buffer.committed

In Manual mode (turn_detection is null), after the client sends an input_audio_buffer.commit event, the server returns this event as confirmation and automatically starts generating the translation response.
Example
{
  "event_id": "event_xxx",
  "type": "input_audio_buffer.committed"
}
​
string
body
The unique identifier for this event.
​
string
body
Always input_audio_buffer.committed.

input_audio_buffer.cleared

After the client sends an input_audio_buffer.clear event, the server returns this event as confirmation that the uncommitted audio data in the buffer has been cleared.
Example
{
  "event_id": "event_xxx",
  "type": "input_audio_buffer.cleared"
}
​
string
body
The unique identifier for this event.
​
string
body
Always input_audio_buffer.cleared.

conversation.item.created

Returned when a new message item is created in the conversation. This event is triggered in the following scenarios:
  • When the server starts generating a translation response, it creates a corresponding assistant message item (at this point, content is an empty array, and the content is progressively populated with the streaming response).
  • In Manual mode, after the client sends an input_audio_buffer.commit event, the server additionally creates a message item corresponding to the user's input audio (content contains {"type": "input_audio"}).
For the same VAD segment, the server creates separate message items for the speech recognition result and the translation result. The item.id of the speech recognition result matches the previous_item_id in the translation result event. Use these values to associate and display the source text with its translation.
Example
{
  "event_id": "event_xxx",
  "type": "conversation.item.created",
  "previous_item_id": "item_asr_xxx",
  "item": {
    "id": "item_translation_xxx",
    "object": "realtime.item",
    "type": "message",
    "status": "in_progress",
    "role": "assistant",
    "content": []
  }
}
​
string
body
The unique identifier for this event.
​
string
body
Always conversation.item.created.
​
string
body
The unique identifier of the previous message item. For a translation result event, this value matches the item.id of the speech recognition result for the same VAD segment.
​
object
body
The message item information.

conversation.item.input_audio_transcription.text

Streams speech recognition results in the source language. Requires input_audio_transcription.model to be set.
Example
{
  "event_id": "event_xxx",
  "type": "conversation.item.input_audio_transcription.text",
  "item_id": "item_xxx",
  "content_index": 0,
  "text": "",
  "stash": "The weather is really nice today",
  "language": "zh",
  "emotion": "neutral"
}
​
string
body
Event ID.
​
string
body
Always conversation.item.input_audio_transcription.text.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
string
body
Confirmed recognition text.
​
string
body
Pending recognition text. May be corrected by later events.
​
string
body
Detected source language.
​
string
body
The detected emotion of the recognized audio. Supported emotions are as follows:
  • surprised: Surprised
  • neutral: Neutral
  • happy: Happy
  • sad: Sad
  • disgusted: Disgusted
  • angry: Angry
  • fearful: Fearful

conversation.item.input_audio_transcription.completed

Sent when speech recognition finishes with the final result. Requires input_audio_transcription.model to be set.
Example
{
  "event_id": "event_xxx",
  "type": "conversation.item.input_audio_transcription.completed",
  "item_id": "item_xxx",
  "content_index": 0,
  "transcript": "What a beautiful day! Let's go for a walk in the park together.",
  "language": "en",
  "emotion": ""
}
​
string
body
Event ID.
​
string
body
Always conversation.item.input_audio_transcription.completed.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
string
body
Final recognition result in the source language.
​
string
body
Detected source language.
​
string
body
The detected emotion of the recognized audio. Supported emotions:
  • surprised: surprised
  • neutral: calm/neutral
  • happy: happy
  • sad: sad
  • disgusted: disgusted
  • angry: angry
  • fearful: fearful

conversation.item.input_audio_transcription.failed

Returned when audio input was received but speech recognition failed. This event is handled separately from other error events, making it easier for the client to identify the specific associated item.
Example
{
  "event_id": "event_xxx",
  "type": "conversation.item.input_audio_transcription.failed",
  "item_id": "item_xxx",
  "content_index": 0,
  "error": {
    "code": "xxx",
    "message": "xxx",
    "param": "xxx"
  }
}
​
string
body
Always conversation.item.input_audio_transcription.failed.
​
string
body
The ID of the associated conversation item.
​
integer
body
The index of the content part that contains the audio.
​
string
body
The error code.
​
string
body
The error message.

response.audio_transcript.text

Streams the translation text in real time when output includes audio.
Example
{
  "event_id": "event_xxx",
  "type": "response.audio_transcript.text",
  "response_id": "resp_xxx",
  "item_id": "item_xxx",
  "output_index": 0,
  "content_index": 0,
  "text": "Hello,",
  "stash": " who are you?"
}
​
string
body
Event ID.
​
string
body
Always response.audio_transcript.text.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.
​
string
body
Confirmed translation segment.
​
string
body
Temporary text appended to text to form a partial result. The server updates text and stash through response.audio_transcript.text events until response.audio_transcript.done, which contains the final translation in transcript.

response.audio_transcript.done

Sent when translation text generation is complete for audio output.
Example
{
  "event_id": "event_VN4Q4GJugLcc1S23viW8E",
  "type": "response.audio_transcript.done",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "item_id": "item_JvJauNH2CTXb1D9WV6pD4",
  "output_index": 0,
  "content_index": 0,
  "transcript": "How can I assist you today?"
}
​
string
body
Event ID.
​
string
body
Always response.audio_transcript.done.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.
​
string
body
Final translation text.

response.output_item.added

Sent when a new output item is created during response generation.
Example
{
  "event_id": "event_B4O5yPt3Gjnjy5eYH3plG",
  "type": "response.output_item.added",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "output_index": 0,
  "item": {
    "id": "item_OFaPGtzfWCPyGzxnuEX9i",
    "object": "realtime.item",
    "type": "message",
    "status": "in_progress",
    "role": "assistant",
    "content": []
  }
}
​
string
body
Event ID.
​
string
body
Always response.output_item.added.
​
string
body
Response ID.
​
integer
body
Always 0.
​
object
body
Output item.

response.output_item.done

Sent when an output item is complete.
Example
{
  "event_id": "event_XkiwbYTBC9Wcdwy6uYJ2G",
  "type": "response.output_item.done",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "output_index": 0,
  "item": {
    "id": "item_JvJauNH2CTXb1D9WV6pD4",
    "object": "realtime.item",
    "type": "message",
    "status": "completed",
    "role": "assistant",
    "content": [
      {
        "type": "audio",
        "text": "Hello, I am a large language model developed by Alibaba Cloud. My name is Qwen. How can I help you?"
      }
    ]
  }
}
​
string
body
Event ID.
​
string
body
Always response.output_item.done.
​
string
body
Response ID.
​
integer
body
Always 0.
​
object
body
Output item.

response.content_part.added

Sent when a new content part begins.
Example
{
  "event_id": "event_J2UixwYKZsXg7c9YXZetL",
  "type": "response.content_part.added",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "item_id": "item_OFaPGtzfWCPyGzxnuEX9i",
  "output_index": 0,
  "content_index": 0,
  "part": {
    "type": "audio",
    "text": ""
  }
}
​
string
body
Event ID.
​
string
body
Always response.content_part.added.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.
​
object
body
Content part.

response.content_part.done

Sent when a content part is complete.
Example
{
  "event_id": "event_VN4Q4GJugLcc1S23viW8E",
  "type": "response.content_part.done",
  "response_id": "resp_P79OOMs8LnrXVpiIHUCKR",
  "item_id": "item_JvJauNH2CTXb1D9WV6pD4",
  "output_index": 0,
  "content_index": 0,
  "part": {
    "type": "audio",
    "text": "Hello, I am a large language model developed by Alibaba Cloud. My name is Qwen. How can I help you?"
  }
}
​
string
body
Event ID.
​
string
body
Always response.content_part.done.
​
string
body
Response ID.
​
string
body
Message item ID.
​
integer
body
Always 0.
​
integer
body
Always 0.
​
object
body
Content part.
LiveTranslate server events - QwenCloud