Qwen-Omni client events

Events sent from the client to the server over WebSocket.

For non-realtime usage, Qwen-Omni is available through the Chat API.

session.update

Send this event after connecting to update the session configuration. The service validates your parameters and returns the full configuration or an error.

Example

{
  "event_id": "event_ToPZqeobitzUJnt3QqtWg",
  "type": "session.update",
  "session": {
  "modalities": ["text", "audio"],
  "voice": "Chelsie",
  "input_audio_format": "pcm16",
  "output_audio_format": "pcm24",
  "instructions": "You are an AI customer service agent for a five-star hotel. Please answer customer inquiries about room types, facilities, prices, and reservation policies accurately and in a friendly manner. Always respond with a professional and helpful attitude. Do not provide unconfirmed information or information beyond the scope of the hotel's services.",
  "turn_detection": {
      "type": "server_vad",
      "threshold": 0.5,
      "silence_duration_ms": 800
  },
  "seed": 1314,
  "max_tokens": 16384,
  "repetition_penalty": 1.05,
  "presence_penalty": 0.0,
  "top_k": 50,
  "top_p": 1.0,
  "temperature": 0.9
  }
}

string

body

required

Event type. Always session.update.

object

body

Session configuration.

Show properties

array

body

Output modalities. Valid values:

["text"] — Text only.
["text", "audio"] (Default) — Text and audio.

string

body

Voice for audio output. See Voice list for supported voices. Defaults:

Qwen3-Omni-Flash-Realtime: Cherry
Qwen-Omni-Turbo-Realtime: Chelsie

string

body

Input audio format. Supports pcm16 only.

string

body

Output audio format:

Qwen3-Omni-Flash-Realtime: pcm24 only
Qwen-Omni-Turbo-Realtime: pcm16 only

boolean | null

body

Qwen3-Omni-Flash-Realtime only. Controls reply style:

true (Default): Conversational tone.
false: Formal, written tone. May not suit complex content.
null: Model decides automatically.

string

body

System message defining the model's role or goal.

object

body

Voice activity detection (VAD) configuration. Set to null to disable VAD and trigger responses manually. Omit to use VAD with default parameters.

Show properties

string

body

Server-side VAD type. Always server_vad.

float

body

VAD sensitivity. Lower values detect more sounds (including background noise). Higher values require clearer speech. Range: [-1.0, 1.0]. Default: 0.5.

integer

body

Silence duration (ms) after speech ends before the model responds. Lower values give faster responses but may trigger on brief pauses. Range: 200-6000. Default: 800.

float

body

Sampling temperature for output diversity. Higher values increase diversity; lower values increase determinism. Range: [0, 2). Set only one of temperature or top_p. Defaults:

qwen3-omni-flash-realtime models: 0.9
qwen-omni-turbo-realtime models: 1.0

qwen-omni-turbo models do not support this parameter.

float

body

Nucleus sampling threshold for output diversity. Higher values increase diversity; lower values increase determinism. Range: (0, 1.0]. Set only one of temperature or top_p. Defaults:

qwen3-omni-flash-realtime models: 1.0
qwen-omni-turbo-realtime models: 0.01

qwen-omni-turbo models do not support this parameter.

integer

body

Candidate token count for sampling. Higher values increase randomness; lower values increase determinism. If null or >100, top_k is disabled and only top_p applies. Must be >=0. Defaults:

qwen3-omni-flash-realtime models: 50
qwen-omni-turbo-realtime models: 20

qwen-omni-turbo models do not support this parameter.

integer

body

Maximum tokens to return. Output is truncated beyond this limit; generation itself is not affected. Defaults and maximums match the model's max output length (see Model list). Use to control word count, costs, or latency. qwen-omni-turbo models do not support this parameter.

float

body

Penalty for consecutive repetition. Higher values reduce repetition. 1.0 = no penalty. Must be >0. Default: 1.05. qwen-omni-turbo models do not support this parameter.

float

body

Controls repetition. Range: [-2.0, 2.0]. Default: 0.0. Positive values reduce repetition; negative values increase it. Use higher values for creative tasks, lower values for formal content. qwen-omni-turbo models do not support this parameter.

integer

body

Makes output deterministic. With the same seed and parameters, the model returns identical results. Range: 0 to 2^31-1. Default: -1. qwen-omni-turbo models do not support this parameter.

response.create

Tells the service to generate a model response. In VAD mode, responses are automatic and you do not need this event. The service responds with response.created, then item and content events (conversation.item.created, response.content_part.added), and finally response.done.

Example

{
  "type": "response.create",
  "event_id": "event_1718624400000"
}

string

body

required

Event type. Always response.create.

response.cancel

Cancels an ongoing response. Returns an error if no response is in progress.

Example

{
  "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
  "type": "response.cancel"
}

string

body

required

Event type. Always response.cancel.

input_audio_buffer.append

Appends audio bytes to the input buffer.

Example

{
  "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
  "type": "input_audio_buffer.append",
  "audio": "UklGR..."
}

string

body

required

Event type. Always input_audio_buffer.append.

string

body

required

The Base64-encoded audio data.

input_audio_buffer.commit

Submits the input audio buffer as a user message. Returns an error if the buffer is empty.

VAD mode: Automatic. You do not need this event.
Manual mode: Required to create a user message.

Submitting the buffer does not trigger a model response. The service responds with input_audio_buffer.committed.

If you have sent an input_image_buffer.append event, input_audio_buffer.commit submits the image buffer along with the audio buffer.

Example

{
  "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
  "type": "input_audio_buffer.commit"
}

string

body

required

Event type. Always input_audio_buffer.commit.

input_audio_buffer.clear

Clears the audio buffer. The service responds with input_audio_buffer.cleared.

Example

{
  "event_id": "event_xxx",
  "type": "input_audio_buffer.clear"
}

string

body

required

Event type. Always input_audio_buffer.clear.

input_image_buffer.append

Adds image data to the image buffer from local files or video streams. Limits:

Format: JPG or JPEG. Recommended: 480p or 720p. Maximum: 1080p.
Size: ≤500 KB before Base64 encoding.
Encoding: Base64.
Frequency: 1 image per second.
Prerequisite: Send at least one input_audio_buffer.append event first.

The image buffer is submitted with the audio buffer through the input_audio_buffer.commit event.

Example

{
  "event_id": "event_xxx",
  "type": "input_image_buffer.append",
  "image": "xxx"
}

string

body

required

Event type. Always input_image_buffer.append.

string

body

required

The Base64-encoded image data.

​session.update

​response.create

​response.cancel

​input_audio_buffer.append

​input_audio_buffer.commit

​input_audio_buffer.clear

​input_image_buffer.append

session.update

response.create

response.cancel

input_audio_buffer.append

input_audio_buffer.commit

input_audio_buffer.clear

input_image_buffer.append