Qwen-Omni client events

Events sent from the client to the server over WebSocket.

For non-realtime usage, Qwen-Omni is available through the Chat API.

session.update

Send this event after connecting to update the session configuration. The service validates your parameters and returns the updated configuration (excluding MCP endpoint and credential fields for Qwen3.8-Omni-Flash-Realtime), or an error.

Example

{
  "event_id": "event_ToPZqeobitzUJnt3QqtWg",
  "type": "session.update",
  "session": {
  "modalities": ["text", "audio"],
  "voice": "Chelsie",
  "input_audio_format": "pcm",
  "output_audio_format": "pcm",
  "instructions": "You are an AI customer service agent for a five-star hotel. Please answer customer inquiries about room types, facilities, prices, and reservation policies accurately and in a friendly manner. Always respond with a professional and helpful attitude. Do not provide unconfirmed information or information beyond the scope of the hotel's services.",
  "turn_detection": {
      "type": "server_vad",
      "threshold": 0.5,
      "silence_duration_ms": 800
  },
  "enable_search": true,
  "search_options": {
      "enable_source": true
  },
  "tools": [
      {
      "type": "function",
      "function": {
          "name": "get_current_weather",
          "description": "Useful for querying the weather in a specific city.",
          "parameters": {
          "type": "object",
          "properties": {
              "location": {
              "type": "string",
              "description": "The city or district, such as Beijing, Hangzhou, or Yuhang District."
              }
          },
          "required": ["location"]
          }
      }
      }
  ],
  "seed": 1314,
  "max_tokens": 16384,
  "repetition_penalty": 1.05,
  "presence_penalty": 0.0,
  "top_k": 50,
  "top_p": 1.0,
  "temperature": 0.9
  }
}

string

body

required

Event type. Always session.update.

object

body

Session configuration.For Qwen3.8-Omni-Flash-Realtime, session is required when updating a session. The total input token limit is 196608, calculated using the same rules as Qwen3.5-Omni-Realtime.

Show properties

array

body

Output modalities. Valid values:

["text"] — Text only.
["text", "audio"] (Default) — Text and audio.

string

body

Voice for audio output. Accepts preset voices or cloned voices (Qwen3.5-Omni-Realtime only) created via the Voice cloning API. See Voice list for supported preset voices. Defaults:

Qwen3.5-Omni-Realtime: Tina
Qwen3-Omni-Flash-Realtime: Cherry
Qwen-Omni-Turbo-Realtime: Chelsie

Qwen3.8-Omni-Flash-Realtime defaults to Tina. See Output voice for nested-field precedence and examples.

object

body

Input and output audio configuration. See Audio configuration.

string

body

The format of the user's input audio. Currently only supports pcm. The input audio must be a PCM audio stream at a 16 kHz sample rate.

string

body

The format of the output audio. Currently only supports pcm. The output audio is a PCM audio stream at a 24 kHz sample rate. Custom output sample rates are not supported.

boolean | null

body

Applies to Qwen3-Omni-Flash-Realtime series models only. Whether to enable a colloquial response style. Valid values:

true (default): Colloquial responses.
false: Formal, written responses. Content that is difficult to read aloud may not perform well.
null: The model automatically selects between colloquial and formal style.

string

body

System message defining the model's role or goal.

object

body

Voice activity detection (VAD) configuration. Set to null to disable VAD and trigger responses manually. Omit to use VAD with default parameters.

Show properties

string

body

VAD type. Valid values:

server_vad (Default): Detects the end of user speech based on acoustic features.
semantic_vad: Detects the end of user speech based on semantic validity. Filters out meaningless speech such as backchannels and background noise. Supported by Qwen3.8-Omni-Flash-Realtime and the Qwen3.5-Omni-Realtime series.

float

body

VAD sensitivity. Lower values detect more sounds (including background noise). Higher values require clearer speech. Range: [-1.0, 1.0]. Default: 0.5.

integer

body

Silence duration (ms) after speech ends before the model responds. Lower values give faster responses but may trigger on brief pauses. Range: 200-6000. Default: 800.

integer

body

Applies only to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime models in server_vad mode. The idle timeout in milliseconds. After the server finishes audio playback and the user remains silent beyond this duration (no speech.started triggered), the model proactively generates a response to prompt the user to continue the conversation based on the current context. The timeout starts after the last model response audio finishes playing. Range: 5000-30000.

boolean

body

Applies to Qwen3.8-Omni-Flash-Realtime and Qwen3.5-Omni-Realtime models. Whether to enable web search. Default: false. When enabled, the model can autonomously determine whether a search is needed to answer the user's real-time questions.

tools and enable_search are incompatible. Do not enable both at the same time.

object

body

Web search option configuration. Takes effect only when enable_search is enabled.

Show properties

boolean

body

Whether to return a list of search result sources. Set to true to enable.

object

body

Video input configuration. Use video.input.representation_compact for video aggregation. See Video configuration.

array

body

A list of tool definitions. When configured, the model can decide whether to call a tool based on user input.The properties below describe custom Function Calling tools. For MCP tool configuration in Qwen3.8-Omni-Flash-Realtime, see Qwen3.8 client events.

Show properties

string

body

required

For a custom Function Calling tool, the value must be function.

string

body

required

The name of the custom tool function, such as get_current_weather or get_current_time.

string

body

A description of the tool function's capabilities. The model uses this field to decide whether to use the tool function.

object

body

A description of the tool function's input parameters. The model uses this field to extract the input parameters. If the tool function does not require input parameters, do not specify this field.

Show properties

string

body

required

The value must be object.

object

body

Describes the name, data type, and description of each input parameter. The key is the parameter name. The value is an object that contains type and description.

array

body

Specifies which input parameters are required.

float

body

Sampling temperature for output diversity. Higher values increase diversity; lower values increase determinism. Range: [0, 2). Set only one of temperature or top_p. Defaults:

qwen3.8-omni-flash-realtime models: 0.6
qwen3.5-omni-realtime models: 0.7
qwen3-omni-flash-realtime models: 0.9
qwen-omni-turbo-realtime models: 1.0

qwen-omni-turbo models do not support this parameter.

float

body

Nucleus sampling threshold for output diversity. Higher values increase diversity; lower values increase determinism. Range: (0, 1.0]. Set only one of temperature or top_p. Defaults:

qwen3.8-omni-flash-realtime models: 0.95
qwen3.5-omni-realtime models: 0.8
qwen3-omni-flash-realtime models: 1.0
qwen-omni-turbo-realtime models: 0.01

qwen-omni-turbo models do not support this parameter.

integer

body

Candidate token count for sampling. Higher values increase randomness; lower values increase determinism. If null or >100, top_k is disabled and only top_p applies. Must be >=0. Defaults:

qwen3.8-omni-flash-realtime models: 20
qwen3.5-omni-realtime models: 20
qwen3-omni-flash-realtime models: 50
qwen-omni-turbo-realtime models: 20

qwen-omni-turbo models do not support this parameter.

integer

body

Maximum tokens to return. Output is truncated beyond this limit; generation itself is not affected. For qwen3.8-omni-flash-realtime, the valid range is [1, 65536]. For other models, the default and maximum values equal the model's maximum output length. See Model list. Use to control word count, costs, or latency. qwen-omni-turbo models do not support this parameter.

float

body

Penalty for consecutive repetition. Higher values reduce repetition. 1.0 = no penalty. qwen3.8-omni-flash-realtime also accepts 0; for other models the value must be greater than 0. Defaults: qwen3.8-omni-flash-realtime models: 0, qwen3.5-omni-realtime models: 1.0, qwen3-omni-flash-realtime models: 1.05. qwen-omni-turbo models do not support this parameter.

float

body

Controls repetition. Range: [-2.0, 2.0]. Defaults: qwen3.8-omni-flash-realtime models: 1, qwen3.5-omni-realtime models: 1.5, qwen3-omni-flash-realtime models: 0.0. Positive values reduce repetition; negative values increase it. Use higher values for creative tasks, lower values for formal content. qwen-omni-turbo models do not support this parameter.

integer

body

Makes output deterministic. With the same seed and parameters, the model returns identical results. Range: 0 to 2^31-1. Default: -1. qwen-omni-turbo models do not support this parameter.

Audio configuration

audio object (optional) Input and output audio configuration. Omitted fields retain their existing defaults. The format and sample rate options below apply to qwen3.5-omni-plus-realtime and qwen3.5-omni-flash-realtime. For multichannel input fields and constraints, see Multichannel audio input.

Hide Properties

audio.input object (optional)User input audio configuration.audio.input.format object (optional)Input audio format and sample rate. Configure these while the session is IDLE, before sending the first audio segment. They cannot change after audio input starts.audio.input.format.type string (optional)Input audio format: pcm (default, mono 16-bit raw PCM) or wav (mono 16-bit PCM in a WAV container).audio.input.format.sample_rate integer (optional)Input sample rate in Hz: 8000, 16000 (default), 24000, or 48000.audio.output object (optional)Model output audio configuration.audio.output.format object (optional)Output audio format and sample rate. Configure these at the start of the session, before audio interaction begins.audio.output.format.type string (optional)Output audio format: pcm (default, mono 16-bit raw PCM) or wav (mono 16-bit PCM in a WAV container).audio.output.format.sample_rate integer (optional)Output sample rate in Hz: 8000, 16000, 24000 (default), or 48000.

Multichannel audio input

The following multichannel audio configuration applies to Qwen3.8-Omni-Flash-Realtime over WebSocket.

Field	Type	Required / default	Values and constraints
`type`	string	Optional; `pcm`	Multichannel input must use `pcm`.
`sample_rate`	integer	Optional; `16000`	Multichannel input must use 16000 Hz.
`sample_format`	string	Optional; `s16le`	Only `s16le`.
`channels`	integer	Optional; `1`	Only `1`, `2`, or `4`.
`packing`	string	Optional; `interleaved`	Only `interleaved`.
`channel_layout`	string	Optional; determined by `channels`	`mono` for 1 channel, `raw_mic_array` for 2, `foa_ambix` for 4. An explicit value must match `channels`.

Multichannel input requires PCM, 16000 Hz, s16le, and interleaved samples. Configure the format before sending the first audio segment; these fields cannot change after audio input starts. Omitted fields use their defaults. Supplied fields must have valid types and values; do not send null instead of omitting a field. Two-channel configuration:

{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "format": {
          "type": "pcm",
          "sample_rate": 16000,
          "sample_format": "s16le",
          "channels": 2,
          "packing": "interleaved",
          "channel_layout": "raw_mic_array"
        }
      }
    }
  }
}

Four-channel configuration:

{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "format": {
          "type": "pcm",
          "sample_rate": 16000,
          "sample_format": "s16le",
          "channels": 4,
          "packing": "interleaved",
          "channel_layout": "foa_ambix"
        }
      }
    }
  }
}

Compatible single-channel configuration:

{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "format": {
          "type": "pcm",
          "sample_rate": 16000
        }
      }
    }
  }
}

For 2-channel and 4-channel spatial audio, the input token count is twice that of ordinary audio. See Token calculation.

Output voice

The following configuration applies to Qwen3.8-Omni-Flash-Realtime. session.audio.output.voice is an optional string for the output voice, which defaults to Tina. longanlingxin is newly supported. The optional compatibility field session.voice remains available; use session.audio.output.voice for new integrations. If both are supplied, session.audio.output.voice takes precedence. See voices.

{
  "type": "session.update",
  "session": {
    "audio": {
      "output": {
        "voice": "longanlingxin"
      }
    }
  }
}

Video configuration

The following configuration applies to Qwen3.8-Omni-Flash-Realtime. representation_compact is an optional string. The initial session default is none. none preserves the full fine-grained video representation; normal aggregates it to reduce computation for scenarios that do not require fine visual detail. For the same video input, normal uses one quarter of the tokens used by none. See Token calculation. Set this field before the first audio segment; it cannot change after audio input starts.

{
  "type": "session.update",
  "session": {
    "video": {
      "input": {
        "representation_compact": "normal"
      }
    }
  }
}

MCP tool configuration

The following configuration applies to Qwen3.8-Omni-Flash-Realtime. Required fields must be supplied when their containing object is present. IDs are opaque strings; do not rely on their lengths, prefixes, or generation rules. MCP connections, discovery, and calls are subject to service quotas and timeouts. An element with type="mcp" configures an MCP server. A session can contain both Function Calling and MCP tools. tools and enable_search are mutually exclusive, including MCP tools. For server and tool quotas, timeouts, and result size limits, see MCP call limits.

Field	Type	Required / default	Constraints
`type`	string	Required	`mcp`
`server_label`	string	Required	Unique within the session; 1-64 characters; letters, digits, underscores, and hyphens only.
`server_url`	string	Required	Public HTTPS address on port 443; up to 4096 characters; MCP Streamable HTTP endpoint.
`authorization`	string	Optional	Outbound Authorization value; up to 8192 printable ASCII characters.
`headers`	object	Optional	Up to 16 string-to-string HTTP header pairs.
`allowed_tools`	array of strings	Optional; all tools when omitted	An empty array exposes no tools. Each name is 1-64 characters, with letters, digits, underscores, periods, and hyphens only. Applied after discovery.
`require_approval`	string	Optional; `always`	`always` or `never`.

server_url must not contain a username, password, or fragment. Its domain must resolve to public addresses. Header names must be 1-128 characters and use standard HTTP header-name characters. Header values must contain only printable ASCII and be at most 8192 characters. The following headers are prohibited:

Prefixes: mcp-, proxy-, x-forwarded-
Names: host, authorization, connection, content-length, transfer-encoding,
accept, content-type, forwarded, cookie, origin, upgrade, te, trailer

Every MCP configuration addition or update requires both server_label and server_url; supplying only a label does not reuse a configuration. Update MCP configuration only when there is no active Response. server_url, authorization, and headers are used only for the server-side connection and are not echoed in session.updated. Do not reconstruct sensitive connection settings from that event. Replace the placeholder endpoint and credential in this configuration fragment with your MCP server details before sending it over an established connection:

{
  "type": "session.update",
  "session": {
    "tools": [
      {
        "type": "mcp",
        "server_label": "amap",
        "server_url": "https://example.com/mcp",
        "authorization": "Bearer ***",
        "allowed_tools": ["maps_weather"],
        "require_approval": "always"
      }
    ]
  }
}

response.create

Tells the service to generate a model response. In VAD mode, responses are automatic and you do not need this event. The service responds with response.created, then item and content events (conversation.item.created, response.content_part.added), and finally response.done.

Example

{
  "type": "response.create",
  "event_id": "event_1718624400000"
}

string

body

required

Event type. Always response.create.

response.cancel

Cancels an ongoing response. Returns an error if no response is in progress.

Example

{
  "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
  "type": "response.cancel"
}

string

body

required

Event type. Always response.cancel.

input_audio_buffer.append

Appends audio bytes to the input buffer.

Example

{
  "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
  "type": "input_audio_buffer.append",
  "audio": "UklGR..."
}

string

body

required

Event type. Always input_audio_buffer.append.

string

body

required

The Base64-encoded audio data.

input_audio_buffer.commit

Submits the input audio buffer as a user message. Returns an error if the buffer is empty.

VAD mode: Automatic. You do not need this event.
Manual mode: Required to create a user message.

Submitting the buffer does not trigger a model response. The service responds with input_audio_buffer.committed.

If you have sent an input_image_buffer.append event, input_audio_buffer.commit submits the image buffer along with the audio buffer.

Example

{
  "event_id": "event_B4o9RHSTWobB5OQdEHLTo",
  "type": "input_audio_buffer.commit"
}

string

body

required

Event type. Always input_audio_buffer.commit.

input_audio_buffer.clear

Clears the audio buffer. The service responds with input_audio_buffer.cleared.

Example

{
  "event_id": "event_xxx",
  "type": "input_audio_buffer.clear"
}

string

body

required

Event type. Always input_audio_buffer.clear.

input_image_buffer.append

Adds image data to the image buffer from local files or video streams. Limits:

Format: JPG or JPEG. Recommended: 480p or 720p. Maximum: 1080p.
Size: ≤256 KB after Base64 encoding (recommend raw image below 190 KB).
Encoding: Base64.
Frequency: 1 image per second.
Prerequisite: Send at least one input_audio_buffer.append event first.

The image buffer is submitted with the audio buffer through the input_audio_buffer.commit event.

Example

{
  "event_id": "event_xxx",
  "type": "input_image_buffer.append",
  "image": "xxx"
}

string

body

required

Event type. Always input_image_buffer.append.

string

body

required

The Base64-encoded image data.

conversation.item.create

Returns the execution result of a tool function to the server. After the model triggers a tool call, execute the tool function locally, send the result back using this event, then send a response.create event to trigger the model to generate the final response.

This section describes function_call_output items. For Qwen3.8-Omni-Flash-Realtime MCP approval replies, see mcp_approval_response.

Example

{
  "event_id": "event_55099cddb51b4f208cb95d1a994eef80",
  "type": "conversation.item.create",
  "item": {
    "id": "item_2a80d7682b4e473c9c2154da135041e9",
    "type": "function_call_output",
    "call_id": "call_62c24725afdb4c2680ac54",
    "output": "The weather in Beijing today is changing from haze to clear, with a temperature of 4/-4°C and a light breeze."
  }
}

string

body

required

Event type. Always conversation.item.create.

object

body

required

The conversation item to create.

Show properties

string

body

The conversation item ID. The client can specify an ID to align with the local state. If not provided, the server generates one.

string

body

required

The conversation item type described here is function_call_output. Qwen3.8-Omni-Flash-Realtime also supports mcp_approval_response.

string

body

required

Corresponds to the call_id returned in the response.function_call_arguments.done event.

string

body

required

The execution result of the tool function.

MCP approval response (Qwen3.8-Omni-Flash-Realtime)

When require_approval is always or omitted, the server may send mcp_approval_request. Reply with the existing conversation.item.create event.

Field	Type	Required	Meaning
`event_id`	string	No	Client-generated tracing ID.
`type`	string	Yes	`conversation.item.create`
`item`	object	Yes	Approval response.
`item.type`	string	Yes	`mcp_approval_response`
`item.approval_request_id`	string	Yes	Exact `item.id` of an unhandled `mcp_approval_request`.
`item.approve`	boolean	Yes	`true` allows execution; `false` rejects it and the call enters `failed`.

{
  "event_id": "event_client_xxx",
  "type": "conversation.item.create",
  "item": {
    "type": "mcp_approval_response",
    "approval_request_id": "opaque_approval_id",
    "approve": true
  }
}

For Qwen3.8-Omni-Flash-Realtime, session.updated does not echo MCP endpoint or credential fields; see the Qwen3.8 client events.

​session.update

​Audio configuration

​Multichannel audio input

​Output voice

​Video configuration

​MCP tool configuration

​response.create

​response.cancel

​input_audio_buffer.append

​input_audio_buffer.commit

​input_audio_buffer.clear

​input_image_buffer.append

​conversation.item.create

​MCP approval response (Qwen3.8-Omni-Flash-Realtime)

session.update

Audio configuration

Multichannel audio input

Output voice

Video configuration

MCP tool configuration

response.create

response.cancel

input_audio_buffer.append

input_audio_buffer.commit

input_audio_buffer.clear

input_image_buffer.append

conversation.item.create

MCP approval response (Qwen3.8-Omni-Flash-Realtime)