Live speech to text
The real-time speech recognition service receives an audio stream and transcribes it into punctuated text in real time. Use it for live captioning, online meetings, voice chat, smart assistants, and similar scenarios.
The service streams audio and returns transcribed text with low latency.
In addition to WebSocket, Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime models also support the AOQ protocol. For client-side integration that prioritizes stable latency, resilience on weak networks, and built-in full-duplex noise suppression and echo cancellation, AOQ is recommended. For a protocol comparison, see Realtime API overview.
Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime output timestamps at both the sentence level and the word level by default, which supports subtitle alignment, keyword highlighting, karaoke-style read-along, and similar scenarios. Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime) does not currently return timestamps. If you need timestamps, use Qwen-Audio-3.0-ASR-Flash-Streaming or Fun-ASR-Realtime. For file transcription, the recording-file transcription model
The field names above follow the WebSocket JSON paths. Different SDKs expose these fields with their own naming conventions (dictionary keys, object properties, getter methods, and so on). For the complete field mapping, see Server events.
Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime) can include the speaker's emotional state in the transcription result. It is always on and requires no configuration. The emotion is returned through a top-level
The field names above follow the WebSocket JSON paths. Different SDKs expose these fields with their own naming conventions (dictionary keys, object properties, getter methods, and so on). For the full field definitions, value constraints, and examples, see Server events.
WebSocket connections for Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime can be reused: after one recognition task finishes, you can start the next task without establishing a new connection.
Reuse flow: The client sends
Qwen3-ASR-Flash-Realtime uses a session model. You must close the connection after each session ends, and connection reuse is not supported.
For the events of each model, see the corresponding API reference.
The DashScope SDK includes a built-in pooling mechanism that reuses WebSocket connections and recognition objects, avoiding the overhead of frequent creation and destruction.
2. Configure the connection poolConfigure the key connection pool parameters through environment variables:
3. Configure the object poolConfigure the object pool size through an environment variable:
Create the object pool with the following code:4. Borrow a Recognition object from the object poolWhen the number of unreturned objects exceeds the object pool limit, the system creates additional 5. Run speech recognitionCall the
By providing context, you can optimize the recognition of domain-specific vocabulary, such as names, places, and product terms.
Length limit: The context content cannot exceed 10,000 tokens.
Usage:
To achieve the result above, add any of the following content to the context:
Sensitive word filtering replaces or removes sensitive words in the recognition result. Use it for call-center quality inspection, content compliance, subtitle review, and similar scenarios.
Supported models: Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime only.
Limit: You can set up to 32 sensitive words.
Default behavior: When the
Different SDKs expose these parameters with their own naming conventions (dictionary keys, object properties, methods, and so on). For the complete field mapping, see the API reference.
Voice Activity Detection (VAD) determines when a continuous segment of speech ends, which triggers the final recognition result event. All three model families enable server-side VAD by default, but their parameter names and tuning granularity differ:
Qwen3-ASR-Flash-Realtime and some Paraformer models can include the speaker's emotional state in the transcription result, but the two differ in output granularity and in how the feature is enabled.
Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime): Always on, no configuration required. The emotion is returned through a top-level
Qwen real-time speech recognition streams audio over WebSocket. Two modes are available: VAD mode (default) and Manual mode.
Replace
The server detects speech boundaries and segments sentences. The client streams audio, and the server returns results when each sentence ends. Best for conversations and meeting transcription.
Enable: Set
The client controls sentence segmentation by sending audio for a complete sentence, then sending
You can also use Qwen-Omni (
ASR prompt template:
Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime support pcm, wav, mp3, opus, speex, aac, and amr. For Qwen3-ASR-Flash-Realtime, use pcm or opus. Other formats such as wav, aac, and amr pass the
The DashScope SDK encapsulates WebSocket connection management, authentication, reconnection, and other details, which makes it suitable for quick integration. Connecting to the WebSocket API directly gives you finer-grained control and suits programming languages the SDK does not cover or scenarios that need custom connection management. We recommend the SDK.
Use hotwords or context enhancement. For detailed configuration methods and usage notes, see Improve recognition accuracy.
Implement client-side reconnection and enable the heartbeat parameter (
Overview
The service streams audio and returns transcribed text with low latency.
In addition to WebSocket, Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime models also support the AOQ protocol. For client-side integration that prioritizes stable latency, resilience on weak networks, and built-in full-duplex noise suppression and echo cancellation, AOQ is recommended. For a protocol comparison, see Realtime API overview.
- Recognizes Mandarin Chinese with high accuracy, plus Cantonese, Sichuanese, and other dialects.
- Handles complex acoustic environments, with automatic language detection and intelligent filtering of non-speech audio.
- Recognizes a range of emotional states, including surprise, calm, happiness, sadness, disgust, anger, and fear.
- Supports custom hotwords to improve recognition accuracy for specific terms.
- Supports context enhancement to improve recognition accuracy by passing in conversation history or domain terms.
- Outputs timestamps to produce structured recognition results.
- Accepts flexible sample rates and multiple audio formats to fit different recording environments.
For model availability, supported languages, and feature comparison, see Speech-to-text models.
Prerequisites
- Obtain an API key and set it as an environment variable.
- To call the service through the DashScope SDK, install the latest SDK.
- To connect to Qwen-Audio-3.0-ASR-Flash-Streaming or Fun-ASR-Realtime over the AOQ protocol, download and integrate the AOQ client SDK. See AOQ SDK overview.
Getting started
- Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime
- Qwen3-ASR-Flash-Realtime
- DashScope SDK
- WebSocket API
For more code samples, see GitHub.Get an API key and set it as an environment variable. To use the SDK, install it.
Model availability
| Model | Version | Unit price | Free quota (Note) |
|---|---|---|---|
| fun-asr-realtime Currently, fun-asr-realtime-2025-11-07 | Stable | $0.00009/second | 36,000 seconds (10 hours) Valid for 90 days |
| fun-asr-realtime-2025-11-07 | Snapshot | $0.00009/second | 36,000 seconds (10 hours) Valid for 90 days |
- Languages: Mandarin, Cantonese, Wu, Minnan, Hakka, Gan, Xiang, and Jin. Also supports Mandarin accents from Zhongyuan, Southwest, Jilu, Jianghuai, Lanyin, Jiaoliao, Northeast, Beijing, and Hong Kong-Taiwan regions -- including Henan, Shaanxi, Hubei, Sichuan, Chongqing, Yunnan, Guizhou, Guangdong, Guangxi, Hebei, Tianjin, Shandong, Anhui, Nanjing, Jiangsu, Hangzhou, Gansu, and Ningxia. English and Japanese are also supported.
- Sample rate: 16 kHz
- Audio formats: pcm, wav, mp3, opus, speex, aac, amr
Recognize speech from a microphone
Recognize speech from a microphone and output text in real time, so words appear as the speaker talks.Before you run the Python example, install the third-party audio playback and capture toolkit with
pip install pyaudio. pyaudio requires the portaudio library. On Ubuntu/Debian: sudo apt-get install libportaudio2 portaudio19-dev. On macOS: brew install portaudio.Recognize a local audio file
Recognize a local audio file and output the result. This suits shorter, near-real-time scenarios such as chat conversations, voice commands, voice input methods, and voice search.Advanced features
Get timestamps
Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime output timestamps at both the sentence level and the word level by default, which supports subtitle alignment, keyword highlighting, karaoke-style read-along, and similar scenarios. Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime) does not currently return timestamps. If you need timestamps, use Qwen-Audio-3.0-ASR-Flash-Streaming or Fun-ASR-Realtime. For file transcription, the recording-file transcription model qwen3-asr-flash-filetrans supports word-level timestamps. For details, see Non-real-time speech recognition.
Timestamps are returned in milliseconds at two levels:
- Sentence level:
payload.output.sentence.begin_timeandpayload.output.sentence.end_timemark the start and end of a full sentence in the audio. In an intermediate result,end_timemay benulland is filled with the final value when the sentence ends (sentence_end = true). - Word level: The
payload.output.sentence.wordsarray, where each element containsbegin_time,end_time,text(the word or character text), andpunctuation(the punctuation that follows the word, or an empty string if none).
Emotion recognition
Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime) can include the speaker's emotional state in the transcription result. It is always on and requires no configuration. The emotion is returned through a top-level emotion field in both the conversation.item.input_audio_transcription.text and conversation.item.input_audio_transcription.completed events. The value is one of seven fine-grained emotions: surprised, neutral, happy, sad, disgusted, angry, and fearful.
Going live
Reuse connections (WebSocket)
WebSocket connections for Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime can be reused: after one recognition task finishes, you can start the next task without establishing a new connection.
Reuse flow: The client sends finish-task. After the server returns task-finished, the client can send run-task again to start a new task.
- Wait for the server to return the
task-finishedevent before starting a new task. - Different tasks on a reused connection must use different
task_idvalues. - When a task fails, the server returns an error event and closes the connection. That connection cannot be reused.
- If no new task starts within 60 seconds after a task ends, the connection is closed automatically.
High-concurrency best practices
The DashScope SDK includes a built-in pooling mechanism that reuses WebSocket connections and recognition objects, avoiding the overhead of frequent creation and destruction.
Only the Java SDK supports this feature.
View high-concurrency best practices
View high-concurrency best practices
Prerequisites
- Obtain an API key and set it as an environment variable.
- Install a DashScope SDK that meets the version requirement. We recommend installing the latest version: Java SDK 2.16.9 or later.
- Connection pool: The OkHttp3 connection pool integrated into the SDK manages and reuses the underlying WebSocket connections, reducing handshake overhead. It is enabled by default.
- Object pool: Implemented with
commons-pool2, it maintains a set ofRecognitionobjects whose connections are already established. Borrowing an object from the pool removes the connection setup latency and significantly reduces first-packet latency.
Implementation steps
1. Add dependenciesAdd dashscope-sdk-java and commons-pool2 to your dependency configuration file, according to your build tool.- Maven
- Gradle
Add the following dependencies inside the
<dependencies> tag of pom.xml, then run mvn clean install or mvn compile to update the dependencies.| Environment variable | Description |
|---|---|
DASHSCOPE_CONNECTION_POOL_SIZE | Connection pool size. Recommended: at least twice your peak concurrency. Default: 32. |
DASHSCOPE_MAXIMUM_ASYNC_REQUESTS | Maximum number of async requests. Recommended: the same as DASHSCOPE_CONNECTION_POOL_SIZE. Default: 32. |
DASHSCOPE_MAXIMUM_ASYNC_REQUESTS_PER_HOST | Maximum number of async requests per host. Recommended: the same as DASHSCOPE_CONNECTION_POOL_SIZE. Default: 32. |
| Environment variable | Description |
|---|---|
RECOGNITION_OBJECTPOOL_SIZE | Object pool size. Recommended: 1.5 to 2 times your peak concurrency. Default: 500. |
- The object pool size (
RECOGNITION_OBJECTPOOL_SIZE) must be less than or equal to the connection pool size (DASHSCOPE_CONNECTION_POOL_SIZE). Otherwise, when the object pool requests an object and the connection pool is full, the calling thread blocks. - The object pool size should not exceed your account's QPS limit.
Recognition objects. These new objects must establish a new WebSocket connection and cannot be reused.call or streamCall method of the Recognition object to run speech recognition.6. Return the Recognition objectAfter the recognition task finishes, return the Recognition object so it can be reused. Do not return objects whose tasks are unfinished or failed.Full code
Recommended configuration
The following configuration is based on test results from running only the real-time speech recognition service on servers of the specified sizes. Single-machine concurrency here means the number of real-time speech recognition tasks running at the same time (that is, the number of worker threads).| Machine size | Max concurrency per machine | Object pool size | Connection pool size |
|---|---|---|---|
| 4 cores, 8 GiB | 100 | 500 | 2000 |
| 8 cores, 16 GiB | 200 | 500 | 2000 |
| 16 cores, 32 GiB | 400 | 500 | 2000 |
Resource management and exception handling
-
Task succeeded: You must call
GenericObjectPool.returnObject()to return the Recognition object to the pool for reuse.Do not return a Recognition object whose task is unfinished or failed. - Task failed: When an exception thrown by the SDK or your business logic interrupts the task, you must close the underlying WebSocket connection and invalidate the object in the pool so it is not used again.
- No extra handling is needed when the service returns a TaskFailed error.
Warm-up and latency measurement
When you evaluate performance such as concurrent-call latency for the DashScope Java SDK, warm up thoroughly before the formal test.Connection reuse mechanismThe DashScope Java SDK manages and reuses WebSocket connections through a global singleton connection pool. The mechanism works as follows:- Created on demand: The SDK does not pre-create WebSocket connections at service startup. Connections are established on the first call.
- Time-limited reuse: After a request completes, the connection stays in the pool for up to 60 seconds for reuse. A new request within 60 seconds reuses the existing connection and avoids another handshake. A connection idle for more than 60 seconds is closed automatically to release resources.
- The application has just started and has not made any calls yet.
- The service has been idle for more than 60 seconds and pooled connections have timed out and closed.
- Issue calls at the concurrency level of the formal test in advance (for example, for 1 to 2 minutes) to fill the connection pool.
- Confirm that the connection pool has established and maintains enough active connections, then start collecting formal performance data.
Improve recognition accuracy
- Choose a model that matches the sample rate: For 8 kHz telephone audio, use an 8 kHz model directly. This avoids the information loss caused by upsampling to 16 kHz.
- Use hotwords or context enhancement: For proprietary nouns, names, and brand names specific to your business, you can configure hotwords or context enhancement to significantly improve recognition accuracy. For detailed configuration methods and usage notes, see Improve recognition accuracy.
- Improve the input audio quality: Use a high-quality microphone and record in an environment with a high signal-to-noise ratio and no echo. At the application layer, you can integrate algorithms such as noise reduction (for example, RNNoise) and acoustic echo cancellation (AEC) for preprocessing.
- Specify the recognition language: For multilingual models, if you can predetermine the audio language when making a call, it helps the model converge and avoid confusion between similarly pronounced languages, which improves accuracy.
Set up a fault-tolerance strategy
- Client-side reconnection: The client should implement automatic reconnection to handle network jitter. The following is a reference implementation for the Python SDK:
- Catch exceptions: Implement the
on_errormethod in theCallbackclass. ThedashscopeSDK calls this method when it encounters a network error or another issue. - Signal the state: When
on_erroris triggered, set a reconnection signal. In Python, you can usethreading.Event, a thread-safe signal flag. - Reconnection loop: Wrap the main logic in a
forloop (for example, retry 3 times). When the reconnection signal is detected, the current recognition round is interrupted, resources are cleaned up, and after a few seconds the loop runs again to create a brand-new connection.
- Catch exceptions: Implement the
- Set a heartbeat to keep the connection alive: To maintain a long-lived connection with the server, set the heartbeat parameter to
true. The connection to the server then stays open even when the audio contains no sound for a long time. - Model rate limits: When you call the model API, note the model's Rate limiting rules.
Core usage: Context biasing (Qwen3-ASR-Flash-Realtime)
By providing context, you can optimize the recognition of domain-specific vocabulary, such as names, places, and product terms.
Length limit: The context content cannot exceed 10,000 tokens.
Usage:
- WebSocket API: Set the
session.input_audio_transcription.corpus.textparameter in the session.update event. - Python SDK: Set the
corpus_textparameter. - Java SDK: Set the
corpusTextparameter.
- Hotword lists in various separator formats, such as Hotword 1, Hotword 2, Hotword 3, Hotword 4
- Text paragraphs or chapters of any format and length
- Mixed content: Any combination of word lists and paragraphs
- Irrelevant or meaningless text, including garbled text. The feature is highly fault-tolerant and is almost never negatively affected by irrelevant text.
| Without context enhancement | With context enhancement |
|---|---|
| Without context enhancement, some investment bank names may be misrecognized. For example, "Bird Rock" should be "Bulge Bracket". Recognition result: "What internal jargon from the investment banking circle do you know? First, the nine major foreign investment banks, Bird Rock, BB..." | With context enhancement, investment bank names are recognized correctly. Recognition result: "What internal jargon from the investment banking circle do you know? First, the nine major foreign investment banks, the Bulge Bracket, BB..." |
- Word lists:
- Word list 1:
- Word list 2:
- Word list 3:
- Natural language:
- Natural language with interference: Some text is irrelevant to the recognition content, such as the names in the example below.
Core usage: Sensitive word filtering (Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime)
Sensitive word filtering replaces or removes sensitive words in the recognition result. Use it for call-center quality inspection, content compliance, subtitle review, and similar scenarios.
Supported models: Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime only.
Limit: You can set up to 32 sensitive words.
Default behavior: When the special_word_filter parameter is not passed, no sensitive words are filtered.
How to configure: special_word_filter is a JSON object with three subfields:
-
filter_with_signed.word_list: A string array that lists the sensitive words to replace with an equal-length string of*characters. For example, with["test"], "Help me test it" becomes "Help me **** it". -
filter_with_empty.word_list: A string array that lists the sensitive words to remove entirely from the result. For example, with["start"], "Is the game about to start" becomes "Is the game about to". -
system_reserved_filter: A boolean that defaults tofalse. It determines whether sensitive word filtering is enabled.
VAD segmentation configuration
Voice Activity Detection (VAD) determines when a continuous segment of speech ends, which triggers the final recognition result event. All three model families enable server-side VAD by default, but their parameter names and tuning granularity differ:
- Qwen-Audio-3.0-ASR-Flash-Streaming / Fun-ASR-Realtime / Paraformer: Configured through
max_sentence_silence(the VAD silence threshold for segmentation, in milliseconds). When the silence after a segment of speech exceeds this threshold, the system treats the sentence as complete. - Qwen3-ASR-Flash-Realtime: Configured through
session.turn_detection, which includessilence_duration_ms(the silence duration threshold that ends a turn when exceeded; server default800, with400recommended for conversation and chat scenarios that need fast segmentation) andthreshold(VAD detection sensitivity; server default0.2). Qwen3-ASR-Flash-Realtime also supports Manual mode, which disables VAD and uses client-side commit for segmentation. For details, see the Interaction flow section below.
max_sentence_silence in Qwen-Audio-3.0-ASR-Flash-Streaming / Fun-ASR-Realtime / Paraformer, and silence_duration_ms in Qwen3-ASR-Flash-Realtime. For the full field definitions, see the API reference below.
Emotion recognition
Qwen3-ASR-Flash-Realtime and some Paraformer models can include the speaker's emotional state in the transcription result, but the two differ in output granularity and in how the feature is enabled.
Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime): Always on, no configuration required. The emotion is returned through a top-level emotion field in both the conversation.item.input_audio_transcription.text and conversation.item.input_audio_transcription.completed events. The value is one of seven fine-grained emotions: surprised, neutral, happy, sad, disgusted, angry, and fearful.
API reference
- Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime
- Qwen3-ASR-Flash-Realtime
- Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime real-time speech recognition API reference
- AOQ Client SDK (for Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime)
Interaction flow (Qwen3-ASR-Flash-Realtime)
Qwen real-time speech recognition streams audio over WebSocket. Two modes are available: VAD mode (default) and Manual mode.
URL
Replace <model_name> with your model name.
Headers
VAD mode (default)
The server detects speech boundaries and segments sentences. The client streams audio, and the server returns results when each sentence ends. Best for conversations and meeting transcription.
Enable: Set session.turn_detection in session.update.
-
The client sends
input_audio_buffer.appendto add audio to the buffer. -
The server returns
input_audio_buffer.speech_startedwhen it detects speech.If the client sendssession.finishbefore this event, the server returnssession.finishedand the client must disconnect. -
The client continues sending
input_audio_buffer.append. -
After all audio is sent, the client sends
session.finishto end the session. -
The server returns
input_audio_buffer.speech_stoppedwhen it detects the end of speech. -
The server returns
input_audio_buffer.committed. -
The server returns
conversation.item.created. -
The server returns
conversation.item.input_audio_transcription.textwith real-time transcription results. -
The server returns
conversation.item.input_audio_transcription.completedwith the final transcription result. -
The server returns
session.finishedwhen recognition completes. The client must then disconnect.
Manual mode
The client controls sentence segmentation by sending audio for a complete sentence, then sending input_audio_buffer.commit. Best when the client knows sentence boundaries, for example in chat app voice messages.
Enable: Set session.turn_detection to null in session.update.
-
The client sends
input_audio_buffer.appendto add audio to the buffer. -
The client sends
input_audio_buffer.committo create a new user message. -
The client sends
session.finishto end the session. -
The server returns
input_audio_buffer.committed. -
The server returns
conversation.item.input_audio_transcription.textwith real-time transcription results. -
The server returns
conversation.item.input_audio_transcription.completedwith the final transcription result. -
The server returns
session.finishedwhen recognition completes. The client must then disconnect.
Alternative: Use Qwen-Omni
You can also use Qwen-Omni (qwen3-omni-flash-realtime) for real-time speech recognition via WebSocket. Omni is an LLM that understands audio — you provide domain context through the system prompt instead of hotword lists.
When to use Omni for ASR: Clean speech inputs (microphone, voice calls) where you need domain-specific terminology handling via prompt.
When to use dedicated ASR models instead: Noisy or mixed audio (meetings with background music, videos with sound effects), or when you need hotwords, speaker diarization, or timestamps.
Qwen-Omni interprets all audio, not just speech. Music, typing, or ambient noise may produce descriptions instead of transcription. For mixed audio, preprocess with VAD to isolate speech, or use a dedicated ASR model.
Qwen-Omni-Realtime uses WebSocket for bidirectional streaming. For the full API and SDK reference, see Realtime conversation.
FAQ
Which audio formats does real-time speech recognition support?
Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime support pcm, wav, mp3, opus, speex, aac, and amr. For Qwen3-ASR-Flash-Realtime, use pcm or opus. Other formats such as wav, aac, and amr pass the session.update validation layer, but server-side decoding may fail. Confirm that the audio stream uses a recommended format before sending it.
What's the difference between the SDK and the WebSocket API, and how do I choose?
The DashScope SDK encapsulates WebSocket connection management, authentication, reconnection, and other details, which makes it suitable for quick integration. Connecting to the WebSocket API directly gives you finer-grained control and suits programming languages the SDK does not cover or scenarios that need custom connection management. We recommend the SDK.
How do I improve recognition accuracy for proper nouns?
Use hotwords or context enhancement. For detailed configuration methods and usage notes, see Improve recognition accuracy.
What should I do when the connection drops frequently?
Implement client-side reconnection and enable the heartbeat parameter (heartbeat=true) to prevent the connection from dropping when there is no audio for a long time. For the full fault-tolerance strategy, see Set up a fault-tolerance strategy.