Skip to main content
Speech-to-text

Realtime speech recognition

Live speech to text

The real-time speech recognition service receives an audio stream and transcribes it into punctuated text in real time. Use it for live captioning, online meetings, voice chat, smart assistants, and similar scenarios.

Overview

The service streams audio and returns transcribed text with low latency. In addition to WebSocket, Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime models also support the AOQ protocol. For client-side integration that prioritizes stable latency, resilience on weak networks, and built-in full-duplex noise suppression and echo cancellation, AOQ is recommended. For a protocol comparison, see Realtime API overview.
  • Recognizes Mandarin Chinese with high accuracy, plus Cantonese, Sichuanese, and other dialects.
  • Handles complex acoustic environments, with automatic language detection and intelligent filtering of non-speech audio.
  • Recognizes a range of emotional states, including surprise, calm, happiness, sadness, disgust, anger, and fear.
  • Supports custom hotwords to improve recognition accuracy for specific terms.
  • Supports context enhancement to improve recognition accuracy by passing in conversation history or domain terms.
  • Outputs timestamps to produce structured recognition results.
  • Accepts flexible sample rates and multiple audio formats to fit different recording environments.
For batch scenarios such as meeting transcription, call analysis, and subtitle generation, use Non-real-time speech recognition. For guidance on choosing a model, see Speech-to-text.
For model availability, supported languages, and feature comparison, see Speech-to-text models.

Prerequisites

Getting started

  • Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime
  • Qwen3-ASR-Flash-Realtime
  • DashScope SDK
  • WebSocket API
For more code samples, see GitHub.Get an API key and set it as an environment variable. To use the SDK, install it.

Model availability

ModelVersionUnit priceFree quota (Note)
fun-asr-realtime
Currently, fun-asr-realtime-2025-11-07
Stable$0.00009/second36,000 seconds (10 hours)
Valid for 90 days
fun-asr-realtime-2025-11-07Snapshot$0.00009/second36,000 seconds (10 hours)
Valid for 90 days
  • Languages: Mandarin, Cantonese, Wu, Minnan, Hakka, Gan, Xiang, and Jin. Also supports Mandarin accents from Zhongyuan, Southwest, Jilu, Jianghuai, Lanyin, Jiaoliao, Northeast, Beijing, and Hong Kong-Taiwan regions -- including Henan, Shaanxi, Hubei, Sichuan, Chongqing, Yunnan, Guizhou, Guangdong, Guangxi, Hebei, Tianjin, Shandong, Anhui, Nanjing, Jiangsu, Hangzhou, Gansu, and Ningxia. English and Japanese are also supported.
  • Sample rate: 16 kHz
  • Audio formats: pcm, wav, mp3, opus, speex, aac, amr

Recognize speech from a microphone

Recognize speech from a microphone and output text in real time, so words appear as the speaker talks.
import com.alibaba.dashscope.audio.asr.recognition.Recognition;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionParam;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionResult;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.utils.Constants;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.TargetDataLine;

import java.nio.ByteBuffer;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

public class Main {
  public static void main(String[] args) throws InterruptedException {
    Constants.baseWebsocketApiUrl = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference";
    ExecutorService executorService = Executors.newSingleThreadExecutor();
    executorService.submit(new RealtimeRecognitionTask());
    executorService.shutdown();
    executorService.awaitTermination(1, TimeUnit.MINUTES);
    System.exit(0);
  }
}

class RealtimeRecognitionTask implements Runnable {
  @Override
  public void run() {
    RecognitionParam param = RecognitionParam.builder()
        .model("qwen-audio-3.0-asr-flash-streaming")
        // If you have not configured an environment variable, replace the following line with your API key: .apiKey("sk-xxx")
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .format("pcm")
        .sampleRate(16000)
        .build();
    Recognition recognizer = new Recognition();

    ResultCallback<RecognitionResult> callback = new ResultCallback<RecognitionResult>() {
      @Override
      public void onEvent(RecognitionResult result) {
        if (result.isSentenceEnd()) {
          System.out.println("Final Result: " + result.getSentence().getText());
        } else {
          System.out.println("Intermediate Result: " + result.getSentence().getText());
        }
      }

      @Override
      public void onComplete() {
        System.out.println("Recognition complete");
      }

      @Override
      public void onError(Exception e) {
        System.out.println("RecognitionCallback error: " + e.getMessage());
      }
    };
    try {
      recognizer.call(param, callback);
      // Create the audio format
      AudioFormat audioFormat = new AudioFormat(16000, 16, 1, true, false);
      // Match the default recording device based on the format
      TargetDataLine targetDataLine =
          AudioSystem.getTargetDataLine(audioFormat);
      targetDataLine.open(audioFormat);
      // Start recording
      targetDataLine.start();
      ByteBuffer buffer = ByteBuffer.allocate(1024);
      long start = System.currentTimeMillis();
      // Record for 50s and perform real-time transcription
      while (System.currentTimeMillis() - start < 50000) {
        int read = targetDataLine.read(buffer.array(), 0, buffer.capacity());
        if (read > 0) {
          buffer.limit(read);
          // Send the recorded audio data to the streaming recognition service
          recognizer.sendAudioFrame(buffer);
          buffer = ByteBuffer.allocate(1024);
          // The recording rate is limited; sleep for a short while to prevent excessive CPU usage
          Thread.sleep(20);
        }
      }
      recognizer.stop();
    } catch (Exception e) {
      e.printStackTrace();
    } finally {
      // Close the WebSocket connection after the task is complete
      recognizer.getDuplexApi().close(1000, "bye");
    }

    System.out.println(
        "[Metric] requestId: "
            + recognizer.getLastRequestId()
            + ", first package delay ms: "
            + recognizer.getFirstPackageDelay()
            + ", last package delay ms: "
            + recognizer.getLastPackageDelay());
  }
}
Before you run the Python example, install the third-party audio playback and capture toolkit with pip install pyaudio. pyaudio requires the portaudio library. On Ubuntu/Debian: sudo apt-get install libportaudio2 portaudio19-dev. On macOS: brew install portaudio.

Recognize a local audio file

Recognize a local audio file and output the result. This suits shorter, near-real-time scenarios such as chat conversations, voice commands, voice input methods, and voice search.
import com.alibaba.dashscope.api.GeneralApi;
import com.alibaba.dashscope.audio.asr.recognition.Recognition;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionParam;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionResult;
import com.alibaba.dashscope.base.HalfDuplexParamBase;
import com.alibaba.dashscope.common.GeneralListParam;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.protocol.GeneralServiceOption;
import com.alibaba.dashscope.protocol.HttpMethod;
import com.alibaba.dashscope.protocol.Protocol;
import com.alibaba.dashscope.protocol.StreamingMode;
import com.alibaba.dashscope.utils.Constants;

import java.io.FileInputStream;
import java.nio.ByteBuffer;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;

class TimeUtils {
  private static final DateTimeFormatter formatter =
      DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS");

  public static String getTimestamp() {
    return LocalDateTime.now().format(formatter);
  }
}

public class Main {
  public static void main(String[] args) throws InterruptedException {
    Constants.baseWebsocketApiUrl = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference";
    // In real applications, this method only needs to be executed once at the very beginning of the program; there is no need to execute it multiple times.
    warmUp();

    ExecutorService executorService = Executors.newSingleThreadExecutor();
    executorService.submit(new RealtimeRecognitionTask(Paths.get(System.getProperty("user.dir"), "{YOUR_AUDIO_FILE}")));
    executorService.shutdown();

    // Wait for all tasks to complete.
    executorService.awaitTermination(1, TimeUnit.MINUTES);
    System.exit(0);
  }

  public static void warmUp() {
    try {
      // Lightweight GET request to establish connection
      GeneralServiceOption warmupOption = GeneralServiceOption.builder()
          .protocol(Protocol.HTTP)
          .httpMethod(HttpMethod.GET)
          .streamingMode(StreamingMode.OUT)
          .path("assistants")
          .build();

      warmupOption.setBaseHttpUrl(Constants.baseHttpApiUrl);
      GeneralApi<HalfDuplexParamBase> api = new GeneralApi<>();
      api.get(GeneralListParam.builder().limit(1L).build(), warmupOption);
    } catch (Exception e) {
      // Reset flag to allow retry if pre-warming failed
    }
  }
}

class RealtimeRecognitionTask implements Runnable {
  private Path filepath;

  public RealtimeRecognitionTask(Path filepath) {
    this.filepath = filepath;
  }

  @Override
  public void run() {
    RecognitionParam param = RecognitionParam.builder()
        .model("qwen-audio-3.0-asr-flash-streaming")
        // If you have not configured an environment variable, replace the following line with your API key: .apiKey("sk-xxx")
        .apiKey(System.getenv("DASHSCOPE_API_KEY"))
        .format("wav")
        .sampleRate(16000)
        .build();
    Recognition recognizer = new Recognition();

    String threadName = Thread.currentThread().getName();

    ResultCallback<RecognitionResult> callback = new ResultCallback<RecognitionResult>() {
      @Override
      public void onEvent(RecognitionResult message) {
        if (message.isSentenceEnd()) {

          System.out.println(TimeUtils.getTimestamp()+" "+
              "[process " + threadName + "] Final Result:" + message.getSentence().getText());
        } else {
          System.out.println(TimeUtils.getTimestamp()+" "+
              "[process " + threadName + "] Intermediate Result: " + message.getSentence().getText());
        }
      }

      @Override
      public void onComplete() {
        System.out.println(TimeUtils.getTimestamp()+" "+"[" + threadName + "] Recognition complete");
      }

      @Override
      public void onError(Exception e) {
        System.out.println(TimeUtils.getTimestamp()+" "+
            "[" + threadName + "] RecognitionCallback error: " + e.getMessage());
      }
    };

    try {
      recognizer.call(param, callback);
      // Please replace the path with your audio file path
      System.out.println(TimeUtils.getTimestamp()+" "+"[" + threadName + "] Input file_path is: " + this.filepath);
      // Read file and send audio by chunks
      FileInputStream fis = new FileInputStream(this.filepath.toFile());
      byte[] allData = new byte[fis.available()];
      int ret = fis.read(allData);
      fis.close();

      int sendFrameLength = 3200;
      for (int i = 0; i * sendFrameLength < allData.length; i ++) {
        int start = i * sendFrameLength;
        int end = Math.min(start + sendFrameLength, allData.length);
        ByteBuffer byteBuffer = ByteBuffer.wrap(allData, start, end - start);
        recognizer.sendAudioFrame(byteBuffer);
        Thread.sleep(100);
      }

      System.out.println(TimeUtils.getTimestamp()+" "+LocalDateTime.now());
      recognizer.stop();
    } catch (Exception e) {
      e.printStackTrace();
    } finally {
      // Close the WebSocket connection after the task is complete
      recognizer.getDuplexApi().close(1000, "bye");
    }

    System.out.println(
        "["
            + threadName
            + "][Metric] requestId: "
            + recognizer.getLastRequestId()
            + ", first package delay ms: "
            + recognizer.getFirstPackageDelay()
            + ", last package delay ms: "
            + recognizer.getLastPackageDelay());
  }
}

Advanced features

Get timestamps

Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime output timestamps at both the sentence level and the word level by default, which supports subtitle alignment, keyword highlighting, karaoke-style read-along, and similar scenarios. Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime) does not currently return timestamps. If you need timestamps, use Qwen-Audio-3.0-ASR-Flash-Streaming or Fun-ASR-Realtime. For file transcription, the recording-file transcription model qwen3-asr-flash-filetrans supports word-level timestamps. For details, see Non-real-time speech recognition. Timestamps are returned in milliseconds at two levels:
  • Sentence level: payload.output.sentence.begin_time and payload.output.sentence.end_time mark the start and end of a full sentence in the audio. In an intermediate result, end_time may be null and is filled with the final value when the sentence ends (sentence_end = true).
  • Word level: The payload.output.sentence.words array, where each element contains begin_time, end_time, text (the word or character text), and punctuation (the punctuation that follows the word, or an empty string if none).
The following excerpt shows the response structure:
{
  "payload": {
    "output": {
      "sentence": {
        "begin_time": 170,
        "end_time": 920,
        "text": "OK, I got it",
        "sentence_end": true,
        "words": [
          { "begin_time": 170, "end_time": 295, "text": "OK", "punctuation": "," },
          { "begin_time": 295, "end_time": 503, "text": "I", "punctuation": "" },
          { "begin_time": 503, "end_time": 711, "text": "got", "punctuation": "" },
          { "begin_time": 711, "end_time": 920, "text": "it", "punctuation": "" }
        ]
      }
    }
  }
}
The field names above follow the WebSocket JSON paths. Different SDKs expose these fields with their own naming conventions (dictionary keys, object properties, getter methods, and so on). For the complete field mapping, see Server events.

Emotion recognition

Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime) can include the speaker's emotional state in the transcription result. It is always on and requires no configuration. The emotion is returned through a top-level emotion field in both the conversation.item.input_audio_transcription.text and conversation.item.input_audio_transcription.completed events. The value is one of seven fine-grained emotions: surprised, neutral, happy, sad, disgusted, angry, and fearful.
{
  "type": "conversation.item.input_audio_transcription.text",
  "emotion": "neutral",
  "text": "The weather is nice today",
  "stash": ""
}
The field names above follow the WebSocket JSON paths. Different SDKs expose these fields with their own naming conventions (dictionary keys, object properties, getter methods, and so on). For the full field definitions, value constraints, and examples, see Server events.

Going live

Reuse connections (WebSocket)

WebSocket connections for Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime can be reused: after one recognition task finishes, you can start the next task without establishing a new connection. Reuse flow: The client sends finish-task. After the server returns task-finished, the client can send run-task again to start a new task.
  1. Wait for the server to return the task-finished event before starting a new task.
  2. Different tasks on a reused connection must use different task_id values.
  3. When a task fails, the server returns an error event and closes the connection. That connection cannot be reused.
  4. If no new task starts within 60 seconds after a task ends, the connection is closed automatically.
Qwen3-ASR-Flash-Realtime uses a session model. You must close the connection after each session ends, and connection reuse is not supported. For the events of each model, see the corresponding API reference.

High-concurrency best practices

The DashScope SDK includes a built-in pooling mechanism that reuses WebSocket connections and recognition objects, avoiding the overhead of frequent creation and destruction.
Only the Java SDK supports this feature.

Prerequisites

The Java SDK combines a built-in connection pool with a custom object pool for best performance:
  • Connection pool: The OkHttp3 connection pool integrated into the SDK manages and reuses the underlying WebSocket connections, reducing handshake overhead. It is enabled by default.
  • Object pool: Implemented with commons-pool2, it maintains a set of Recognition objects whose connections are already established. Borrowing an object from the pool removes the connection setup latency and significantly reduces first-packet latency.

Implementation steps

1. Add dependenciesAdd dashscope-sdk-java and commons-pool2 to your dependency configuration file, according to your build tool.
  • Maven
  • Gradle
Add the following dependencies inside the <dependencies> tag of pom.xml, then run mvn clean install or mvn compile to update the dependencies.
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>dashscope-sdk-java</artifactId>
    <!-- Replace 'the-latest-version' with 2.16.9 or later -->
    <version>the-latest-version</version>
</dependency>

<dependency>
    <groupId>org.apache.commons</groupId>
    <artifactId>commons-pool2</artifactId>
    <!-- Replace 'the-latest-version' with the latest version -->
    <version>the-latest-version</version>
</dependency>
2. Configure the connection poolConfigure the key connection pool parameters through environment variables:
Environment variableDescription
DASHSCOPE_CONNECTION_POOL_SIZEConnection pool size. Recommended: at least twice your peak concurrency. Default: 32.
DASHSCOPE_MAXIMUM_ASYNC_REQUESTSMaximum number of async requests. Recommended: the same as DASHSCOPE_CONNECTION_POOL_SIZE. Default: 32.
DASHSCOPE_MAXIMUM_ASYNC_REQUESTS_PER_HOSTMaximum number of async requests per host. Recommended: the same as DASHSCOPE_CONNECTION_POOL_SIZE. Default: 32.
3. Configure the object poolConfigure the object pool size through an environment variable:
Environment variableDescription
RECOGNITION_OBJECTPOOL_SIZEObject pool size. Recommended: 1.5 to 2 times your peak concurrency. Default: 500.
  • The object pool size (RECOGNITION_OBJECTPOOL_SIZE) must be less than or equal to the connection pool size (DASHSCOPE_CONNECTION_POOL_SIZE). Otherwise, when the object pool requests an object and the connection pool is full, the calling thread blocks.
  • The object pool size should not exceed your account's QPS limit.
Create the object pool with the following code:
class RecognitionObjectPool {
    // For the complete example, see the full code below
    public static GenericObjectPool<Recognition> getInstance() {
        lock.lock();
        if (recognitionGenericObjectPool == null) {
            int objectPoolSize = getObjectivePoolSize();
            RecognitionObjectFactory recognitionObjectFactory =
                    new RecognitionObjectFactory();
            GenericObjectPoolConfig<Recognition> config =
                    new GenericObjectPoolConfig<>();
            config.setMaxTotal(objectPoolSize);
            config.setMaxIdle(objectPoolSize);
            config.setMinIdle(objectPoolSize);
            recognitionGenericObjectPool =
                    new GenericObjectPool<>(recognitionObjectFactory, config);
        }
        lock.unlock();
        return recognitionGenericObjectPool;
    }
}
4. Borrow a Recognition object from the object poolWhen the number of unreturned objects exceeds the object pool limit, the system creates additional Recognition objects. These new objects must establish a new WebSocket connection and cannot be reused.
recognizer = RecognitionObjectPool.getInstance().borrowObject();
5. Run speech recognitionCall the call or streamCall method of the Recognition object to run speech recognition.6. Return the Recognition objectAfter the recognition task finishes, return the Recognition object so it can be reused. Do not return objects whose tasks are unfinished or failed.
RecognitionObjectPool.getInstance().returnObject(recognizer);

Full code

package com.example.speech;

import com.alibaba.dashscope.audio.asr.recognition.Recognition;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionParam;
import com.alibaba.dashscope.audio.asr.recognition.RecognitionResult;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.utils.ApiKey;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
import org.apache.commons.pool2.impl.DefaultPooledObject;
import org.apache.commons.pool2.impl.GenericObjectPool;
import org.apache.commons.pool2.impl.GenericObjectPoolConfig;

import java.io.FileInputStream;
import java.nio.ByteBuffer;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.Lock;
import com.alibaba.dashscope.utils.Constants;

public class Main {
    public static void checkoutEnv(String envName, int defaultSize) {
        if (System.getenv(envName) != null) {
            System.out.println("[ENV CHECK]: " + envName + " "
                    + System.getenv(envName));
        } else {
            System.out.println("[ENV CHECK]: " + envName
                    + " Using Default which is " + defaultSize);
        }
    }

    public static void main(String[] args)
            throws NoApiKeyException, InterruptedException {
        Constants.baseWebsocketApiUrl = "wss://dashscope-intl.aliyuncs.com/api-ws/v1/inference";
        checkoutEnv("DASHSCOPE_CONNECTION_POOL_SIZE", 32);
        checkoutEnv("DASHSCOPE_MAXIMUM_ASYNC_REQUESTS", 32);
        checkoutEnv("DASHSCOPE_MAXIMUM_ASYNC_REQUESTS_PER_HOST", 32);
        checkoutEnv(RecognitionObjectPool.RECOGNITION_OBJECTPOOL_SIZE_ENV,
                RecognitionObjectPool.DEFAULT_OBJECT_POOL_SIZE);

        int threadNums = 3;
        String currentDir = System.getProperty("user.dir");
        Path[] filePaths = {
                Paths.get(currentDir, "{YOUR_AUDIO_FILE}"),
                Paths.get(currentDir, "{YOUR_AUDIO_FILE}"),
                Paths.get(currentDir, "{YOUR_AUDIO_FILE}"),
        };
        ExecutorService executorService = Executors.newFixedThreadPool(threadNums);
        for (int i = 0; i < threadNums; i++) {
            executorService.submit(new RealtimeRecognizeTask(filePaths));
        }
        executorService.shutdown();
        executorService.awaitTermination(10, TimeUnit.MINUTES);
        System.exit(0);
    }
}

class RecognitionObjectFactory extends BasePooledObjectFactory<Recognition> {
    public RecognitionObjectFactory() {
        super();
    }

    @Override
    public Recognition create() throws Exception {
        return new Recognition();
    }

    @Override
    public PooledObject<Recognition> wrap(Recognition obj) {
        return new DefaultPooledObject<>(obj);
    }
}

class RecognitionObjectPool {
    public static GenericObjectPool<Recognition> recognitionGenericObjectPool;
    public static String RECOGNITION_OBJECTPOOL_SIZE_ENV =
            "RECOGNITION_OBJECTPOOL_SIZE";
    public static int DEFAULT_OBJECT_POOL_SIZE = 500;
    private static Lock lock = new java.util.concurrent.locks.ReentrantLock();

    public static int getObjectivePoolSize() {
        try {
            Integer n = Integer.parseInt(
                    System.getenv(RECOGNITION_OBJECTPOOL_SIZE_ENV));
            return n;
        } catch (NumberFormatException e) {
            return DEFAULT_OBJECT_POOL_SIZE;
        }
    }

    public static GenericObjectPool<Recognition> getInstance() {
        lock.lock();
        if (recognitionGenericObjectPool == null) {
            int objectPoolSize = getObjectivePoolSize();
            System.out.println("RECOGNITION_OBJECTPOOL_SIZE: "
                    + objectPoolSize);
            RecognitionObjectFactory recognitionObjectFactory =
                    new RecognitionObjectFactory();
            GenericObjectPoolConfig<Recognition> config =
                    new GenericObjectPoolConfig<>();
            config.setMaxTotal(objectPoolSize);
            config.setMaxIdle(objectPoolSize);
            config.setMinIdle(objectPoolSize);
            recognitionGenericObjectPool =
                    new GenericObjectPool<>(recognitionObjectFactory, config);
        }
        lock.unlock();
        return recognitionGenericObjectPool;
    }
}

class RealtimeRecognizeTask implements Runnable {
    private static final Object lock = new Object();
    private Path[] filePaths;

    public RealtimeRecognizeTask(Path[] filePaths) {
        this.filePaths = filePaths;
    }

    private static String getDashScopeApiKey() throws NoApiKeyException {
        String dashScopeApiKey = null;
        try {
            ApiKey apiKey = new ApiKey();
            dashScopeApiKey = ApiKey.getApiKey(null);
        } catch (NoApiKeyException e) {
            System.out.println("No API key found in environment.");
        }
        if (dashScopeApiKey == null) {
            dashScopeApiKey = "your-dashscope-apikey";
        }
        return dashScopeApiKey;
    }

    public void runCallback() {
        for (Path filePath : filePaths) {
            RecognitionParam param = null;
            try {
                param = RecognitionParam.builder()
                        .model("fun-asr-realtime")
                        .format("pcm")
                        .sampleRate(16000)
                        .apiKey(getDashScopeApiKey())
                        .build();
            } catch (Exception e) {
                throw new RuntimeException(e);
            }

            Recognition recognizer = null;
            final boolean[] hasError = {false};
            try {
                recognizer = RecognitionObjectPool.getInstance().borrowObject();
                String threadName = Thread.currentThread().getName();

                ResultCallback<RecognitionResult> callback =
                        new ResultCallback<RecognitionResult>() {
                            @Override
                            public void onEvent(RecognitionResult message) {
                                synchronized (lock) {
                                    if (message.isSentenceEnd()) {
                                        System.out.println("[process " + threadName
                                                + "] Fix:" + message.getSentence().getText());
                                    } else {
                                        System.out.println("[process " + threadName
                                                + "] Result: " + message.getSentence().getText());
                                    }
                                }
                            }

                            @Override
                            public void onComplete() {
                                System.out.println("[" + threadName
                                        + "] Recognition complete");
                            }

                            @Override
                            public void onError(Exception e) {
                                System.out.println("[" + threadName
                                        + "] RecognitionCallback error: " + e.getMessage());
                                hasError[0] = true;
                            }
                        };
                System.out.println("[" + threadName
                        + "] Input file_path is: " + filePath);
                FileInputStream fis = null;
                try {
                    fis = new FileInputStream(filePath.toFile());
                } catch (Exception e) {
                    System.out.println("Error when loading file: " + filePath);
                    e.printStackTrace();
                }
                recognizer.call(param, callback);

                // chunk size set to 100 ms for 16KHz sample rate
                byte[] buffer = new byte[3200];
                int bytesRead;
                while ((bytesRead = fis.read(buffer)) != -1) {
                    ByteBuffer byteBuffer;
                    if (bytesRead < buffer.length) {
                        byteBuffer = ByteBuffer.wrap(buffer, 0, bytesRead);
                    } else {
                        byteBuffer = ByteBuffer.wrap(buffer);
                    }
                    recognizer.sendAudioFrame(byteBuffer);
                    Thread.sleep(100);
                    buffer = new byte[3200];
                }
                System.out.println("[" + threadName + "] send audio done");
                recognizer.stop();
                System.out.println("[" + threadName + "] asr task finished");
            } catch (Exception e) {
                e.printStackTrace();
                hasError[0] = true;
            }
            if (recognizer != null) {
                try {
                    if (hasError[0] == true) {
                        recognizer.getDuplexApi().close(1000, "bye");
                        RecognitionObjectPool.getInstance()
                                .invalidateObject(recognizer);
                    } else {
                        RecognitionObjectPool.getInstance()
                                .returnObject(recognizer);
                    }
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        }
    }

    @Override
    public void run() {
        runCallback();
    }
}

Recommended configuration

The following configuration is based on test results from running only the real-time speech recognition service on servers of the specified sizes. Single-machine concurrency here means the number of real-time speech recognition tasks running at the same time (that is, the number of worker threads).
Machine sizeMax concurrency per machineObject pool sizeConnection pool size
4 cores, 8 GiB1005002000
8 cores, 16 GiB2005002000
16 cores, 32 GiB4005002000

Resource management and exception handling

  • Task succeeded: You must call GenericObjectPool.returnObject() to return the Recognition object to the pool for reuse.
    Do not return a Recognition object whose task is unfinished or failed.
  • Task failed: When an exception thrown by the SDK or your business logic interrupts the task, you must close the underlying WebSocket connection and invalidate the object in the pool so it is not used again.
// Close the connection
recognizer.getDuplexApi().close(1000, "bye");
// Invalidate the failed recognizer in the object pool
RecognitionObjectPool.getInstance().invalidateObject(recognizer);
  • No extra handling is needed when the service returns a TaskFailed error.

Warm-up and latency measurement

When you evaluate performance such as concurrent-call latency for the DashScope Java SDK, warm up thoroughly before the formal test.Connection reuse mechanismThe DashScope Java SDK manages and reuses WebSocket connections through a global singleton connection pool. The mechanism works as follows:
  • Created on demand: The SDK does not pre-create WebSocket connections at service startup. Connections are established on the first call.
  • Time-limited reuse: After a request completes, the connection stays in the pool for up to 60 seconds for reuse. A new request within 60 seconds reuses the existing connection and avoids another handshake. A connection idle for more than 60 seconds is closed automatically to release resources.
Why warm-up mattersIn the following cases, the connection pool may have no reusable active connection, so the request must create a new one:
  • The application has just started and has not made any calls yet.
  • The service has been idle for more than 60 seconds and pooled connections have timed out and closed.
In these cases, the first or early requests trigger the full WebSocket setup (TCP handshake, TLS negotiation, and protocol upgrade), so their end-to-end latency is significantly higher than requests that reuse a connection.Recommended practiceBefore formal load testing or latency measurement, follow these warm-up steps:
  1. Issue calls at the concurrency level of the formal test in advance (for example, for 1 to 2 minutes) to fill the connection pool.
  2. Confirm that the connection pool has established and maintains enough active connections, then start collecting formal performance data.

Improve recognition accuracy

  • Choose a model that matches the sample rate: For 8 kHz telephone audio, use an 8 kHz model directly. This avoids the information loss caused by upsampling to 16 kHz.
  • Use hotwords or context enhancement: For proprietary nouns, names, and brand names specific to your business, you can configure hotwords or context enhancement to significantly improve recognition accuracy. For detailed configuration methods and usage notes, see Improve recognition accuracy.
  • Improve the input audio quality: Use a high-quality microphone and record in an environment with a high signal-to-noise ratio and no echo. At the application layer, you can integrate algorithms such as noise reduction (for example, RNNoise) and acoustic echo cancellation (AEC) for preprocessing.
  • Specify the recognition language: For multilingual models, if you can predetermine the audio language when making a call, it helps the model converge and avoid confusion between similarly pronounced languages, which improves accuracy.

Set up a fault-tolerance strategy

  • Client-side reconnection: The client should implement automatic reconnection to handle network jitter. The following is a reference implementation for the Python SDK:
    1. Catch exceptions: Implement the on_error method in the Callback class. The dashscope SDK calls this method when it encounters a network error or another issue.
    2. Signal the state: When on_error is triggered, set a reconnection signal. In Python, you can use threading.Event, a thread-safe signal flag.
    3. Reconnection loop: Wrap the main logic in a for loop (for example, retry 3 times). When the reconnection signal is detected, the current recognition round is interrupted, resources are cleaned up, and after a few seconds the loop runs again to create a brand-new connection.
  • Set a heartbeat to keep the connection alive: To maintain a long-lived connection with the server, set the heartbeat parameter to true. The connection to the server then stays open even when the audio contains no sound for a long time.
  • Model rate limits: When you call the model API, note the model's Rate limiting rules.

Core usage: Context biasing (Qwen3-ASR-Flash-Realtime)

By providing context, you can optimize the recognition of domain-specific vocabulary, such as names, places, and product terms. Length limit: The context content cannot exceed 10,000 tokens. Usage:
  • WebSocket API: Set the session.input_audio_transcription.corpus.text parameter in the session.update event.
  • Python SDK: Set the corpus_text parameter.
  • Java SDK: Set the corpusText parameter.
Supported text types: These include but are not limited to:
  • Hotword lists in various separator formats, such as Hotword 1, Hotword 2, Hotword 3, Hotword 4
  • Text paragraphs or chapters of any format and length
  • Mixed content: Any combination of word lists and paragraphs
  • Irrelevant or meaningless text, including garbled text. The feature is highly fault-tolerant and is almost never negatively affected by irrelevant text.
Example: The correct transcription of an audio segment should be: "What internal jargon from the investment banking circle do you know? First, the nine major foreign investment banks, the Bulge Bracket, BB..."
Without context enhancementWith context enhancement
Without context enhancement, some investment bank names may be misrecognized. For example, "Bird Rock" should be "Bulge Bracket". Recognition result: "What internal jargon from the investment banking circle do you know? First, the nine major foreign investment banks, Bird Rock, BB..."With context enhancement, investment bank names are recognized correctly. Recognition result: "What internal jargon from the investment banking circle do you know? First, the nine major foreign investment banks, the Bulge Bracket, BB..."
To achieve the result above, add any of the following content to the context:
  • Word lists:
    • Word list 1:
Bulge Bracket, Boutique, Middle Market, domestic securities firms
  • Word list 2:
Bulge Bracket Boutique Middle Market domestic securities firms
  • Word list 3:
['Bulge Bracket', 'Boutique', 'Middle Market', 'domestic securities firms']
  • Natural language:
Investment Banking Categories Revealed!
Recently, many friends from Australia have asked me, what exactly is an investment bank? Today, I'll explain it. For international students, investment banks can be mainly divided into four categories: Bulge Bracket, Boutique, Middle Market, and domestic securities firms.
Bulge Bracket Investment Banks: These are what we often call the nine major investment banks, including Goldman Sachs, Morgan Stanley, etc. These large banks are enormous in both business scope and scale.
Boutique Investment Banks: These banks are relatively small but highly specialized in their business areas. For example, Lazard, Evercore, etc., have deep professional knowledge and experience in specific fields.
Middle Market Investment Banks: This type of bank mainly serves medium-sized companies, providing services such as mergers and acquisitions, and IPOs. Although not as large as the major banks, they have a high influence in specific markets.
Domestic Securities Firms: With the rise of the Chinese market, domestic securities firms are also playing an increasingly important role in the international market.
In addition, there are some Position and business divisions, you can refer to the relevant charts. I hope this information helps you better understand investment banking and prepare for your future career!
  • Natural language with interference: Some text is irrelevant to the recognition content, such as the names in the example below.
Investment Banking Categories Revealed!
Recently, many friends from Australia have asked me, what exactly is an investment bank? Today, I'll explain it. For international students, investment banks can be mainly divided into four categories: Bulge Bracket, Boutique, Middle Market, and domestic securities firms.
Bulge Bracket Investment Banks: These are what we often call the nine major investment banks, including Goldman Sachs, Morgan Stanley, etc. These large banks are enormous in both business scope and scale.
Boutique Investment Banks: These banks are relatively small but highly specialized in their business areas. For example, Lazard, Evercore, etc., have deep professional knowledge and experience in specific fields.
Middle Market Investment Banks: This type of bank mainly serves medium-sized companies, providing services such as mergers and acquisitions, and IPOs. Although not as large as the major banks, they have a high influence in specific markets.
Domestic Securities Firms: With the rise of the Chinese market, domestic securities firms are also playing an increasingly important role in the international market.
In addition, there are some Position and business divisions, you can refer to the relevant charts. I hope this information helps you better understand investment banking and prepare for your future career!
Wang Haoxuan, Li Zihan, Zhang Jingxing, Liu Xinyi, Chen Junjie, Yang Siyuan, Zhao Yutong, Huang Zhiqiang, Zhou Zimo, Wu Yajing, Xu Ruoxi, Sun Haoran, Hu Jinyu, Zhu Chenxi, Guo Wenbo, He Jingshu, Gao Yuhang, Lin Yifei,
Zheng Xiaoyan, Liang Bowen, Luo Jiaqi, Song Mingzhe, Xie Wanting, Tang Ziqian, Han Mengyao, Feng Yiran, Cao Qinxue, Deng Zirui, Xiao Wangshu, Xu Jiashu,
Cheng Yinuo, Yuan Zhiruo, Peng Haoyu, Dong Simiao, Fan Jingyu, Su Zijin, Lv Wenxuan, Jiang Shihan, Ding Muchen,
Wei Shuyao, Ren Tianyou, Jiang Yichen, Hua Qingyu, Shen Xinghe, Fu Jinyu, Yao Xingchen, Zhong Lingyu, Yan Licheng, Jin Ruoshui, Taoranting, Qi Shaoshang, Xue Zhilan, Zou Yunfan, Xiong Ziang, Bai Wenfeng, Yi Qianfan

Core usage: Sensitive word filtering (Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime)

Sensitive word filtering replaces or removes sensitive words in the recognition result. Use it for call-center quality inspection, content compliance, subtitle review, and similar scenarios. Supported models: Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime only. Limit: You can set up to 32 sensitive words. Default behavior: When the special_word_filter parameter is not passed, no sensitive words are filtered. How to configure: special_word_filter is a JSON object with three subfields:
  • filter_with_signed.word_list: A string array that lists the sensitive words to replace with an equal-length string of * characters. For example, with ["test"], "Help me test it" becomes "Help me **** it".
  • filter_with_empty.word_list: A string array that lists the sensitive words to remove entirely from the result. For example, with ["start"], "Is the game about to start" becomes "Is the game about to".
  • system_reserved_filter: A boolean that defaults to false. It determines whether sensitive word filtering is enabled.
Configuration example:
{
  "special_word_filter": {
    "filter_with_signed": {
      "word_list": ["test"]
    },
    "filter_with_empty": {
      "word_list": ["start", "occur"]
    },
    "system_reserved_filter": true
  }
}
Different SDKs expose these parameters with their own naming conventions (dictionary keys, object properties, methods, and so on). For the complete field mapping, see the API reference.

VAD segmentation configuration

Voice Activity Detection (VAD) determines when a continuous segment of speech ends, which triggers the final recognition result event. All three model families enable server-side VAD by default, but their parameter names and tuning granularity differ:
  • Qwen-Audio-3.0-ASR-Flash-Streaming / Fun-ASR-Realtime / Paraformer: Configured through max_sentence_silence (the VAD silence threshold for segmentation, in milliseconds). When the silence after a segment of speech exceeds this threshold, the system treats the sentence as complete.
  • Qwen3-ASR-Flash-Realtime: Configured through session.turn_detection, which includes silence_duration_ms (the silence duration threshold that ends a turn when exceeded; server default 800, with 400 recommended for conversation and chat scenarios that need fast segmentation) and threshold (VAD detection sensitivity; server default 0.2). Qwen3-ASR-Flash-Realtime also supports Manual mode, which disables VAD and uses client-side commit for segmentation. For details, see the Interaction flow section below.
Parameter names vary by protocol: the same concept is called max_sentence_silence in Qwen-Audio-3.0-ASR-Flash-Streaming / Fun-ASR-Realtime / Paraformer, and silence_duration_ms in Qwen3-ASR-Flash-Realtime. For the full field definitions, see the API reference below.

Emotion recognition

Qwen3-ASR-Flash-Realtime and some Paraformer models can include the speaker's emotional state in the transcription result, but the two differ in output granularity and in how the feature is enabled. Qwen3-ASR-Flash-Realtime (qwen3-asr-flash-realtime): Always on, no configuration required. The emotion is returned through a top-level emotion field in both the conversation.item.input_audio_transcription.text and conversation.item.input_audio_transcription.completed events. The value is one of seven fine-grained emotions: surprised, neutral, happy, sad, disgusted, angry, and fearful.
{
  "type": "conversation.item.input_audio_transcription.text",
  "emotion": "neutral",
  "text": "The weather is nice today",
  "stash": ""
}

API reference

  • Qwen-Audio-3.0-ASR-Flash-Streaming/Fun-ASR-Realtime
  • Qwen3-ASR-Flash-Realtime

Interaction flow (Qwen3-ASR-Flash-Realtime)

Qwen real-time speech recognition streams audio over WebSocket. Two modes are available: VAD mode (default) and Manual mode.

URL

Replace <model_name> with your model name.
wss://dashscope-intl.aliyuncs.com/api-ws/v1/realtime?model=<model_name>

Headers

"Authorization": "Bearer $DASHSCOPE_API_KEY"

VAD mode (default)

The server detects speech boundaries and segments sentences. The client streams audio, and the server returns results when each sentence ends. Best for conversations and meeting transcription. Enable: Set session.turn_detection in session.update.
VAD mode interaction flow

Manual mode

The client controls sentence segmentation by sending audio for a complete sentence, then sending input_audio_buffer.commit. Best when the client knows sentence boundaries, for example in chat app voice messages. Enable: Set session.turn_detection to null in session.update.
Manual mode interaction flow

Alternative: Use Qwen-Omni

You can also use Qwen-Omni (qwen3-omni-flash-realtime) for real-time speech recognition via WebSocket. Omni is an LLM that understands audio — you provide domain context through the system prompt instead of hotword lists. When to use Omni for ASR: Clean speech inputs (microphone, voice calls) where you need domain-specific terminology handling via prompt. When to use dedicated ASR models instead: Noisy or mixed audio (meetings with background music, videos with sound effects), or when you need hotwords, speaker diarization, or timestamps.
Qwen-Omni interprets all audio, not just speech. Music, typing, or ambient noise may produce descriptions instead of transcription. For mixed audio, preprocess with VAD to isolate speech, or use a dedicated ASR model.
ASR prompt template:
messages = [
  {"role": "system", "content": "Transcribe the following audio exactly as spoken. Output only the transcription text. Ignore non-speech sounds."},
  {"role": "user", "content": [{"type": "input_audio", "input_audio": {"data": audio_data, "format": "wav"}}]}
]
Qwen-Omni-Realtime uses WebSocket for bidirectional streaming. For the full API and SDK reference, see Realtime conversation.

FAQ

Which audio formats does real-time speech recognition support?

Qwen-Audio-3.0-ASR-Flash-Streaming and Fun-ASR-Realtime support pcm, wav, mp3, opus, speex, aac, and amr. For Qwen3-ASR-Flash-Realtime, use pcm or opus. Other formats such as wav, aac, and amr pass the session.update validation layer, but server-side decoding may fail. Confirm that the audio stream uses a recommended format before sending it.

What's the difference between the SDK and the WebSocket API, and how do I choose?

The DashScope SDK encapsulates WebSocket connection management, authentication, reconnection, and other details, which makes it suitable for quick integration. Connecting to the WebSocket API directly gives you finer-grained control and suits programming languages the SDK does not cover or scenarios that need custom connection management. We recommend the SDK.

How do I improve recognition accuracy for proper nouns?

Use hotwords or context enhancement. For detailed configuration methods and usage notes, see Improve recognition accuracy.

What should I do when the connection drops frequently?

Implement client-side reconnection and enable the heartbeat parameter (heartbeat=true) to prevent the connection from dropping when there is no audio for a long time. For the full fault-tolerance strategy, see Set up a fault-tolerance strategy.