Skip to main content

WebSocket connection guide

Qwen-ASR-Realtime receives audio streams and transcribes speech in real time over WebSocket. The service supports two interaction modes: VAD mode and Manual mode .

User guide: For model overviews and selection guidance, see Speech-to-text. For sample code, see Real-time speech recognition. Rate limits: Model calls are subject to rate limits. When a limit is exceeded, the server returns the error Requests rate limit exceeded, please try again later. Reduce your request rate or concurrency and retry. For the rate limits of each model, see Rate limiting.

Service endpoint

Specify the model with the model query parameter: wss://ap-southeast-1.maas.qwencloudapi.com/api-ws/v1/realtime?model=<model_name> This example uses Singapore (ap-southeast-1). For another region, replace ap-southeast-1 in the URL, for example cn-beijing for China North 2 (Beijing), and use an API key from that region. See Regions.
Use wss:// and pass Authorization: Bearer $DASHSCOPE_API_KEY in the WebSocket handshake.

Request headers

Include the following fields in the request headers:
ParameterTypeRequiredDescription
AuthorizationstringYesAuthentication token in the format Bearer $DASHSCOPE_API_KEY. Replace $DASHSCOPE_API_KEY with your API key.
user-agentstringNoClient identifier that helps the server track request sources.
X-DashScope-WorkSpacestringNoThe QwenCloud workspace ID.
X-DashScope-DataInspectionstringNoWhether to enable data inspection. Omit this header unless data inspection is required; if it is, set the value to enable.
Authorization is verified during the WebSocket handshake. If the API key is invalid or missing, the handshake fails with an HTTP 401 or 403 error.

Interaction flows

For details about client and server events, see Client events for Qwen-ASR-Realtime and Server events. Qwen-ASR-Realtime supports two interaction modes:
  • VAD mode (default): The server uses voice activity detection (VAD) to automatically detect the start and end of each utterance. Use this mode for real-time conversations, meeting transcription, and similar scenarios.
  • Manual mode: The client controls utterance boundaries. Use this mode when the client can determine those boundaries explicitly, such as sending a voice message in a messaging app.
This model does not return sentence-level or word-level timestamps for the transcript. Speech start and end times in VAD events describe voice activity boundaries; see those events for the field definitions.

VAD mode (default)

The server default for silence_duration_ms is 800; 400 is a reference setting for conversations that need faster segmentation. The server default for threshold is 0.2. For the full configuration, see Client events. The server automatically detects the start and end of each utterance. Send the audio stream continuously; the server returns the final transcription for each utterance once it detects the end. Use this mode for real-time conversations, meeting transcription, and similar scenarios. To enable: Configure the session.turn_detection parameter in the client session.update event.
image
In VAD mode, the client must send a session.finish event before closing the connection. If the client closes the WebSocket connection without first sending session.finish, the server discards the current in-progress item and events such as conversation.item.input_audio_transcription.completed will not be delivered. Send {"type":"session.finish"} before calling ws.Close(), and wait for the session.finished event before closing the connection.

Manual mode

The client controls utterance boundaries. After sending the audio for a complete utterance, the client sends a input_audio_buffer.commit event to notify the server. Use this mode when the client can determine those boundaries explicitly, such as sending a voice message in a messaging app. To enable: Set session.turn_detection to null in the client session.update event.
image

Examples

These examples use qwen3-asr-flash-realtime. Get your API key and Base URL from the console, and set them as DASHSCOPE_API_KEY and DASHSCOPE_BASE_URL. Download the sample audio as asr_example.wav: a 16 kHz, mono WAV file with 16-bit PCM encoding.
The example code reads asr_example.pcm (PCM16, 16 kHz, mono). If you only have an MP3, WAV, or similar format, convert it with ffmpeg:
ffmpeg -i asr_example.wav -ar 16000 -ac 1 -f s16le asr_example.pcm
  • Java
  • Python
  • Go
  • Node.js
  • C#
  • PHP
Before you run the example, install the Java-WebSocket dependency:
Maven
<dependency>
    <groupId>org.java-websocket</groupId>
    <artifactId>Java-WebSocket</artifactId>
    <version>1.5.6</version>
</dependency>
<dependency>
    <groupId>org.json</groupId>
    <artifactId>json</artifactId>
    <version>20240303</version>
</dependency>
import org.java_websocket.client.WebSocketClient;
import org.java_websocket.handshake.ServerHandshake;
import org.json.JSONObject;

import java.net.URI;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.Base64;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.logging.*;

public class QwenASRRealtimeClient {

    private static final Logger logger = Logger.getLogger(QwenASRRealtimeClient.class.getName());
    private static final String API_KEY = System.getenv("DASHSCOPE_API_KEY");
    private static final String MODEL = "qwen3-asr-flash-realtime";

    // Controls whether to use VAD mode
    private static final boolean enableServerVad = true;

    private static final AtomicBoolean isRunning = new AtomicBoolean(true);
    private static WebSocketClient client;

    public static void main(String[] args) throws Exception {
        initLogger();

        String baseUrl = "wss://" + URI.create(System.getenv("DASHSCOPE_BASE_URL")).getHost() + "/api-ws/v1/realtime";
        String url = baseUrl + "?model=" + MODEL;
        logger.info("Connecting to server: " + url);

        client = new WebSocketClient(new URI(url)) {
            @Override
            public void onOpen(ServerHandshake handshake) {
                logger.info("Connected to server.");
                sendSessionUpdate();
            }

            @Override
            public void onMessage(String message) {
                try {
                    JSONObject data = new JSONObject(message);
                    String eventType = data.optString("type");

                    logger.info("Received event: " + data.toString(2));

                    // The final recognition result is in the transcription.completed event
                    if ("conversation.item.input_audio_transcription.completed".equals(eventType)) {
                        logger.info("Final transcript: " + data.optString("transcript"));
                    }

                    // On receiving the finished event -> stop the sending thread and close the connection
                    if ("session.finished".equals(eventType)) {
                        logger.info("Closing WebSocket connection after session finished...");

                        isRunning.set(false); // Stop the audio sending thread
                        if (this.isOpen()) {
                            this.close(1000, "ASR finished");
                        }
                    }
                } catch (Exception e) {
                    logger.severe("Failed to parse message: " + message);
                }
            }

            @Override
            public void onClose(int code, String reason, boolean remote) {
                logger.info("Connection closed: " + code + " - " + reason);
            }

            @Override
            public void onError(Exception ex) {
                logger.severe("Error: " + ex.getMessage());
            }
        };

        // Add request headers
        client.addHeader("Authorization", "Bearer " + API_KEY);
        client.addHeader("OpenAI-Beta", "realtime=v1");

        client.connectBlocking(); // Block until the connection is established

        // Replace with the path to the audio file to be recognized
        String localAudioPath = "asr_example.pcm";
        Thread audioThread = new Thread(() -> {
            try {
                sendAudio(localAudioPath);
            } catch (Exception e) {
                logger.severe("Audio sending thread error: " + e.getMessage());
            }
        });
        audioThread.start();
    }

    /** Session update event (enable/disable VAD) */
    private static void sendSessionUpdate() {
        JSONObject eventNoVad = new JSONObject()
                .put("event_id", "event_123")
                .put("type", "session.update")
                .put("session", new JSONObject()
                        .put("modalities", new String[]{"text"})
                        .put("input_audio_format", "pcm")
                        .put("sample_rate", 16000)
                        // .put("input_audio_transcription", new JSONObject()
                        //         .put("language", "zh"))
                        .put("turn_detection", JSONObject.NULL) // Manual mode
                );

        JSONObject eventVad = new JSONObject()
                .put("event_id", "event_123")
                .put("type", "session.update")
                .put("session", new JSONObject()
                        .put("modalities", new String[]{"text"})
                        .put("input_audio_format", "pcm")
                        .put("sample_rate", 16000)
                        // .put("input_audio_transcription", new JSONObject()
                        //         .put("language", "zh"))
                        .put("turn_detection", new JSONObject()
                                .put("type", "server_vad")
                                .put("threshold", 0.2)
                                .put("silence_duration_ms", 400))
                );

        if (enableServerVad) {
            logger.info("Sending event (VAD):\n" + eventVad.toString(2));
            client.send(eventVad.toString());
        } else {
            logger.info("Sending event (Manual):\n" + eventNoVad.toString(2));
            client.send(eventNoVad.toString());
        }
    }

    /** Send the audio file stream */
    private static void sendAudio(String localAudioPath) throws Exception {
        Thread.sleep(3000); // Wait for the session to be ready
        byte[] allBytes = Files.readAllBytes(Paths.get(localAudioPath));
        logger.info("File reading started");

        int offset = 0;
        while (isRunning.get() && offset < allBytes.length) {
            int chunkSize = Math.min(3200, allBytes.length - offset);
            byte[] chunk = new byte[chunkSize];
            System.arraycopy(allBytes, offset, chunk, 0, chunkSize);
            offset += chunkSize;

            if (client != null && client.isOpen()) {
                String encoded = Base64.getEncoder().encodeToString(chunk);
                JSONObject eventd = new JSONObject()
                        .put("event_id", "event_" + System.currentTimeMillis())
                        .put("type", "input_audio_buffer.append")
                        .put("audio", encoded);

                client.send(eventd.toString());
                logger.info("Sending audio event: " + eventd.getString("event_id"));
            } else {
                break; // Avoid continuing to send after disconnection
            }

            Thread.sleep(100); // Simulate real-time sending
        }

        logger.info("File reading finished");

        if (client != null && client.isOpen()) {
            // Commit is required in non-VAD mode
            if (!enableServerVad) {
                JSONObject commitEvent = new JSONObject()
                        .put("event_id", "event_789")
                        .put("type", "input_audio_buffer.commit");
                client.send(commitEvent.toString());
                logger.info("Sent commit event for manual mode.");
            }

            JSONObject finishEvent = new JSONObject()
                    .put("event_id", "event_987")
                    .put("type", "session.finish");
            client.send(finishEvent.toString());
            logger.info("Sent finish event.");
        }
    }

    /** Initialize logging */
    private static void initLogger() {
        logger.setLevel(Level.ALL);
        Logger rootLogger = Logger.getLogger("");
        for (Handler h : rootLogger.getHandlers()) {
            rootLogger.removeHandler(h);
        }

        Handler consoleHandler = new ConsoleHandler();
        consoleHandler.setLevel(Level.ALL);
        consoleHandler.setFormatter(new SimpleFormatter());
        logger.addHandler(consoleHandler);
    }
}