Skip to main content
Qwen-TTS-Realtime

WebSocket connection guide

Connect to the Qwen-TTS-Realtime real-time speech synthesis service over WebSocket. Covers the service endpoint, request headers, and interaction flow.

For model overview and selection guidance, see Real-time speech synthesis The Qwen-TTS Realtime API uses the WebSocket protocol. Java and Python applications can use the DashScope SDK to skip WebSocket handling. Other languages can connect with a WebSocket library directly.

Service endpoint

Use the following URL. Set the model with the model query parameter: WebSocket URL (Singapore): wss://ap-southeast-1.maas.qwencloudapi.com/api-ws/v1/realtime?model=qwen3-tts-flash-realtime For another region, replace ap-southeast-1 in the URL with that region's code. API keys are region-specific. See Regions.
The URL must use the wss:// protocol. Set the Authorization header (see Request headers) and specify the model using the model query parameter.

Request headers

Set the following request headers:
ParameterTypeRequiredDescription
AuthorizationstringYesFormat: Bearer $DASHSCOPE_API_KEY. Replace $DASHSCOPE_API_KEY with your API key.
user-agentstringNoClient identifier for server-side source tracking.
X-DashScope-WorkSpacestringNoThe QwenCloud workspace ID.
Authorization is verified during the WebSocket handshake. An invalid or missing API key causes the handshake to fail with HTTP 401/403.

Interaction flow

For detailed descriptions of client events and server events, see Client events and Server events. The API supports two modes:
  • ServerCommit mode: The server determines text segmentation and synthesis timing automatically.
  • Commit mode: The client controls when to submit each text segment. Call input_text_buffer.commit explicitly to trigger synthesis.

Mode details

  • In ServerCommit mode, call input_text_buffer.append to buffer text. The system determines synthesis timing automatically.
  • Calling input_text_buffer.commit in ServerCommit mode synthesizes the current buffer immediately. The session then resumes ServerCommit mode.
  • In Commit mode, input_text_buffer.append alone does not trigger synthesis. Call input_text_buffer.commit to start synthesis.
qwen-tts
Key steps:
  1. Connection: The client opens a WebSocket connection. The server returns session.created when ready.
  2. Session configuration: The client sends session.update to configure voice, format, and mode.
  3. Text input: The client sends input_text_buffer.append to add text to the buffer.
  4. Synthesis trigger:
    • In ServerCommit mode, synthesis starts automatically. The client can call input_text_buffer.commit to force immediate synthesis.
    • In Commit mode, only input_text_buffer.commit triggers synthesis.
  5. Audio generation: The server sends response.created, then streams base64-encoded audio as response.audio.delta events until response.audio.done.
  6. Session termination: The client sends session.finish to release resources. The server returns session.finished and closes the connection.
After connection, the server returns this session.created event:
{
    "event_id": "event_xxx",
    "type": "session.created",
    "session": {
        "object": "realtime.session",
        "mode": "server_commit",
        "model": "qwen3-tts-flash-realtime",
        "voice": "Cherry",
        "response_format": "pcm",
        "sample_rate": 24000,
        "id": "sess_xxx"
    }
}

Code examples

The examples provide Python and Java clients and both commit modes. Set DASHSCOPE_API_KEY for the region used by the example endpoint. Python requires websockets and pyaudio; Java requires Java-WebSocket and Gson and uses Java Sound for playback. Each client event uses a unique UUID.
Java
<dependencies>
    <dependency>
        <groupId>org.java-websocket</groupId>
        <artifactId>Java-WebSocket</artifactId>
        <version>1.5.3</version>
    </dependency>
    <dependency>
        <groupId>com.google.code.gson</groupId>
        <artifactId>gson</artifactId>
        <version>2.10.1</version>
    </dependency>
</dependencies>
  1. Create the client
  • Java
  • Python
Create a Java file named TTSRealtimeClient.java and copy the following code into the file:
import com.google.gson.Gson;
import com.google.gson.JsonObject;
import org.java_websocket.client.WebSocketClient;
import org.java_websocket.handshake.ServerHandshake;

import java.net.URI;
import java.util.Base64;
import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.CountDownLatch;
import java.util.function.Consumer;

/**
 * A client for interacting with the TTS Realtime API.
 *
 * This class provides methods for connecting to the TTS Realtime API, sending text data, retrieving audio output, and managing WebSocket connections.
 */
public class TTSRealtimeClient {

    public enum SessionMode {
        SERVER_COMMIT("server_commit"),
        COMMIT("commit");
        private final String value;
        SessionMode(String value) { this.value = value; }
        public String getValue() { return value; }
    }

    /**
     * Audio callback interface
     */
    public interface AudioCallback {
        void onAudio(byte[] audioData);
    }

    private final String baseUrl;
    private final String apiKey;
    private final String voice;
    private final SessionMode mode;
    private final String languageType;
    private final AudioCallback audioCallback;
    private final Gson gson = new Gson();

    private WebSocketClient ws;
    private CountDownLatch responseDoneLatch;
    private CountDownLatch sessionFinishedLatch;

    public TTSRealtimeClient(String baseUrl, String apiKey, String voice,
                             SessionMode mode, AudioCallback audioCallback,
                             String languageType) {
        this.baseUrl = baseUrl;
        this.apiKey = apiKey;
        this.voice = voice;
        this.mode = mode;
        this.audioCallback = audioCallback;
        this.languageType = languageType;
    }

    public TTSRealtimeClient(String baseUrl, String apiKey, String voice,
                             SessionMode mode, AudioCallback audioCallback) {
        this(baseUrl, apiKey, voice, mode, audioCallback, "Auto");
    }

    /**
     * Establish a WebSocket connection to the TTS Realtime API.
     */
    public void connect() throws Exception {
        Map<String, String> headers = new HashMap<>();
        headers.put("Authorization", "Bearer " + apiKey);

        responseDoneLatch = new CountDownLatch(0);
        sessionFinishedLatch = new CountDownLatch(1);

        ws = new WebSocketClient(new URI(baseUrl), headers) {
            @Override
            public void onOpen(ServerHandshake handshake) {
                System.out.println("WebSocket connection established");
                // Send default session configuration
                JsonObject session = new JsonObject();
                session.addProperty("mode", mode.getValue());
                session.addProperty("voice", TTSRealtimeClient.this.voice);
                session.addProperty("language_type", languageType);
                session.addProperty("response_format", "pcm");
                session.addProperty("sample_rate", 24000);
                updateSession(session);
            }

            @Override
            public void onMessage(String message) {
                JsonObject event = gson.fromJson(message, JsonObject.class);
                String eventType = event.has("type") ? event.get("type").getAsString() : "";

                if (!"response.audio.delta".equals(eventType)) {
                    System.out.println("Received event: " + eventType);
                }

                switch (eventType) {
                    case "error":
                        System.err.println("Error: " + event.get("error"));
                        break;
                    case "session.created":
                        System.out.println("Session created, ID: " +
                            event.getAsJsonObject("session").get("id").getAsString());
                        break;
                    case "session.updated":
                        System.out.println("Session updated, ID: " +
                            event.getAsJsonObject("session").get("id").getAsString());
                        break;
                    case "input_text_buffer.committed":
                        System.out.println("Text buffer committed, item ID: " + event.get("item_id"));
                        break;
                    case "input_text_buffer.cleared":
                        System.out.println("Text buffer cleared");
                        break;
                    case "response.created":
                        System.out.println("Response created, ID: " +
                            event.getAsJsonObject("response").get("id").getAsString());
                        responseDoneLatch = new CountDownLatch(1);
                        break;
                    case "response.output_item.added":
                        System.out.println("Output item added, ID: " +
                            event.getAsJsonObject("item").get("id").getAsString());
                        break;
                    case "response.audio.delta":
                        if (audioCallback != null) {
                            byte[] audioBytes = Base64.getDecoder().decode(
                                event.get("delta").getAsString());
                            audioCallback.onAudio(audioBytes);
                        }
                        break;
                    case "response.audio.done":
                        System.out.println("Audio generation completed");
                        break;
                    case "response.done":
                        System.out.println("Response completed");
                        responseDoneLatch.countDown();
                        break;
                    case "session.finished":
                        System.out.println("Session finished");
                        sessionFinishedLatch.countDown();
                        break;
                }
            }

            @Override
            public void onClose(int code, String reason, boolean remote) {
                System.out.println("Connection closed: " + reason);
            }

            @Override
            public void onError(Exception ex) {
                System.err.println("WebSocket error: " + ex.getMessage());
            }
        };
        ws.connectBlocking();
    }

    /**
     * Send an event to the server.
     */
    public void sendEvent(JsonObject event) {
        String eventId = "event_" + java.util.UUID.randomUUID().toString();
        event.addProperty("event_id", eventId);
        System.out.println("Sending event: type=" + event.get("type").getAsString()
            + ", event_id=" + eventId);
        ws.send(gson.toJson(event));
    }

    /**
     * Update the session configuration.
     */
    public void updateSession(JsonObject config) {
        JsonObject event = new JsonObject();
        event.addProperty("type", "session.update");
        event.add("session", config);
        System.out.println("Updating session configuration: " + event);
        sendEvent(event);
    }

    /**
     * Send text data to the API.
     */
    public void appendText(String text) {
        JsonObject event = new JsonObject();
        event.addProperty("type", "input_text_buffer.append");
        event.addProperty("text", text);
        sendEvent(event);
    }

    /**
     * Commit the text buffer to trigger processing.
     */
    public void commitTextBuffer() {
        JsonObject event = new JsonObject();
        event.addProperty("type", "input_text_buffer.commit");
        sendEvent(event);
    }

    /**
     * Clear the text buffer.
     */
    public void clearTextBuffer() {
        JsonObject event = new JsonObject();
        event.addProperty("type", "input_text_buffer.clear");
        sendEvent(event);
    }

    /**
     * End the session.
     */
    public void finishSession() {
        JsonObject event = new JsonObject();
        event.addProperty("type", "session.finish");
        sendEvent(event);
    }

    /**
     * Wait for the response.done event.
     */
    public void waitForResponseDone() throws InterruptedException {
        responseDoneLatch.await();
    }

    /**
     * Wait for the session.finished event.
     */
    public void waitForSessionFinished() throws InterruptedException {
        sessionFinishedLatch.await();
    }

    /**
     * Close the WebSocket connection.
     */
    public void close() {
        if (ws != null) {
            ws.close();
        }
    }
}
  1. Select a speech synthesis mode Realtime API supports two modes:
    • server_commit mode The server handles text segmentation and synthesis timing automatically. The client only sends text. Suited for low-latency scenarios (such as GPS navigation).
    • commit mode The client adds text to a buffer and explicitly triggers synthesis. Suited for scenarios that require precise sentence segmentation control (such as news broadcast).
  • server_commit mode
  • commit mode
  • Java
  • Python
In the same directory as TTSRealtimeClient.java, create another Java file named ServerCommit.java and copy the following code into the file:
import javax.sound.sampled.*;
import java.io.*;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicBoolean;

public class ServerCommit {
    // Singapore (ap-southeast-1). For another region, replace ap-southeast-1 in the URL.
    // API keys are region-specific. See https://docs.qwencloud.com/developer-guides/getting-started/regions
    private static final String URL = "wss://ap-southeast-1.maas.qwencloudapi.com/api-ws/v1/realtime?model=qwen3-tts-flash-realtime";
    // If you have not configured the environment variable, replace the next line with your QwenCloud API Key: private static final String API_KEY = "sk-xxx";
    private static final String API_KEY = System.getenv("DASHSCOPE_API_KEY");
    private static final int SAMPLE_RATE = 24000;

    // Audio data cache
    private static final List<byte[]> audioChunks = new ArrayList<>();
    // Real-time playback queue
    private static final ConcurrentLinkedQueue<byte[]> playbackQueue = new ConcurrentLinkedQueue<>();
    private static final AtomicBoolean playing = new AtomicBoolean(true);

    public static void main(String[] args) throws Exception {
        if (API_KEY == null || API_KEY.isEmpty()) {
            throw new IllegalStateException("Please set the DASHSCOPE_API_KEY environment variable");
        }

        // Initialize audio playback
        AudioFormat format = new AudioFormat(SAMPLE_RATE, 16, 1, true, false);
        DataLine.Info info = new DataLine.Info(SourceDataLine.class, format);
        SourceDataLine audioLine = (SourceDataLine) AudioSystem.getLine(info);
        audioLine.open(format);
        audioLine.start();

        // Start the playback thread
        Thread playerThread = new Thread(() -> {
            while (playing.get() || !playbackQueue.isEmpty()) {
                byte[] chunk = playbackQueue.poll();
                if (chunk != null) {
                    audioLine.write(chunk, 0, chunk.length);
                } else {
                    try { Thread.sleep(10); } catch (InterruptedException ignored) {}
                }
            }
        });
        playerThread.start();

        // Create the TTS client
        TTSRealtimeClient client = new TTSRealtimeClient(
            URL, API_KEY, "Cherry",
            TTSRealtimeClient.SessionMode.SERVER_COMMIT,
            audioData -> {
                playbackQueue.add(audioData);
                audioChunks.add(audioData);
                System.out.println("Received audio data: " + audioData.length + " bytes");
            }
        );

        client.connect();

        // Send text fragments
        String[] textFragments = {
            "QwenCloud is an all-in-one platform for developing and building large language model applications.",
            "Both developers and business users can deeply participate in the design and development of large language model applications.",
            "You can develop a large language model application in five minutes using a simple interface,",
            "or train a dedicated model in a few hours, allowing you to focus more energy on application innovation."
        };

        System.out.println("Starting to send text...");
        for (String text : textFragments) {
            System.out.println("Sending fragment: " + text);
            client.appendText(text);
            Thread.sleep(100);
        }

        Thread.sleep(1000);
        client.finishSession();

        // Wait for the response to complete
        client.waitForResponseDone();
        client.waitForSessionFinished();
        client.close();

        // Wait for playback to complete
        playing.set(false);
        playerThread.join();
        audioLine.drain();
        audioLine.close();

        // Save the audio file
        saveWav("output.wav");
        System.out.println("Done");
    }

    private static void saveWav(String filename) throws IOException {
        if (audioChunks.isEmpty()) {
            System.out.println("No audio data to save");
            return;
        }
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        for (byte[] chunk : audioChunks) {
            bos.write(chunk);
        }
        byte[] allAudio = bos.toByteArray();
        AudioFormat format = new AudioFormat(SAMPLE_RATE, 16, 1, true, false);
        AudioInputStream ais = new AudioInputStream(
            new ByteArrayInputStream(allAudio), format, allAudio.length / 2);
        new File("outputs").mkdirs();
        AudioSystem.write(ais, AudioFileFormat.Type.WAVE,
            new File("outputs/" + filename));
        System.out.println("Audio saved to: outputs/" + filename);
    }
}
Compile and run ServerCommit.java to hear the audio generated by Realtime API in real time.