Skip to main content
QwenCloud home page
Docs
Realtime API

Real-time voice conversation with AOQ + Qwen-Audio

Use AOQ to connect to qwen-audio-3.0-realtime-plus and use server-side VAD to build low-latency real-time voice conversations. Client code examples use Android Java.

Solution overview

Qwen-Audio is an end-to-end real-time voice interaction model for low-latency scenarios such as voice assistants, customer service, and AI companions. AOQ transports audio and events on separate tracks. The Audio track carries microphone PCM uplink and model PCM downlink, and the Data track carries Realtime protocol events. This tutorial uses server_vad. The client continuously sends audio, and the service detects when the user starts and stops speaking and triggers a response.

Prerequisites

  1. Activate QwenCloud and obtain an API Key following Get and configure an API Key. Store the API Key only on your application server. Do not include it in client code or commit it to a code repository.
  2. Download the latest AOQ Client SDK as described in SDK download.
  3. Build an application server and implement proxy authentication as described in Token authentication. Before each new connection, the client must obtain new connection credentials from the application server.

Import the SDK

Import the SDK for your development platform. The client implementation uses Android Java. Other platforms provide the same interfaces and event flow. This tutorial uses PCM audio streams. Opus encoding is provided by a plugin. Import the Opus plugin if the uplink uses Opus.
  • Android
  • iOS
  • HarmonyOS
  • Linux (Python)
  1. Place AoqClientSdk-release.aar in app/libs, and configure the dependency and SDK-supported ABIs in app/build.gradle:
android {
  defaultConfig {
    minSdk 21
    ndk { abiFilters 'armeabi-v7a', 'arm64-v8a' }
  }
}
dependencies {
  implementation fileTree(dir: 'libs', include: ['*.aar'])
}
  1. Declare the following permissions in AndroidManifest.xml:
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
  1. Request the RECORD_AUDIO permission at runtime before the corresponding devices are used.

Try the demo

QwenCloud provides an Android demo app to quickly verify AOQ connectivity. Download the APK and configure the API key and workspaceId to try selected models. Scan the following QR code to download the demo:
QR code for downloading the demo

Implementation flow

  1. The application server obtains credentials for the current AOQ connection to qwen-audio-3.0-realtime-plus from the Realtime token URL.
  2. The client configures the SDK uplink encoder and downlink decoder for the selected model and the application's audio format.
  3. The client initializes the recording and playback devices and creates AoqConnectConfig. It populates the credential fields for the current connection and configures the Audio and Data tracks to publish and subscribe to. The client keeps Audio-track sending disabled and calls connect to establish the AOQ connection.
  4. After the connection is established, the client sends session.update. It enables the Audio track only after session.updated is received.
  5. Server-side VAD automatically determines turn boundaries. Model audio is played over the Audio track and conversation events are returned over the Data track.
  6. To finish, disconnect and destroy the engine. The SDK automatically closes the audio devices.
Sequence diagram for real-time voice conversations over AOQ

Obtain a token from the application server

Set DASHSCOPE_API_KEY on the application server and send the request to the endpoint. clientIp is the actual public IP address of the client. This field is optional, but specifying it helps the service allocate an appropriate relay endpoint. 
curl -X POST \
  "https://dashscope-intl.aliyuncs.com/api/v1/webrtc/realtime?model=qwen-audio-3.0-realtime-plus" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${DASHSCOPE_API_KEY}" \
  -H "x-dashscope-rtc-transport: moq" \
  -d "{\"clientIp\": \"${CLIENT_REAL_IP}\"}"
If the application server cannot obtain the actual public IP address of the client, omit clientIp instead of passing an empty string.
The application server returns the following response fields to the client. An AOQ token can be used for only one connection. Before each connect call, the client must request a new token instead of caching or reusing one. Never return the API key to a client in production. For all request and response fields, see Token authentication.
Response fieldSDK field
aoqTokenForClientAoqConnectConfig.token
sidAoqConnectConfig.sid
clientRelayCertFingerprintAoqConnectConfig.certFingerprint
clientRelayEndpointsAoqConnectConfig.relayEndpoints

Implement the Android client

Before each connection, the client obtains new connection credentials from the application server and creates AoqConnectConfig. Map the token response fields and add client-side connection settings such as the publish and subscribe tracks. Follow these steps to implement real-time voice conversations on Android.

1. Create the engine and register callbacks

Create the singleton AOQ engine and register event callbacks. Configure the session after the connection succeeds, and dispatch server events to the UI and application state machine. 
AoqClientListener listener = new AoqClientListener() {
  @Override
  public void onConnectionStatusChange(AoqClientEngine.AoqConnectionStatus status) {
    if (status == AoqClientEngine.AoqConnectionStatus.AoqConnectionStatusConnected) {
      configureSession();
    }
  }
  @Override
  public void onDataMsg(AoqClientEngine.AoqDataMsg msg) {
    handleServerEvent(msg);
  }
};
AoqClientEngine.AoqCreateConfig createConfig = new AoqClientEngine.AoqCreateConfig();
createConfig.workDir = context.getFilesDir().getAbsolutePath();
engine = AoqClientEngine.createEngine(context, createConfig, listener);

2. Configure audio codecs

Configure the SDK uplink encoder and downlink decoder for the selected model and the application's audio format. The following values are PCM examples for this tutorial and do not restrict the audio format of your application. 
AoqClientEngine.AoqAudioCodecConfig audioEncoderConfig =
    new AoqClientEngine.AoqAudioCodecConfig();
audioEncoderConfig.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
audioEncoderConfig.codecType = AoqClientEngine.AoqEncoderType.AoqEncoderTypeAudioPCM;
audioEncoderConfig.sampleRate = 16000;
audioEncoderConfig.channel = 1;
engine.setAudioEncoderConfig(audioEncoderConfig);
AoqClientEngine.AoqAudioCodecConfig audioDecoderConfig =
    new AoqClientEngine.AoqAudioCodecConfig();
audioDecoderConfig.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
audioDecoderConfig.codecType = AoqClientEngine.AoqEncoderType.AoqEncoderTypeAudioPCM;
audioDecoderConfig.sampleRate = 24000;
audioDecoderConfig.channel = 1;
engine.setAudioDecoderConfig(audioDecoderConfig);

3. Configure tracks and connect

Use SDK interfaces to start audio capture and playback. Map the current application-server token response to the credential fields in AoqConnectConfig, and configure the Audio and Data tracks in publishTracks and subscribeTracks. Keep Audio-track sending disabled when you call connect. Enable sending only after session.updated is received. 
AoqClientEngine.AoqAudioCaptureConfig captureConfig =
    new AoqClientEngine.AoqAudioCaptureConfig();
captureConfig.channel = 1;
captureConfig.isVoipMode = true;
engine.startAudioCapture(captureConfig);
AoqClientEngine.AoqAudioPlaybackConfig playbackConfig =
    new AoqClientEngine.AoqAudioPlaybackConfig();
playbackConfig.channel = 1;
playbackConfig.isVoipMode = true;
playbackConfig.isDefaultSpeaker = true;
engine.startAudioPlayer(playbackConfig);
AoqClientEngine.AoqTrackParam publishAudioTrack = new AoqClientEngine.AoqTrackParam();
publishAudioTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
connectConfig.publishTracks.add(publishAudioTrack);
AoqClientEngine.AoqTrackParam publishDataTrack = new AoqClientEngine.AoqTrackParam();
publishDataTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeData;
connectConfig.publishTracks.add(publishDataTrack);
AoqClientEngine.AoqTrackParam subscribeAudioTrack = new AoqClientEngine.AoqTrackParam();
subscribeAudioTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
connectConfig.subscribeTracks.add(subscribeAudioTrack);
AoqClientEngine.AoqTrackParam subscribeDataTrack = new AoqClientEngine.AoqTrackParam();
subscribeDataTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeData;
connectConfig.subscribeTracks.add(subscribeDataTrack);
engine.enableSendMediaStream(
    AoqClientEngine.AoqTrackType.AoqTrackTypeAudio, false);
engine.connect(connectConfig);

4. Send session.update

After the connection succeeds, configure output modalities, voice, audio formats, instructions, and VAD. Both input_audio_format and output_audio_format use pcm. The SDK codec configuration determines the sample rates. For all parameters, see Client events. 
JSONObject vad = new JSONObject()
    .put("type", "server_vad")
    .put("threshold", 0.5)
    .put("silence_duration_ms", 800);
JSONObject session = new JSONObject()
    .put("modalities", new JSONArray().put("text").put("audio"))
    .put("voice", "longanqian")
    .put("input_audio_format", "pcm")
    .put("output_audio_format", "pcm")
    .put("instructions", "You are a helpful voice assistant.")
    .put("turn_detection", vad);
JSONObject sessionUpdate = new JSONObject()
    .put("type", "session.update")
    .put("session", session);
AoqClientEngine.AoqDataMsg dataMessage = new AoqClientEngine.AoqDataMsg();
dataMessage.data = sessionUpdate.toString().getBytes(StandardCharsets.UTF_8);
engine.sendDataMsg(dataMessage);
session.updated indicates that the session configuration is active. Enable Audio-track sending only at this point so that audio captured earlier is not sent to the model. 
if ("session.updated".equals(type)) {
  engine.enableSendMediaStream(
      AoqClientEngine.AoqTrackType.AoqTrackTypeAudio, true);
}

6. Handle server events

In onDataMsg, use type to display user and model transcripts and handle errors. For all event fields, see Server events. 
if ("response.audio_transcript.delta".equals(type)) {
  String delta = event.optString("delta");
  // Append delta to the model transcript in the UI.
} else if ("conversation.item.input_audio_transcription.completed".equals(type)) {
  String transcript = event.optString("transcript");
  // Display the final user transcript in the UI.
} else if ("error".equals(type)) {
  // Read the error fields and update the application state.
}

7. Disconnect and destroy the engine

When the conversation ends, disconnect and destroy the singleton engine. disconnect or destroy automatically closes audio capture and playback, so you do not need to stop the devices separately. 
engine.disconnect();
AoqClientEngine.destroy();

Main server events

Data-track events are identified by type. The client must handle the following key events. For complete event schemas, see Server events.
EventDescription
session.createdThe session is created and default settings are returned
session.updatedClient settings are active and audio uplink can be enabled
input_audio_buffer.speech_startedThe service detects that the user started speaking
input_audio_buffer.speech_stoppedThe service detects that the user stopped speaking
input_audio_buffer.committedAudio for the turn is committed
response.createdThe model starts generating a response
response.audio_transcript.deltaIncremental model transcript
conversation.item.input_audio_transcription.completedThe final user transcript is available
response.doneThe response is complete
errorA server error occurs

Complete example

The following class accepts an AoqConnectConfig populated with credentials for the current connection and adds the audio-device, publish-track, and subscribe-track settings. Obtain new credentials and create a new connection configuration for every reconnection. Add permissions, UI state, and reconnection logic in production. 
import android.content.Context;
import com.alibaba.aoq.clientsdk.AoqClientEngine;
import com.alibaba.aoq.clientsdk.AoqClientListener;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
import java.nio.charset.StandardCharsets;
public final class RealtimeVoiceChatClient {
  private AoqClientEngine engine;
  public RealtimeVoiceChatClient(Context context, AoqClientEngine.AoqConnectConfig connectConfig) {
    AoqClientListener listener = new AoqClientListener() {
      @Override
      public void onConnectionStatusChange(AoqClientEngine.AoqConnectionStatus status) {
        if (status == AoqClientEngine.AoqConnectionStatus.AoqConnectionStatusConnected) {
          configureSession();
        }
      }
      @Override
      public void onDataMsg(AoqClientEngine.AoqDataMsg msg) {
        try {
          JSONObject event = new JSONObject(
              new String(msg.data, StandardCharsets.UTF_8));
          String type = event.optString("type");
          if ("session.updated".equals(type)) {
            engine.enableSendMediaStream(
                AoqClientEngine.AoqTrackType.AoqTrackTypeAudio, true);
          } else if ("response.audio_transcript.delta".equals(type)) {
            String delta = event.optString("delta");
            // Display delta in the UI.
          } else if ("conversation.item.input_audio_transcription.completed".equals(type)) {
            String transcript = event.optString("transcript");
            // Display transcript in the UI.
          } else if ("error".equals(type)) {
            // Read error fields and update the application state.
          }
        } catch (JSONException e) {
          throw new IllegalArgumentException("Invalid server event", e);
        }
      }
    };
    AoqClientEngine.AoqCreateConfig createConfig = new AoqClientEngine.AoqCreateConfig();
    createConfig.workDir = context.getFilesDir().getAbsolutePath();
    engine = AoqClientEngine.createEngine(context, createConfig, listener);
    AoqClientEngine.AoqAudioCodecConfig audioEncoderConfig =
        new AoqClientEngine.AoqAudioCodecConfig();
    audioEncoderConfig.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
    audioEncoderConfig.codecType = AoqClientEngine.AoqEncoderType.AoqEncoderTypeAudioPCM;
    audioEncoderConfig.sampleRate = 16000;
    audioEncoderConfig.channel = 1;
    engine.setAudioEncoderConfig(audioEncoderConfig);
    AoqClientEngine.AoqAudioCodecConfig audioDecoderConfig =
        new AoqClientEngine.AoqAudioCodecConfig();
    audioDecoderConfig.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
    audioDecoderConfig.codecType = AoqClientEngine.AoqEncoderType.AoqEncoderTypeAudioPCM;
    audioDecoderConfig.sampleRate = 24000;
    audioDecoderConfig.channel = 1;
    engine.setAudioDecoderConfig(audioDecoderConfig);
    AoqClientEngine.AoqAudioCaptureConfig captureConfig =
        new AoqClientEngine.AoqAudioCaptureConfig();
    captureConfig.channel = 1;
    captureConfig.isVoipMode = true;
    engine.startAudioCapture(captureConfig);
    AoqClientEngine.AoqAudioPlaybackConfig playbackConfig =
        new AoqClientEngine.AoqAudioPlaybackConfig();
    playbackConfig.channel = 1;
    playbackConfig.isVoipMode = true;
    playbackConfig.isDefaultSpeaker = true;
    engine.startAudioPlayer(playbackConfig);
    AoqClientEngine.AoqTrackParam publishAudioTrack =
        new AoqClientEngine.AoqTrackParam();
    publishAudioTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
    connectConfig.publishTracks.add(publishAudioTrack);
    AoqClientEngine.AoqTrackParam publishDataTrack =
        new AoqClientEngine.AoqTrackParam();
    publishDataTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeData;
    connectConfig.publishTracks.add(publishDataTrack);
    AoqClientEngine.AoqTrackParam subscribeAudioTrack =
        new AoqClientEngine.AoqTrackParam();
    subscribeAudioTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeAudio;
    connectConfig.subscribeTracks.add(subscribeAudioTrack);
    AoqClientEngine.AoqTrackParam subscribeDataTrack =
        new AoqClientEngine.AoqTrackParam();
    subscribeDataTrack.trackType = AoqClientEngine.AoqTrackType.AoqTrackTypeData;
    connectConfig.subscribeTracks.add(subscribeDataTrack);
    engine.enableSendMediaStream(AoqClientEngine.AoqTrackType.AoqTrackTypeAudio, false);
    engine.connect(connectConfig);
  }
  private void configureSession() {
    try {
      JSONObject vad = new JSONObject()
          .put("type", "server_vad")
          .put("threshold", 0.5)
          .put("silence_duration_ms", 800);
      JSONObject session = new JSONObject()
          .put("modalities", new JSONArray().put("text").put("audio"))
          .put("voice", "longanqian")
          .put("input_audio_format", "pcm")
          .put("output_audio_format", "pcm")
          .put("turn_detection", vad);
      JSONObject sessionUpdate = new JSONObject()
          .put("type", "session.update")
          .put("session", session);
      AoqClientEngine.AoqDataMsg dataMessage = new AoqClientEngine.AoqDataMsg();
      dataMessage.data = sessionUpdate.toString().getBytes(StandardCharsets.UTF_8);
      engine.sendDataMsg(dataMessage);
    } catch (JSONException e) {
      throw new IllegalStateException("Failed to create session.update", e);
    }
  }
  public void close() {
    engine.disconnect();
    AoqClientEngine.destroy();
  }
}

Run and verify

  1. Microphone audio starts streaming only after session.updated is received.
  2. After the user stops speaking, the service commits the audio and starts responding. Text events and Audio-track audio are returned continuously.

Common scenarios

Change the interaction mode

Use server_vad for silence-based turn detection, smart_turn for acoustic and semantic turn detection, or set turn_detection to null for push-to-talk. turn_detection can be changed only before the first audio input. Establish a new session to change modes.

Change the voice

Set session.voice in the first session.update. Supported system voices vary by model. For supported voices and voice cloning, see Qwen-Audio real-time voice conversation.

Speaker or earpiece

Set the default output device by using AoqAudioPlaybackConfig.isDefaultSpeaker, and call enableSpeakerphone to switch while the session is active.

Background calls on Android

On Android 10 or later, use a foreground service with foregroundServiceType="microphone|mediaPlayback" to continue capture and playback in the background. Start it while the app is visible to the user.

Troubleshooting

IssueSolution
The connection failsMake sure that the token is valid, the endpoint matches the deployment region, and AoqConnectConfig fields are mapped correctly.
The session is established but no response is returnedMake sure that the Audio track is enabled after session.updated and that the SDK uplink encoder matches the model and application audio format.
The response has no audioMake sure that the Audio track is subscribed and the audio player is running, and then verify that the SDK downlink decoder matches the model output audio format.
Real-time voice conversation with AOQ + Qwen-Audio - QwenCloud