The AOQ Client SDK provides comprehensive audio capabilities, covering audio capture, playback, codec configuration, speaker management, file mixing, external audio stream injection, and audio frame data callbacks. This document introduces common audio features across Android (Java), iOS (Objective-C), and HarmonyOS (ArkTS).
Audio capture
Audio capture opens the device microphone and feeds real-time audio data into the SDK encoding pipeline. The SDK supports two capture modes:
- Internal capture (default): The SDK automatically manages the microphone -- opening, recording, and closing it.
- External capture: The application manages the microphone directly and feeds the captured PCM data into the SDK through the external audio stream API.
Configuration parameters
| Parameter | Type | Default | Description |
|---|
| isExternal | bool | false | Whether to use external capture mode |
| isVoipMode | bool | false | Whether to enable VoIP mode (hardware AEC). Valid on mobile. If both capture and playback are configured, whichever is set first takes effect. |
| channel | int | 1 | Number of capture channels. Supports 1 (mono) or 2 (stereo). |
API reference
| Function | Android | iOS | HarmonyOS |
|---|
| Start capture | startAudioCapture(config) | startAudioCapture:config: | startAudioCapture(config) |
| Stop capture | stopAudioCapture() | stopAudioCapture | stopAudioCapture() |
| Mute/unmute | muteAudioCapture(mute) | muteAudioCapture: | muteAudioCapture(mute) |
Example
Android
AoqAudioCaptureConfig config = new AoqAudioCaptureConfig();
config.isVoipMode = true;
config.channel = 1;
engine.startAudioCapture(config);
AoqAudioCaptureConfig *config = [[AoqAudioCaptureConfig alloc] init];
config.isVoipMode = YES;
config.channel = 1;
[engine startAudioCapture:config];
const config: AoqAudioCaptureConfig = { isVoipMode: true, channel: 1 };
engine.startAudioCapture(config);
Audio playback
Audio playback renders received remote audio data to the local speaker or headset. The SDK supports advanced controls including pause/resume with fade in/out and interrupting the current turn of an audio conversation.
Configuration parameters
| Parameter | Type | Default | Description |
|---|
| isVoipMode | bool | false | Whether to enable VoIP mode (hardware AEC). Valid on mobile. If both capture and playback are configured, whichever is set first takes effect. |
| isDefaultSpeaker | bool | true | Whether to use the speaker by default. Valid on mobile and only in non-VoIP mode. |
| isExternal | bool | false | Whether to use external playback mode. |
| channel | int | 1 | Number of playback channels. Supports 1 (mono) or 2 (stereo). |
API reference
| Function | Android | iOS | HarmonyOS |
|---|
| Start playback | startAudioPlayer(config) | startAudioPlayer:config: | startAudioPlayer(config) |
| Stop playback | stopAudioPlayer() | stopAudioPlayer | stopAudioPlayer() |
| Pause playback | pauseAudioPlayer(fadeMs) | pauseAudioPlayer: | pauseAudioPlayer(fadeMs) |
| Resume playback | resumeAudioPlayer(fadeMs) | resumeAudioPlayer: | resumeAudioPlayer(fadeMs) |
| Interrupt conversation | interruptAudioPlayer(trackType, fadeMs) | interruptAudioPlayer:fadeMs: | interruptAudioPlayer(trackType, fadeMs) |
fadeMs parameter: The fade-in or fade-out duration in milliseconds when pausing or resuming playback. Set to 0 for an immediate switch.
Speaker management
Switch the audio output device between the speaker and earpiece.
| Function | Android | iOS | HarmonyOS |
|---|
| Switch speaker | enableSpeakerphone(enable) | enableSpeakerphone: | enableSpeakerphone(enable) |
| Query speaker state | isSpeakerphoneEnabled() | isSpeakerphoneEnabled | isSpeakerphoneEnabled() |
Speaker switching is only allowed in VoIP mode. When not in VoIP mode, calling enableSpeakerphone triggers an OnError(AoqECAudioDeviceEarpieceRequiresVoipMode) error notification.
iOS-specific behavior: iPad devices have only speaker mode. When the AVAudioSession category is not PlayAndRecord, this method always returns YES.
Audio codec configuration
Configure the encoding format, sample rate, channel count, and bitrate for audio uplink (encoder) and downlink (decoder). These settings determine the format for publishing and pulling streams.
Configuration parameters
| Parameter | Type | Default | Description |
|---|
| trackType | AoqTrackType | Audio | Audio track type. Currently only one audio stream is supported. |
| codecType | AoqEncoderType | AudioPCM | Encoding type: AudioPCM(1) or AudioOpus(2) |
| sampleRate | int | 48000 | Sample rate. Opus supports 8K/16K/48K. PCM supports 8K/16K/32K/48K. |
| channel | int | 1 | Channel count. Supports 1 (mono) or 2 (stereo). |
| bitrate | int | 32000 | Bitrate in bps. |
API reference
| Function | Android | iOS | HarmonyOS |
|---|
| Set encoder config | setAudioEncoderConfig(config) | setAudioEncoderConfig: | setAudioEncoderConfig(config) |
| Set decoder config | setAudioDecoderConfig(config) | setAudioDecoderConfig: | setAudioDecoderConfig(config) |
| Enum value | Number | Description |
|---|
| AoqEncoderTypeAudioPCM | 1 | Raw PCM audio |
| AoqEncoderTypeAudioOpus | 2 | Opus encoding |
Audio file mixing
Mix a local audio file into the current audio stream for publishing and/or local playback. Each audio file is identified by an application-assigned fileId, allowing multiple file instances to be managed simultaneously.
Mixing configuration parameters
| Parameter | Type | Default | Description |
|---|
| fileName | String | - | Audio file path (including filename) |
| cycles | int | -1 | Number of loops. -1 means loop indefinitely. |
| startPosMs | long | 0 | Start playback position in milliseconds |
| publishVolume | int | 100 | Publishing volume [0-100] |
| playoutVolume | int | 100 | Local playback volume [0-100] |
API reference
| Function | Android | iOS | HarmonyOS |
|---|
| Start playback | startAudioFile(fileId, config) | startAudioFile:config: | startAudioFile(fileId, config) |
| Stop playback | stopAudioFile(fileId) | stopAudioFile: | stopAudioFile(fileId) |
| Pause | pauseAudioFile(fileId) | pauseAudioFile: | pauseAudioFile(fileId) |
| Resume | resumeAudioFile(fileId) | resumeAudioFile: | resumeAudioFile(fileId) |
| Get file duration | getAudioFileDuration(fileId) | getAudioFileDuration: | getAudioFileDuration(fileId) |
| Get current position | getAudioFileCurrentPosition(fileId) | getAudioFileCurrentPosition: | getAudioFileCurrentPosition(fileId) |
| Seek to position | setAudioFilePositionMillis(fileId, pos) | setAudioFilePositionMillis:positionMillis: | setAudioFilePositionMillis(fileId, pos) |
| Set volume | setAudioFileVolume(fileId, type, vol) | setAudioFileVolume:type:volume: | setAudioFileVolume(fileId, type, vol) |
| Get volume | getAudioFileVolume(fileId, type) | getAudioFileVolume:type: | getAudioFileVolume(fileId, type) |
Volume direction (type): AoqAudioStreamPublish(0) controls the publishing volume. AoqAudioStreamPlayout(1) controls the local playback volume.
State callbacks
| State code | Value | Description |
|---|
| AoqAudioFileNone | 0 | Initial state |
| AoqAudioFileStarted | 1 | Playback started |
| AoqAudioFileStopped | 2 | Playback stopped |
| AoqAudioFilePaused | 3 | Playback paused |
| AoqAudioFileResumed | 4 | Playback resumed |
| AoqAudioFileEnded | 5 | Playback ended |
| AoqAudioFileBuffering | 6 | Buffering |
| AoqAudioFileBufferingEnd | 7 | Buffering ended |
| AoqAudioFileFailed | 8 | Playback failed |
External audio streams
External audio streams let you inject application-generated PCM audio data into the SDK's audio pipeline for publishing and/or local playback. Typical use cases include TTS synthesis output, AI model audio output, and background sound effects. Each external audio stream is identified by an application-assigned streamId.
Configuration parameters
| Parameter | Type | Default | Description |
|---|
| trackType | AoqTrackType | Audio | Audio track type |
| codecType | AoqEncoderType | AudioPCM | Audio stream format |
| channels | int | 1 | Channel count |
| sampleRate | int | 48000 | Sample rate. Supports 8/12/16/24/32/44.1/48/64/88.2/96/176.4/192 kHz. |
| playoutVolume | int | 100 | Local playback volume [0-100] |
| publishVolume | int | 100 | Publishing volume [0-100] |
| maxBufferDuration | int | 600000 | Maximum buffer duration in milliseconds. Valid range: [100, ~]. Push fails if the buffer is full. |
| enable3A | bool | false | Whether to apply 3A processing to the input PCM |
API reference
| Function | Android | iOS | HarmonyOS |
|---|
| Add external stream | addAudioExternalStream(streamId, config) | addAudioExternalStream:config: | addAudioExternalStream(streamId, config) |
| Push audio data | pushAudioExternalStreamData(streamId, data) | pushAudioExternalStreamData:data: | pushAudioExternalStreamData(streamId, data) |
| Set volume | setAudioExternalStreamVolume(streamId, type, vol) | setAudioExternalStreamVolume:type:volume: | setAudioExternalStreamVolume(streamId, type, vol) |
| Get volume | getAudioExternalStreamVolume(streamId, type) | getAudioExternalStreamVolume:type: | getAudioExternalStreamVolume(streamId, type) |
| Clear buffer | clearAudioExternalStreamBuffer(streamId, fadeoutMs) | clearAudioExternalStreamBuffer:fadeoutMs: | clearAudioExternalStreamBuffer(streamId, fadeoutMs) |
| Remove stream | removeAudioExternalStream(streamId) | removeAudioExternalStream: | removeAudioExternalStream(streamId) |
Best practices for pushing data
- Call
pushAudioExternalStreamData in a loop to ensure data is pushed successfully.
- If error code 110 (buffer full) is returned, sleep for 30 ms and retry. Do not discard data.
- Before the engine exits, stop the push loop first, then call
removeAudioExternalStream.
- For real-time capture, each frame is 10 ms -- call push whenever data is available. For file-based input, each frame is 40 ms -- call push once every 30 ms.
Audio frame callbacks
Audio frame callbacks let you obtain raw PCM data at different points in the audio pipeline, for use in audio analysis, custom processing, recording, and similar scenarios.
Supported data source positions
| Data source | Enum value | Description |
|---|
| Captured | 0 | Raw audio data after capture, before 3A processing |
| ProcessCaptured | 1 | Audio data after 3A processing. Callbacks start only after a successful connection. |
| Publish | 2 | Audio data about to be published. Requires a successful connection. |
| Playback | 3 | Audio data about to be played back (remote downlink) |
Callback configuration parameters
| Parameter | Type | Default | Description |
|---|
| sampleRate | int | 48000 | Sample rate for callback audio |
| channels | int | 1 | Channel count for callback audio. Supports 1 or 2. |
| mode | AoqAudioObserverMode | ReadOnly | Read-only (0) or read-write (1) mode |
Usage steps
- Register the observer: Call
setAudioFrameObserver to set the audio frame callback listener.
- Enable the data source: Call
enableAudioFrameObserver to select the data source position and start callbacks.
- Handle callback data: Process PCM data in the callback.
API reference
| Function | Android | iOS | HarmonyOS |
|---|
| Register observer | setAudioFrameObserver(listener) | setAudioFrameObserver: | setAudioFrameObserver(observer) |
| Enable callbacks | enableAudioFrameObserver(enabled, source, config) | enableAudioFrameObserver:audioSource:config: | enableAudioFrameObserver(enabled, source, config) |
Callback methods
| Callback | Android | iOS | HarmonyOS |
|---|
| Captured data | onCapturedAudioFrame(frame) | onCapturedAudioFrame: | onCapturedAudioFrame(frame) |
| Post-3A data | onProcessCapturedAudioFrame(frame) | onProcessCapturedAudioFrame: | onProcessCapturedAudioFrame(frame) |
| Publish data | onPublishAudioFrame(trackType, frame) | onPublishAudioFrame:frame: | onPublishAudioFrame(trackType, frame) |
| Playback data | onPlaybackAudioFrame(frame) | onPlaybackAudioFrame: | onPlaybackAudioFrame(frame) |
Audio state and routing
The SDK automatically monitors audio device state changes and routing switches, and notifies the application layer through callbacks.
Device state codes
| State code | Value | Description |
|---|
| AoqAudioDeviceNone | 0 | Initial state |
| RecordStarting | 1 | Capture starting |
| RecordStarted | 2 | Capture started |
| RecordStopping | 3 | Capture stopping |
| RecordStopped | 4 | Capture stopped |
| RecordFail | 5 | Capture failed |
| PlayStarting | 6 | Playback starting |
| PlayStarted | 7 | Playback started |
| PlayStopping | 8 | Playback stopping |
| PlayStopped | 9 | Playback stopped |
| PlayFail | 10 | Playback failed |
Device routing types
| Route | Value | Description |
|---|
| Default | 0 | Default |
| Headset | 1 | Wired headset |
| Earpiece | 2 | Earpiece |
| HeadsetNoMic | 3 | Headset without microphone |
| SpeakerPhone | 4 | Speaker |
| Usb | 5 | USB device |
| Bluetooth | 6 | Bluetooth SCO |
| BluetoothA2dp | 7 | Bluetooth A2DP |
Callback reference
| Callback | Android | iOS | HarmonyOS |
|---|
| Device state change | onAudioDeviceStateChanged(state) | onAudioDeviceStateChanged: | onAudioDeviceStateChanged(state, reason) |
| Route change | onAudioDeviceRouteChanged(routeType) | onAudioDeviceRouteChanged: | onAudioDeviceRouteChanged(routeType) |
| Device interrupted | onAudioDeviceInterrupted(interrupt) | onAudioDeviceInterrupted: | onAudioDeviceInterrupted(interrupt) |
| File state | onAudioFileState(state) | onAudioFileState: | onAudioFileState(fileId, stateCode, errorCode) |
Audio error and warning codes
Audio error codes
| Error code | Value | Description |
|---|
| AoqErrorCodeAudio | 100 | General audio error |
| AudioExternalBufferFull | 110 | External buffer full |
| AudioDevice | 120 | General device error |
| RecordingAuthFailed | 121 | Microphone permission denied |
| RecordingOccupied | 122 | Microphone in use by another process |
| RecordingBackgroundStart | 123 | Started recording in the background |
| RecordingStartFail | 124 | Failed to start recording |
| PlayoutOccupied | 125 | Playback device in use by another process |
| PlayoutBackgroundStart | 126 | Started playback in the background |
| PlayoutStartFail | 127 | Failed to start playback |
| EarpieceRequiresVoipMode | 128 | Earpiece requires VoIP mode to be enabled |
Audio warning codes
| Warning code | Value | Description |
|---|
| AoqWCAudio | 100 | General audio warning |
| AudioHowling | 101 | Howling detected |
| AudioDevice | 120 | General device warning |
| MicEnumerateError | 121 | Microphone enumeration error |
| MicStartTimeout | 122 | Microphone start timeout |
| RecordingError | 123 | Recording error |
| SpeakerEnumerateError | 124 | Speaker enumeration error |
| SpeakerStartTimeout | 125 | Speaker start timeout |
| PlayoutError | 126 | Playback error |
iOS only: AVAudioSession control
On iOS, the setAudioSessionRestriction API gives you fine-grained control over how the SDK manages the system AVAudioSession.
| Control | Description |
|---|
| SetCategory | Whether the SDK can set the session category |
| ConfigureSession | Whether the SDK can configure session parameters |
| DeactivateSession | Whether the SDK can deactivate the session |
| ActivateSession | Whether the SDK can activate the session |
