Skip to main content
SDK Introduction

Linux Python SDK

AOQ Client SDK Linux Python API reference

The Linux platform provides APIs through the aoq_client_sdk Python module. All callbacks are triggered on native threads, and you must ensure thread safety yourself.

API index

Engine lifecycle

APIDescription
create_engineCreate the engine instance (singleton)
destroyDestroy the engine instance
get_versionGet the SDK version
connectConnect to the relay server
disconnectDisconnect from the server

Audio device management

APIDescription
start_audio_captureStart audio capture (empty implementation on Linux; does not open a microphone)
stop_audio_captureStop audio capture (no actual effect on Linux)
mute_audio_captureMute or unmute audio capture (device capture is not available on Linux)
start_audio_playerStart audio playback (empty implementation on Linux; does not open a speaker)
stop_audio_playerStop audio playback (no actual effect on Linux)
pause_audio_playerPause audio playback (device playback is not available on Linux)
resume_audio_playerResume audio playback (device playback is not available on Linux)
interrupt_audio_playerInterrupt the current audio session

Audio encoding configuration

APIDescription
set_audio_encoder_configSet audio encoding parameters
set_audio_decoder_configSet audio decoding parameters

Video device management

APIDescription
start_video_captureStart video capture (only external capture mode is supported on Linux)
stop_video_captureStop video capture (applies only to external capture mode on Linux)
set_local_viewSet or remove the local video rendering window (no rendering implementation on Linux)
set_remote_viewSet or remove the remote video rendering window (no rendering implementation on Linux)

Video codec and external input

APIDescription
set_video_encoder_configSet video encoding parameters
set_video_decoder_configSet video decoding parameters
push_external_video_framePush an externally captured video frame
push_external_video_encoded_framePush an externally encoded video frame

Media stream control

APIDescription
enable_send_media_streamEnable or disable sending of a local media stream

Audio file playback

APIDescription
start_audio_fileStart playing a local audio file to the publishing stream
stop_audio_fileStop audio file playback
pause_audio_filePause audio file playback
resume_audio_fileResume audio file playback
get_audio_file_durationQuery the total duration of the audio file
get_audio_file_current_positionQuery the current playback position of the audio file
set_audio_file_position_millisSet the playback position of the audio file (seek)
set_audio_file_volumeSet the volume of the audio file
get_audio_file_volumeQuery the current volume of the audio file

External audio streams

APIDescription
add_audio_external_streamAdd an external audio stream
remove_audio_external_streamRemove an external audio stream
push_audio_external_stream_dataFeed external audio PCM data
set_audio_external_stream_volumeSet the volume of an external audio stream
get_audio_external_stream_volumeQuery the volume of an external audio stream
clear_audio_external_stream_bufferClear the buffer of an external audio stream

Real-time messaging

APIDescription
send_data_msgSend a real-time data message

Audio frame callbacks

APIDescription
set_audio_frame_observerSet the audio frame data callback listener
enable_audio_frame_observerEnable or disable the audio frame callback at a specified position

Video frame callbacks

APIDescription
set_video_frame_observerSet the video frame data callback listener
enable_video_frame_observerEnable or disable the video frame callback at a specified position

Callback interfaces

CallbackDescription
on_errorEngine error callback
on_connection_status_changeConnection status change callback
on_data_msgCallback for a received real-time data message
IAudioFrameObserverBase class for audio frame data listeners
IVideoFrameObserverBase class for video frame data listeners

Utility functions

APIDescription
load_libraryManually specify and load the native shared library

API details

Engine lifecycle

create_engine

Creates the engine instance (class method). The SDK holds the engine as a global singleton, so calling this method again returns the existing instance. After destroy, you must call create_engine again before you can continue to use the engine.
@classmethod
def create_engine(cls, config: AoqCreateConfig, listener: AoqEngineEventListener) -> "AoqClientEngine"
ParameterTypeDescription
configAoqCreateConfigEngine creation configuration
listenerAoqEngineEventListenerEngine event callback listener
Returns: The AoqClientEngine engine instance. If the creation fails, a RuntimeError is thrown.

destroy

Destroys the engine instance (class method) and releases all resources.
@classmethod
def destroy(cls) -> int
Returns: 0 on success; non-zero on failure.

get_version

Gets the current SDK version (static method).
@staticmethod
def get_version() -> str
Returns: The version string, such as "1.0.0".

connect

Connects to the relay server. The application server obtains temporary AOQ connection parameters based on the protocol in use and sends them to the client. For more information, see Token authentication.
def connect(self, config: AoqConnectConfig) -> int
ParameterTypeDescription
configAoqConnectConfigConnection configuration, which includes the token, SID, and the list of relay access points
Returns: 0 indicates that the call is dispatched and runs asynchronously; non-zero indicates that parameter validation failed.

disconnect

Disconnects from the server and releases the resources associated with the connection.
def disconnect(self) -> int
Returns: 0 indicates that the call is dispatched and runs asynchronously; non-zero indicates a failure.

Audio device management

start_audio_capture

def start_audio_capture(self, config: AoqAudioCaptureConfig) -> int
On Linux, this method is an empty implementation. It returns 0 but does not open a sound card or microphone. Provide audio input through an external audio stream.

stop_audio_capture

def stop_audio_capture(self) -> int
This method has no actual effect because the Linux build does not set the audio capture device state.

mute_audio_capture

def mute_audio_capture(self, mute: bool) -> int
The Linux build does not support microphone capture. This method cannot control an actual audio capture device.

start_audio_player

def start_audio_player(self, config: AoqAudioPlaybackConfig) -> int
On Linux, this method is an empty implementation. It returns 0 but does not open a sound card or speaker. Obtain audio output through an audio frame callback and play it yourself.

stop_audio_player / pause_audio_player / resume_audio_player

def stop_audio_player(self) -> int
def pause_audio_player(self, fade_ms: int = 0) -> int
def resume_audio_player(self, fade_ms: int = 0) -> int
The Linux build does not support speaker playback, so these methods cannot control an actual audio playback device. fade_ms specifies the fade-out or fade-in duration in milliseconds. 0 indicates immediate execution.

interrupt_audio_player

def interrupt_audio_player(self, track_type: int, fade_ms: int = 0) -> int
Interrupts the current audio session.

Audio encoding configuration

def set_audio_encoder_config(self, config: AoqAudioCodecConfig) -> int
def set_audio_decoder_config(self, config: AoqAudioCodecConfig) -> int

Video device management

def start_video_capture(self, config: AoqVideoCaptureConfig) -> int
def stop_video_capture(self) -> int
def set_local_view(self, track_type: int, canvas: Optional[AoqVideoCanvas]) -> int
def set_remote_view(self, track_type: int, canvas: Optional[AoqVideoCanvas]) -> int
The Linux build does not support camera capture and has no video rendering backend. When you call start_video_capture, set is_external to True and provide frames by calling push_external_video_frame. set_local_view, set_remote_view, and AoqVideoCanvas.view have no actual effect on Linux. To preview video, obtain frames through a video frame callback and render them yourself.

Video codec and external input

def set_video_encoder_config(self, config: AoqVideoCodecConfig) -> int
def set_video_decoder_config(self, config: AoqVideoCodecConfig) -> int
def push_external_video_frame(self, frame: AoqVideoFrame, track_type: AoqTrackType = AoqTrackType.VIDEO) -> int
def push_external_video_encoded_frame(self, track_type: AoqTrackType, frame: AoqVideoEncodedFrame) -> int
Notes:
  • For set_video_decoder_config, only the track_type/codec_type/width/height/fps/bitrate fields take effect.
  • push_external_video_frame is consumed only after start_video_capture(is_external=True). For packed formats (NV12/NV21/BGRA/RGBA), fill frame.data; for the three I420 planes, fill frame.data_y/u/v and the corresponding stride. If the buffer is full, it returns AoqErrorCode.VIDEO_EXTERNAL_BUFFER_FULL(210).
  • push_external_video_encoded_frame requires start_video_capture(is_external=True) and set_video_encoder_config(codec_type=VIDEO_JPEG), and takes the bypass path without re-encoding.

Media stream control

def enable_send_media_stream(self, track_type: AoqTrackType, enable: bool) -> int
After initialization, disable sending separately for each required track by calling enable_send_media_stream(AoqTrackType.AUDIO, False) and enable_send_media_stream(AoqTrackType.VIDEO, False) as needed. After on_connection_status_change(CONNECTED) is received, enable each track in a separate call.

Audio file playback

def start_audio_file(self, file_id: str, config: AoqAudioFileMixConfig) -> int
def stop_audio_file(self, file_id: str) -> int
def pause_audio_file(self, file_id: str) -> int
def resume_audio_file(self, file_id: str) -> int
def get_audio_file_duration(self, file_id: str) -> int
def get_audio_file_current_position(self, file_id: str) -> int
def set_audio_file_position_millis(self, file_id: str, position_ms: int) -> int
def set_audio_file_volume(self, file_id: str, type_: AoqAudioStreamDirection, volume: int) -> int
def get_audio_file_volume(self, file_id: str, type_: AoqAudioStreamDirection) -> int

External audio streams

def add_audio_external_stream(self, stream_id: str, config: AoqAudioExternalStreamConfig) -> int
def remove_audio_external_stream(self, stream_id: str) -> int
def push_audio_external_stream_data(self, stream_id: str, data: AoqAudioFrameData) -> int
def set_audio_external_stream_volume(self, stream_id: str, type_: AoqAudioStreamDirection, volume: int) -> int
def get_audio_external_stream_volume(self, stream_id: str, type_: AoqAudioStreamDirection) -> int
def clear_audio_external_stream_buffer(self, stream_id: str, fadeout_ms: int = -1) -> None
When the buffer is full, push_audio_external_stream_data returns AoqErrorCode.AUDIO_EXTERNAL_BUFFER_FULL(110). We recommend that you wait about 20 ms and then retry the same frame.

Real-time messaging

def send_data_msg(self, msg: AoqDataMsg) -> int

Audio frame callbacks

def set_audio_frame_observer(self, observer: Optional[IAudioFrameObserver]) -> int
def enable_audio_frame_observer(self, enabled: bool, audio_source: AoqAudioSource, config: AoqAudioObserverConfig) -> int
Pass None to set_audio_frame_observer to unregister the observer.

Video frame callbacks

def set_video_frame_observer(self, observer: Optional[IVideoFrameObserver]) -> int
def enable_video_frame_observer(self, enabled: bool, video_source: AoqVideoSource, config: AoqVideoObserverConfig) -> int
Pass None to set_video_frame_observer to unregister the observer.

Callback interfaces

AoqEngineEventListener

Base class for engine event callbacks. All methods are optional overrides with default empty implementations. Callbacks are triggered on native threads, and you must ensure thread safety yourself.
class AoqEngineEventListener:
    def on_error(self, code: int, message: str) -> None: ...
    def on_connection_status_change(self, status: AoqConnectionStatus) -> None: ...
    def on_data_msg(self, msg: AoqDataMsg) -> None: ...

on_error

def on_error(self, code: int, message: str) -> None
Engine error callback. code corresponds to an AoqErrorCode enum value.

on_connection_status_change

def on_connection_status_change(self, status: AoqConnectionStatus) -> None
Connection status change callback. State transitions: DISCONNECTED -> CONNECTING -> CONNECTED/FAILED -> DISCONNECTED.

on_data_msg

def on_data_msg(self, msg: AoqDataMsg) -> None
Callback for a received real-time data message.

IAudioFrameObserver

Base class for audio frame data listeners. All methods are optional overrides with default empty implementations.
class IAudioFrameObserver:
    def on_captured_audio_frame(self, data: AoqAudioFrameData) -> None: ...
    def on_process_captured_audio_frame(self, data: AoqAudioFrameData) -> None: ...
    def on_publish_audio_frame(self, track_type: AoqTrackType, data: AoqAudioFrameData) -> None: ...
    def on_playback_audio_frame(self, data: AoqAudioFrameData) -> None: ...
The callback is triggered on the native audio thread. Do not perform any time-consuming operations in it. AoqAudioFrameData.data is already a bytes copy and can be safely used asynchronously.

IVideoFrameObserver

Base class for video frame data listeners. All methods are optional overrides and return False by default.
class IVideoFrameObserver:
    def on_captured_video_frame(self, frame: AoqVideoFrame) -> bool: ...
    def on_pre_encode_video_frame(self, track_type: AoqTrackType, frame: AoqVideoFrame) -> bool: ...
    def on_remote_video_frame(self, track_type: AoqTrackType, frame: AoqVideoFrame) -> bool: ...
The callback is triggered on the native video thread. Do not perform any time-consuming operations in it. The pixel data in the frame is already a bytes copy. The Python layer currently uses copy semantics and does not support writing modifications back, so we recommend that you always return False.

Utility functions

load_library

def load_library(path: Optional[str] = None) -> ctypes.CDLL
Loads the native shared library. You usually do not need to call this method manually, because the library is loaded automatically when you first use the engine. If path is empty, libAoqClientSdk.so is searched for in the following order: the AOQ_CLIENT_SDK_LIB environment variable, the directory of the module, the lib directory at the same level, and the default system paths. If the loading fails, an OSError is thrown. A typical cause is that a dependent library is not in LD_LIBRARY_PATH.

Data types and enumerations

All data types are Python dataclasses. You can construct them directly and assign values to the fields.

General types

AoqCreateConfig

FieldTypeDefault valueDescription
work_dirstr""SDK working directory
enable_dump_audioboolFalseSpecifies whether to enable audio dump (for debugging)
extrasstr""Extended parameter string

AoqConnectConfig

FieldTypeDefault valueDescription
tokenstr""Connection authentication token
sidstr""Session ID
certificatestr""Server certificate fingerprint
relay_endpointsList[AoqRelayEndpoint]NoneList of relay access points
workspace_id_hashstr""Workspace ID hash
publish_tracksList[AoqTrackParam]NoneList of local published tracks
subscribe_tracksList[AoqTrackParam]NoneList of local subscribed tracks

AoqRelayEndpoint

FieldTypeDefault valueDescription
endpointstr""Domain name or IP address of the relay server
portint0Port of the relay server
route_indexint-1Path index, which is aligned with routeIndex in SDKs for other platforms. If it is less than 0, the SDK automatically fills it in based on the array subscript.

AoqTrackParam

FieldTypeDefault valueDescription
track_typeAoqTrackTypeAoqTrackType.AUDIOTrack type

AoqDataMsg

FieldTypeDefault valueDescription
databytesb""Message data (a byte string)

Enumerations

AoqErrorCode

Enum valueValueDescription
OK0Success
PARAM_INVALID1Invalid parameter
STATE_INVALID2Invalid state
AUDIO100Generic audio error
AUDIO_EXTERNAL_BUFFER_FULL110External audio buffer is full
AUDIO_DEVICE120Generic audio device error
AUDIO_DEVICE_RECORDING_AUTH_FAILED121Recording permission not granted
AUDIO_DEVICE_RECORDING_OCCUPIED122Recording device is in use
AUDIO_DEVICE_RECORDING_START_FAIL124Failed to start recording
AUDIO_DEVICE_PLAYOUT_OCCUPIED125Playback device is in use
AUDIO_DEVICE_PLAYOUT_START_FAIL127Failed to start playback
VIDEO200Generic video error
VIDEO_EXTERNAL_BUFFER_FULL210External video buffer is full

AoqConnectionStatus

Enum valueValueDescription
DISCONNECTED0Disconnected
CONNECTING1Connecting
CONNECTED2Connected
FAILED3Connection failed

AoqTrackType

Enum valueValueDescription
AUDIO0Audio track
VIDEO1Video track
DATA2Data message track

AoqEncoderType

Enum valueValueDescription
UNKNOWN0Unknown format
AUDIO_PCM1Audio PCM
AUDIO_OPUS2Audio Opus
VIDEO_H2643Video H.264
VIDEO_JPEG4Video JPEG
DATA_TEXT5Message text

AoqMirrorMode

Enum valueValueDescription
DISABLED0Disable mirroring
ENABLED1Enable mirroring

AoqOrientationMode

Enum valueValueDescription
AUTO0Auto-fit
PORTRAIT1Portrait
LANDSCAPE2Landscape

AoqRenderMode

Enum valueValueDescription
AUTO0Adaptive mode
STRETCH1Stretch mode
FILL2Fill mode
CROP3Crop mode

AoqVideoPixelFormat

Enum valueValueDescription
UNKNOWN0Unknown format
I4201I420 (YUV planar format)
NV122NV12 (YUV semi-planar format)
NV213NV21 (YUV semi-planar format)
BGRA4BGRA (32-bit)
RGBA5RGBA (32-bit)

AoqCameraDirection

Enum valueValueDescription
FRONT0Front camera (platform-generic enum value; camera capture is not supported on Linux)
BACK1Rear camera (platform-generic enum value; camera capture is not supported on Linux)

Audio types

AoqAudioCaptureConfig

FieldTypeDefault valueDescription
is_externalboolFalseSpecifies whether to use external capture mode
channelint1Number of channels (mono by default)

AoqAudioPlaybackConfig

FieldTypeDefault valueDescription
is_externalboolFalseSpecifies whether to use external playback mode
channelint1Number of channels (mono by default)

AoqAudioCodecConfig

FieldTypeDefault valueDescription
track_typeAoqTrackTypeAUDIOTrack type
codec_typeAoqEncoderTypeAUDIO_PCMCodec format
sample_rateint48000Sample rate, in Hz
channelint1Number of channels
bitrateint32000Bitrate, in bit/s

AoqAudioFileMixConfig

FieldTypeDefault valueDescription
file_namestr""File name (including the path). It cannot be empty.
cyclesint-1Number of loops. -1 indicates unlimited looping.
start_pos_msint0Start playback position, in milliseconds
publish_volumeint100Publishing volume. Valid values: 0 to 100.
playout_volumeint100Playback volume. Valid values: 0 to 100.

AoqAudioExternalStreamConfig

FieldTypeDefault valueDescription
track_typeAoqTrackTypeAUDIOAudio track type
codec_typeAoqEncoderTypeAUDIO_PCMAudio stream format
channelsint1Number of channels
sample_rateint48000Sample rate, in Hz
playout_volumeint100Playback volume. Valid values: 0 to 100.
publish_volumeint100Publishing volume. Valid values: 0 to 100.
max_buffer_durationint1000Maximum buffer duration, in milliseconds
enable_3aboolFalseSpecifies whether to apply 3A processing to the input PCM

AoqAudioFrameData

Raw audio data, which is used for external input or observer callbacks.
FieldTypeDefault valueDescription
databytesb""Raw audio PCM data (a bytes copy in callbacks)
num_of_samplesint0Number of samples (per channel)
bytes_per_sampleint0Bytes per sample
num_of_channelsint0Number of channels
samples_per_secint0Number of samples per second (sample rate)
push_sequenceint0PCM input round
time_stampint0Timestamp
auto_gen_muteboolFalseTrue indicates silent data generated by the SDK

AoqAudioObserverConfig

FieldTypeDefault valueDescription
sample_rateint48000Sample rate of the callback audio, in Hz
channelsint1Number of audio channels in the callback
modeAoqAudioObserverModeREAD_ONLYRead-write mode

AoqAudioStreamDirection

Enum valueValueDescription
PUBLISH0Publishing stream
PLAYOUT1Playback stream (subscribing stream)

AoqAudioExternalStreamToggle

Enum valueValueDescription
NORMAL0Normal state
PAUSE1Paused state

AoqAudioSource

Enum valueValueDescription
CAPTURED0Captured audio data
PROCESS_CAPTURED1Audio data after 3A processing
PUBLISH2Audio data to be published
PLAYBACK3Audio data to be played

AoqAudioObserverMode

Enum valueValueDescription
READ_ONLY0Read-only mode
READ_WRITE1Read-write mode

Video types

AoqVideoCaptureConfig

FieldTypeDefault valueDescription
widthint1280Capture width, in pixels. This parameter is ineffective when is_external=True.
heightint720Capture height, in pixels. This parameter is ineffective when is_external=True.
fpsint15Capture frame rate. This parameter is ineffective when is_external=True.
is_externalboolFalseSpecifies whether to use external capture. Linux supports only True. After you call start_video_capture, provide frames by calling push_external_video_frame.
camera_directionAoqCameraDirectionFRONTCamera direction for mobile platforms. Linux has no corresponding capability. Keep the value FRONT.

AoqVideoCodecConfig

Video codec parameters, which are shared by encoding and decoding. For decoding, only track_type/codec_type/width/height/fps/bitrate take effect, and the other fields are used only for encoding.
FieldTypeDefault valueDescription
track_typeAoqTrackTypeVIDEOTrack type
codec_typeAoqEncoderTypeVIDEO_H264Codec format
widthint540Encoding width, in pixels
heightint960Encoding height, in pixels
fpsint5Encoding frame rate
bitrateint500000Target bitrate, in bit/s
min_bitrateint128000Minimum bitrate, in bit/s
keyframe_intervalint2Keyframe interval, in seconds
mirror_modeAoqMirrorModeDISABLEDMirror mode
orientation_modeAoqOrientationModeAUTOVideo orientation mode

AoqVideoCanvas

FieldTypeDefault valueDescription
viewint0Rendering window handle. Linux has no rendering backend, so this field has no actual effect. Keep the value 0.
render_modeAoqRenderModeAUTORendering mode. Linux has no rendering backend, so this field has no actual effect.

AoqVideoFrame

External video frame or video frame callback data. When you use a packed format (NV12/NV21/BGRA/RGBA), fill in data. When you use the three I420 planes, fill in data_y/u/v and the corresponding stride. The two methods are mutually exclusive.
FieldTypeDefault valueDescription
formatAoqVideoPixelFormatUNKNOWNPixel format
widthint0Video width, in pixels
heightint0Video height, in pixels
databytesb""Packed-format data (NV12/NV21/BGRA/RGBA)
data_ybytesb""I420 Y plane data
data_ubytesb""I420 U plane data
data_vbytesb""I420 V plane data
stride_yint0Stride of the Y plane
stride_uint0Stride of the U plane
stride_vint0Stride of the V plane
time_stampint0Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local clock.

AoqVideoEncodedFrame

An externally encoded video frame, such as a JPEG frame.
FieldTypeDefault valueDescription
codecAoqEncoderTypeVIDEO_JPEGCodec format
databytesb""Encoded data
widthint0Width, in pixels
heightint0Height, in pixels
time_stampint0Timestamp, in milliseconds. If it is 0, the SDK fills it in with the local clock.

AoqVideoObserverConfig

FieldTypeDefault valueDescription
formatAoqVideoPixelFormatI420Expected pixel format of the callback data
alignmentAoqVideoObserverAlignmentDEFAULTWidth alignment policy
modeAoqVideoObserverModeREAD_ONLYRead-write mode
mirror_appliedboolFalseSpecifies whether to mirror the callback data

AoqVideoSource

Enum valueValueDescription
CAPTURED0Captured video data before preprocessing
PRE_ENCODE1Video data before encoding, after preprocessing
REMOTE2Remote video data after decoding and before rendering

AoqVideoObserverMode

Enum valueValueDescription
READ_ONLY0Read-only mode
READ_WRITE1Read-write mode

AoqVideoObserverAlignment

Enum valueValueDescription
DEFAULT0Default alignment
EVEN1Even-number alignment
ALIGN_424-byte alignment
ALIGN_838-byte alignment
ALIGN_16416-byte alignment