By integrating the HappyOyster Android SDK, your App can enter worlds generated by AI in real time, delivering a real-time interactive video experience in three modes: Adventure (world exploration), Directing (real-time directing), and Acting (role-play).
1. What You Can Do
- Start an experience (Travel): Enter a ready world with a one-time credential; the SDK automatically establishes the real-time video connection.
- Real-time playback: The SDK returns a video View that you mount into your layout to play the AI's real-time generated visuals.
- Real-time interaction:
- Directing mode (directing): Send text instructions to drive the storyline.
- Acting: Also send text instructions; pause/resume are available; do not rewind and do not use
sendCommand. Use theaspectRatiofrom the join response to set the player orientation (see §13 Mode Adaptation). - Adventure mode (adventure): Send direction/viewpoint/action control commands to interact with the world.
- Process control: Pause / resume (directing and Acting), rewind (directing only), end (all three modes).
- Status and error callbacks: Perceive experience status and exceptions in real time through event listeners.
2. Installation
The Happy Oyster SDK (cn.happyoyster:opensdk) is published on Maven Central; its underlying real-time communication engine (Alibaba Cloud ARTC) is published on Alibaba Cloud Maven. Both repositories must be declared.
Environment Requirements
Item | Requirement |
|---|---|
minSdk | 24 (Android 7.0) and above |
compileSdk | 36 |
JDK | JDK 11 bytecode target (host toolchain: JDK 11 or above recommended) |
Language | Kotlin (coroutine |
ABI |
|
Network | Public internet access required |
settings.gradle.kts:
<version> below with that exact version number, and add the dependency in the module build.gradle.kts:
com.aliyun.aio:AliVCSDK_ARTC to fail resolution.Permissions
The SDK library itself declares only INTERNET. The real-time communication engine automatically merges in a small number of network/Bluetooth/audio-settings-type permissions; the library manifest does not include microphone or camera permissions. The video stream is subscribe-only playback, and the SDK does not send real audio to the remote side.
The host must declare on its own: When your targetSdk is 33 or above, you must declare the notification permission in the host AndroidManifest.xml (the real-time communication engine includes a foreground service):
sendCommand) — declaring the microphone permission is recommended: The real-time control commands of adventure mode ride an uplink carried by a silent audio stream that establishes the local "publisher identity"; the SDK does not record or upload your real audio. RECORD_AUDIO is not a hard prerequisite for the DataChannel—even without it, the silent stream still stands as a publisher and the uplink is normally available. Still, for stability across devices we recommend that, if your app uses sendCommand, you declare the permission in the host AndroidManifest.xml and request it at runtime before calling.
105004: 105004 means the real-time channel is not ready / a send failed (e.g. a DataChannel interruption or a real-time channel anomaly). It is not a necessary consequence of a missing permission—it is not triggered directly by RECORD_AUDIO being ungranted. Directing mode (sendInstruct) and video playback are unaffected. If your app does not use adventure mode, this permission is not needed.
To trim merged-in permissions, use tools:node="remove".
3. Authentication Model
The SDK does not obtain or refresh tokens, keeping it lightweight. Authentication has two layers:
- QwenCloud gateway API Key: Injected as a Bearer token by your app via
updateToken(token). The SDK keeps only the latest one; it does not persist or refresh it, and after the Key changes you re-inject it. The gateway requires that all SDK requests, includingstartTravel, use this Bearer. When the Bearer Key expires or is invalid, the SDK throwsSDKError(101002); in this case you should re-obtain and re-inject the Bearer Key (updateToken) rather than exchange for a new ticket. During development / Demo, you may use your main QwenCloud API Key directly as the Bearer (updateToken) to get the flow working. In production, always switch to a short-lived token minted by your server, and never bundle a long-lived API Key into a distributed app. - One-time experience credential
ticket: Exchanged by your server through the open platform and delivered to the client; used for a singlestartTravelonly. It becomes invalid after the experience ends (normally or abnormally) and cannot be reused. Ticket-level credential errors are identified by six-digit server codes (e.g.,401010= ticket invalid / expired,401011= ticket already used), which the SDK passes through verbatim.
SDKConfig.apiHost (of the form dashscope-intl.aliyuncs.com, copied from the "API Host" field on the API Key page of the QwenCloud console) and pass the required, default-free SDKConfig.model (happyoyster-1.0-directing / happyoyster-1.0-acting / happyoyster-1.0-adventure, matching the Open API entry). The SDK completes the request URL as https://{apiHost}/api/v2/apps/{model}/openapi/v1/{endpoint}, so you do not assemble it yourself.
Omitting model when constructing SDKConfig is a compile-time error. If model is blank, initialize synchronously throws SDKError(100002) (raw = "SDKConfig.model must not be blank"), creates or replaces no runtime, and makes no network request. The API Host, model, and injected API Key must match the required account and model authorization, otherwise the gateway typically returns AccessDenied (at runtime this is thrown as SDKError(106003), with the original AccessDenied body available in SDKError.raw; see the 106003 row of the API Reference error-code table for a troubleshooting checklist). For security, we strongly recommend the client inject a short-lived token minted by your server as the Bearer, rather than bundling a long-lived API Key into the app or committing it to a code repository.
4. Quick Start
5. Event Subscription & Error Handling
Use onStatusChanged to drive the host state machine and to gate interaction capabilities; use onError to receive runtime errors uniformly. For the event interface and the complete error codes, see the Happy Oyster Android SDK API Reference.
Must
- Register the listener immediately after
HappyOyster.initialize(...)returns successfully;addListener/removeListenermust be called afterinitialize(calling them before initialization throwsSDKError(100001)). - Gate on
onStatusChanged(Running)—interaction calls are allowed only afterrunning; adventure-modesendCommandis valid only inrunning. - Handle
onErroras well; do not only catchstartTravel. Fatal errors also terminate the experience (see the error-codes section of the API Reference).
- Call
removeListenerat an appropriate lifecycle point (such asonDestroy) to avoid memory leaks.
- Registering listeners repeatedly without calling removeListener.
6. Sending Instructions: sendInstruct and sendCommand
sendInstruct (directing / Acting)
sendInstruct is used in directing and Acting to send text instructions that drive the picture; it can be called in either the running or paused state. In the paused state the SDK does not auto-resume—the host decides whether to call resumeTravel first and then send the instruct (for contract details such as throttling and state validation, see the API Reference). An Acting world's creationModel is always Simple, so the ScriptList restriction applies to directing only.
sendCommand (adventure mode)
sendCommand is used in adventure mode (adventure) to send direction/viewpoint/action control commands; it is valid only in running. The SDK has a built-in 42 ms (24 fps) latest-wins throttle, so the host can call it at the game frame rate and the SDK merges automatically; no manual rate limiting is required. Call once for a one-shot action. For a held action, keep calling every frame while held and explicitly send one None reset when released (for the detailed throttling and failure-surfacing contract, see the API Reference).
Recommended practice: In the host's adventure-mode interaction UI, provide three independent command entry points (movement direction + viewpoint direction + action interaction), maintain the currently held value for each group, and send a complete three-field snapshot on every call. Use "None" only for inactive dimensions. When inputs from different dimensions are held simultaneously, preserve and send all active values instead of resetting the other dimensions to "None".
7. Pause / Resume / Rewind
Applicable modes: pause / resume apply to directing and Acting, and behave identically in both (including the 3-second barrier below); rewind is directing only. Calling pauseTravel / resumeTravel / rewindTravel in adventure mode is rejected by the SDK with 103003. In addition, the version reported by the server must be the v2 token its mode requires (storyV2 for directing, actingV2 for Acting); otherwise pause / resume return 103002.
Pause is asynchronous: The return of the pauseTravel method only means it has been accepted; the actual pause is determined by onStatusChanged(Paused). Between "calling pauseTravel" and "receiving the Paused callback", the host may mark its local experience state as pausing; only after receiving Paused should you allow calling resumeTravel or the paused-state-dependent rewindTravel.
SDK-internal pause→reopen barrier: once Paused is received, the host may call resumeTravel / rewindTravel according to the contract. If the call falls inside the 3-second settle window after pause confirmation, the SDK's suspending method waits for the remaining time before sending the request that reopens the real-time room. The host does not need an extra pause→resume delay, and no delay is added once 3 seconds have elapsed naturally.
Host-side call cooldown (recommended): It is recommended that the host hold off on issuing another pauseTravel within 3 s after resumeTravel returns successfully, to avoid switching too frequently. The SDK itself does not enforce this cooldown.
Rewind: rewindTravel is available only in the paused state; after rewinding, the server automatically resumes and the SDK automatically reconnects RTC, with no host intervention needed. Typical sequence: pause → wait:paused → rewindTravel(sec). The rewind seconds rewindToSec should be a multiple of 4 (e.g., 4, 8, 12); non-multiples are floored by the server (e.g., 7→4), and the actual effective seconds are determined by the returned resumedAtSec. Rewind is Directing only: calling rewindTravel in Acting or adventure mode is rejected by the SDK locally with 103003, without issuing any HTTP request; the host should hide the rewind entry rather than only greying out the button.
For contract details such as asynchronous semantics and 3× retry backoff (resumeTravel backoff of 1 s / 2 s / 3 s), see the Happy Oyster Android SDK API Reference.
8. Logging
SDK Logcat tag: HappyOysterSDK. The SDK provides two independent log output paths:
- Built-in Logcat (off by default): the SDK writes to Logcat only when
SDKConfig.logcatEnabled = true(defaultfalse, fully silent).SDKConfig.logLevelfilters its minimum level; the defaultINFOalready carries the lifecycle anchors needed to reconstruct a session (initialize, Travel start / status transitions / end, RTC join / first frame). Drop toWARNfor errors-only, or raise toDEBUG/VERBOSEfor deeper diagnosis.logLeveldoes not affectlogHandler. - Host callback
logHandler(recommended): receives everyLogRecordfrom the SDK (the full firehose), fully independent oflogLevel/logcatEnabled, and can forward into the host's own logging system (Logcat, files, crash platforms, etc.). The callback must be fast and non-blocking and must not call back into the SDK; exceptions it throws are caught silently;LogRecord.messageis redacted and contains no Bearer token,ticket, or RTC token in plaintext.
travelId (i.e. encryptedTravelId) in SDK logs is emitted in full and is the session key sent to the server — include it when investigating a session or filing a ticket to line up client and server logs. Every HTTP response line also carries the server's requestId (the reqId= field, shown as - when absent) to pinpoint a single request; this ID appears in logs only and is never surfaced on any public return value.
9. Lifecycle & Memory
- Call
initializeinApplication.onCreate, once globally. - Idle re-initialization (for example to switch API Host or
model) closes the previous idle runtime and resets registered listeners and feature-gate state; re-register listeners after it returns. Re-initialization is rejected with103004while a Travel is starting, active, or ending: always awaitendTravel()first. The SDK never discards an active Travel as a side effect ofinitialize. - The experience is bound to the host lifecycle: call
endTravelin theonDestroyof theActivity/Fragment(or the ViewModelonCleared) to ensure the real-time connection and resources are released. - Remove the View returned by
attachVideo()from the layout on end (container.removeAllViews()), and callremoveListener. - The SDK holds only the application context; likewise, do not pass an Activity to the SDK.
10. Coroutines & Threading
- Business methods are
suspend; call them inlifecycleScope/viewModelScopefrom any coroutine context. The SDK coordinates state and RTC work on its main dispatcher while HTTP remains off the main thread. - If the caller coroutine is cancelled, the SDK cancels that call's in-flight HTTP request, if any, and propagates
CancellationExceptionunchanged rather than converting it toSDKError; do not catch or swallow it as an operational failure. Cancellation does not roll back a request that the server has already accepted. - Call
attachVideo()andsendCommand()on the main thread; an off-main call synchronously throwsIllegalStateException.
11. Token Management
- The Bearer API Key has a limited validity period; it is recommended to ensure the token is fresh before entering an experience. On receiving
onError(101002)(token expired), re-obtain the Bearer Key and callupdateToken—there is no need to exchange for a new ticket.
12. Error Recovery
- For fatal errors: clean up the current experience state (including removing the video View), prompt the user, and allow restarting.
- For network jitter (
106001) and temporary upstream service anomalies (106003): a limited number of retries may be performed. - A synchronous
100002from initialization meansmodelis blank; supply the complete model name and version, then initialize again. If a non-blankmodelhas the wrong name or version, has been retired, is not yet published, or is not authorized, the gateway typically returns AccessDenied, mapped to106003; useSDKError.rawand verify that themodel, API Host, API Key, account match.
13. Mode Adaptation
- Use the
modereturned bystartTravelto decide which interaction capabilities to expose: directing and Acting use the text instructionsendInstruct(Acting also usesaspectRatiofor player orientation and hides rewind); adventure mode uses the control commandsendCommand. - In adventure mode you may use the overload
startTravel(ticket, maxExperienceTimeSec)to cap the maximum duration of this experience (seconds; the session ends automatically when reached). The allowed values are server-configured (currently60/90/120, default60); directing and Acting ignore the value (the server ignores it and echoesnull). Passing an unsupported value makes the server return400000and this start fails. See the API Reference for parameter details.
Acting: Setting the Player Orientation from aspectRatio
StartTravelData.aspectRatio is non-null only for Acting: "9:16" (portrait, the server-side default when the world is created) or "16:9" (landscape); it is null for adventure and directing. The canvas is fixed when the world is created (specified by your server through the Open API), so for the client it is a read-only result. The SDK preserves unrecognized values verbatim, so the host must treat any value it does not recognize like null and fall back to its own default orientation.
Timing: decide the playback container's orientation after startTravel() returns and before calling attachVideo() and mounting the returned SurfaceView into your layout. The SDK only starts binding and rendering the remote stream once the host's view is attached, so deciding the orientation at that point still precedes the first rendered frame. The value is delivered with the startTravel() return value and is not guaranteed to arrive before the SDK joins the real-time communication channel — it only has to be applied before attachVideo().
Consequence: the remote view is bound with a clip-to-fill render mode — a container whose orientation disagrees with aspectRatio crops the picture (for example, a 9:16 portrait stream placed in a 16:9 container loses most of its top and bottom) instead of letterboxing it.
startTravel fails with 403007 (rejected before the Travel is created), which the SDK passes through verbatim. Do not retry this code as capacity-full; prompt to request enablement instead.14. Complete Example (ViewModel + Activity snippet)
15. DevOps Troubleshooting Guide
When something goes wrong during integration (cannot enter a Travel, black screen with no video, mid-session stream drop, pause/resume anomalies, ...), opening up the SDK logs is the fastest way to localize the issue. This chapter gives a standard troubleshooting flow — useful both for your own self-check and for handing us enough information in one round.
15.1 Enabling logs
Pick a log output as needed while troubleshooting (see §8 Logging): for a quick self-check, set logcatEnabled = true and read it with adb logcat, raising logLevel to DEBUG or VERBOSE when diagnosing (the default INFO already carries the full lifecycle timeline; drop to WARN for errors-only); to feed your own system, use logHandler to receive every LogRecord (unaffected by logLevel) and write it to your file or crash platform.
15.2 Session correlation ID: travelId
The travelId (i.e. StartTravelData.encryptedTravelId) in SDK logs is emitted in full. It is the single identifier that runs through one session and the session key sent to the server. It is the key to lining up the client logs with the server-side session records — always include the travelId of the failing session when reporting an issue.
15.3 Collecting logs
When reproducing the issue, persist the SDK logs to a file:
15.4 What to include when reporting an issue
To avoid multiple round-trips, please attach the following:
Item | Notes |
|---|---|
SDK version |
|
Session | the |
Time of occurrence | the approximate time (minute-level is fine) |
Error code | the captured |
Reproduction steps | action path + expected result + actual result |
Environment | device model, Android version, network (WiFi/cellular) |
Log file | the |
15.5 On redaction and log shareability
SDK logs are designed to be safe to share: credentials (Bearer token, Travel ticket, RTC token), RTC internal identifiers, and media URLs are redacted before being written, so they never appear in plaintext; travelId is a session identifier (not a credential) and is kept in full only for correlation. Even so, we recommend transferring log files over a trusted channel rather than pasting them publicly on uncontrolled platforms.