HappyOyster iOS SDK Integration Guide

For complete method signatures, fields, and error codes, refer to the iOS SDK API Reference.

0. What You Will Build

A minimal end-to-end loop of "enter a world → real-time video experience → interact → end". The runtime entry points are the global HappyOysterEngine.shared and the single-use session handle OysterTravel it creates.

OysterStream.register()
HappyOysterEngine.shared: initialize → updateToken → createTravel
OysterTravel:             videoView / events → start → sendInstruct / sendCommand → end

1. Requirements

Item	Requirement
Minimum OS	iOS 15.0+
Language	Swift (`async/await`)
Threading	Public APIs are `@MainActor`; call them on the main thread
Import	`import HappyOysterSDK` (aggregate entry point, already `@_exported` Core + World)

Real-time communication (AliRTC) is encapsulated inside the SDK; integrators never touch the RTC API directly.

1.1 Add the SDK via CocoaPods

The SDK is distributed as a precompiled binary (xcframework) via CocoaPods subspecs, published to the public CocoaPods Trunk — reference it directly by version, no local podspec file needed.

# HappyOysterSDK / AliVCSDK_ARTC are both published on the public CocoaPods source.
source 'https://cdn.cocoapods.org/'

platform :ios, '15.0'
use_frameworks!

target 'YourApp' do
  # Aggregate entry point (Core + World); `import HappyOysterSDK` and you're set.
  pod 'HappyOysterSDK'
  # Optional default UI components (video view, control HUD).
  pod 'HappyOysterSDK/UI'
  # Video stream + AliRTC engine adapter (already depends on Stream; no need to declare it separately).
  pod 'HappyOysterSDK/StreamAliRTC'

  # RTC vendor binary: weak-linked by the SDK, not redistributed with it — bring your own (public CocoaPods source).
  pod 'AliVCSDK_ARTC', '7.11.0'
end

Then run pod install and open the generated .xcworkspace (not the .xcodeproj).

AliVCSDK_ARTC is required whenever you pull in HappyOysterSDK/StreamAliRTC: if it's missing, the SDK silently falls back to Loopback — it still connects and reaches running, but shows a black screen with no error.

2. Two Kinds of Credentials (Understand These First to Avoid Pitfalls)

The SDK does not obtain or refresh any credentials on its own; you inject all of them:

Credential	Source	Purpose	How to Inject
HTTP auth token	Your app fetches it from your own backend	General authentication for SDK calls to the gateway (long-lived, needs renewal)	`updateToken(_:)`; the SDK keeps only the latest one
One-time ticket	Issued by your server after exchanging via the Travel credential API	Used for a single experience, invalidated immediately after use	Passed as the `createTravel(ticket:)` argument

The two are not interchangeable: updateToken is for general auth, while ticket is a one-time join credential. AK / signing keys exist only on your server and are never exposed to the client.

3. Integration Steps

Lifecycle: register the stream engine → initialize → inject token → create the session → attach video + subscribe to events → start → interact → end.

Step 1: Register the Stream Engine

The single entry point for rendering video. Call it once at app launch; repeated calls are safe.

import HappyOysterSDK
import HappyOysterStream

OysterStream.register()

Step 2: Initialize the SDK

You must initialize once before calling any other API. To switch gateway or model, just call it again — while idle the latest config wins and the injected token is kept; the call is ignored only while a travel is in flight, so end() it first.

let engine = HappyOysterEngine.shared
engine.initialize(config: OysterConfig(
    apiHost: "dashscope-intl.aliyuncs.com",  // QwenCloud gateway host
    model: "happyoyster-1.0-adventure"                     // Versioned model name enabled for your account, required
))
// Optional: override the log level / signalling callback timeout
// OysterConfig(apiHost: "…", model: "…", logLevel: .debug, callbackTimeoutMs: 30_000)

// Both apiHost and model are required with no default: Happy Oyster is split into
// per-mode sub-models. See the Happy Oyster model documentation for available names
// and versions; they must match the account of your token.

Step 3: Inject the HTTP Auth Token (push)

Fetch a temporary QwenCloud API Key from your own backend and inject it. Re-inject after it expires.

let token = await fetchTokenFromYourBackend()
engine.updateToken(token)

Step 4: Create the Session Handle

Create an OysterTravel from a single-use ticket. Nothing is connected yet, but the video view is already available.

let travel = try engine.createTravel(ticket: ticket)

The SDK provides the view, the host places it. Subscribe before start() so you do not miss early status changes.

containerView.addSubview(travel.videoView)     // SwiftUI: OysterVideoView(travel:)

let eventTask = Task {
    for await event in travel.events {
        switch event {
        case .statusChanged(let status): render(status)   // running / paused / ended / failed…
        case .error(let error):          handle(error)    // error.code / error.kind — see the API Reference
        }
    }
}

Step 6: Start the Experience

Connect and start playing. The SDK then automatically maintains the real-time connection and status polling, surfacing status through events.

let data = try await travel.start()
// data.encryptedTravelId —— identifier of this experience, used for diagnostics / server reconciliation
// data.encryptedWorldId  —— identifier of the world you entered
// data.mode              —— adventure / directing / acting, determines the interaction UI
// data.aspectRatio       —— "9:16" / "16:9"; set for acting only, use it to pick the player orientation

start(maxExperienceTimeSec:) only applies to adventure; directing and acting ignore the parameter.

The world's mode for this travel must match the model you passed to initialize in Step 2. With per-mode models (happyoyster-1.0-adventure / -directing / -acting), each model is its own gateway application route, so a single initialize serves worlds of one mode only; the ticket was issued by your server under the route of that world's mode, and start() sends it to the route of the currently configured model — a mismatch fails at this step. Before entering a world of a different mode, call initialize() again with the matching model — no cleanup() and no re-updateToken():

HappyOysterEngine.shared.initialize(config: OysterConfig(
    apiHost: "…", model: "happyoyster-1.0-acting"))   // the model for this world's mode

While idle (no travel in flight) the latest config wins and the injected token is kept. The SDK cannot verify the model/world match for you upfront: mode only arrives in this step's response (data.mode); before the call the SDK holds nothing but an opaque ticket and a model name. Apps offering a single mode are unaffected and initialize once.

While a travel is in flight, a repeated initialize() call is ignored with a warning — end() it first.

Step 7: Real-Time Interaction (Pick One Based on Mode)

if data.mode == .adventure {
    // Adventure: direction/view/action control (fire-and-forget; never throws, failures come via events)
    travel.sendCommand(OysterAdventureCommand(translation: .front, interaction: .jump))
} else {
    // Directing and acting: text instructions
    _ = try await travel.sendInstruct(content: "Pan to the castle; the hero starts running")
}

sendCommand throttles internally on a 42ms (24FPS) latest-wins cycle, so the host can call it at high frequency:

One-shot actions such as jump, attack, crouch, and sprint are sent once.
Movement and view rotation are sent continuously while held, and you simply stop calling on release.
You do not need to send none or call flushCommands() on release; the SDK never generates a stop command on its own, and the server ends the action once the real-time channel goes quiet.

Step 8: Pause / Resume / Rewind (Mode-Dependent)

_ = try await travel.pause()           // supported by directing and acting
_ = try await travel.resume()
_ = try await travel.rewind(toSec: 10) // directing only

adventure supports none of these; acting supports pause/resume but not rewind — hide the rewind entry point in those modes. Mismatched calls are rejected locally by the SDK (103003 / 103002).

Step 9: End the Experience

Disconnects, stops polling, and releases all session resources; the ticket is consumed. Idempotent — every exit path must funnel into it.

_ = try? await travel.end()
eventTask.cancel()

Call await engine.cleanup() when tearing down the SDK or switching gateways.

4. Full Example (SwiftUI)

import SwiftUI
import HappyOysterSDK
import HappyOysterStream

@main
struct MyApp: App {
    init() {
        OysterStream.register()                                     // Step 1
        HappyOysterEngine.shared.initialize(config: OysterConfig(   // Step 2
            apiHost: "dashscope-intl.aliyuncs.com",
            model: "happyoyster-1.0-adventure"
        ))
    }
    var body: some Scene { WindowGroup { TravelScreen() } }
}

struct TravelScreen: View {
    @State private var travel: OysterTravel?
    @State private var eventTask: Task<Void, Never>?

    var body: some View {
        ZStack {
            if let travel {
                OysterVideoView(travel: travel)      // Step 5: SDK provides the view, host places it
                    .ignoresSafeArea()
            } else {
                Color.black.ignoresSafeArea()
            }
        }
        .task { await start() }
        .onDisappear { Task { await end() } }
    }

    @MainActor private func start() async {
        let engine = HappyOysterEngine.shared
        engine.updateToken(await fetchToken())                      // Step 3
        do {
            let ticket = await fetchTicket()                        // issued by your server
            let travel = try engine.createTravel(ticket: ticket)    // Step 4
            self.travel = travel

            eventTask = Task {                                      // Step 5
                for await event in travel.events {
                    switch event {
                    case .statusChanged(let status): print("status: \(status)")
                    case .error(let error):          print("error: \(error.code)")
                    }
                }
            }

            let data = try await travel.start()                     // Step 6
            if data.mode == .adventure {                            // Step 7
                travel.sendCommand(OysterAdventureCommand(translation: .front))
            } else {
                _ = try await travel.sendInstruct(content: "Suddenly it starts to pour")
            }
        } catch let error as OysterSDKError {
            // Handle start failure (see the error code table in the API Reference)
        } catch {}
    }

    @MainActor private func end() async {
        _ = try? await travel?.end()                                // Step 9
        eventTask?.cancel()
        travel = nil
    }
}

5. Best Practices

Lifecycle: Call OysterStream.register() + initialize() as early as possible at app launch, once each globally; always call end() when leaving the experience page to ensure the real-time connection and resources are released. OysterTravel is single-use — create a new one via createTravel after it reaches a terminal state.
Token renewal: Make sure the HTTP token is fresh before starting an experience; when you receive an auth-related error callback, fetch a new token and call updateToken (non-fatal, does not terminate the experience).
Error severity: For fatal errors, the SDK automatically terminates the current experience, surfaces them via the .error of events, and settles into the failed terminal state; you should return to the screen shown before "start experience". Non-fatal errors are only surfaced and can be retried. See the full error code table in the API Reference.
Mode adaptation: Directing and acting modes show a text input (sendInstruct); adventure mode shows control widgets (sendCommand). Only directing shows a rewind entry point. For acting worlds, pick the player orientation from aspectRatio (portrait 9:16 by default).

6. Next Steps

Full API (pause() / resume() / rewind(toSec:), data models, error codes) → iOS SDK API Reference

​0. What You Will Build

​1. Requirements

​1.1 Add the SDK via CocoaPods

​2. Two Kinds of Credentials (Understand These First to Avoid Pitfalls)

​3. Integration Steps

​Step 1: Register the Stream Engine

​Step 2: Initialize the SDK

​Step 3: Inject the HTTP Auth Token (push)

​Step 4: Create the Session Handle

​Step 5: Attach the Video View and Subscribe to Events

​Step 6: Start the Experience

​Step 7: Real-Time Interaction (Pick One Based on Mode)

​Step 8: Pause / Resume / Rewind (Mode-Dependent)

​Step 9: End the Experience

​4. Full Example (SwiftUI)

​5. Best Practices

​6. Next Steps

0. What You Will Build

1. Requirements

1.1 Add the SDK via CocoaPods

2. Two Kinds of Credentials (Understand These First to Avoid Pitfalls)

3. Integration Steps

Step 1: Register the Stream Engine

Step 2: Initialize the SDK

Step 3: Inject the HTTP Auth Token (push)

Step 4: Create the Session Handle

Step 5: Attach the Video View and Subscribe to Events

Step 6: Start the Experience

Step 7: Real-Time Interaction (Pick One Based on Mode)

Step 8: Pause / Resume / Rewind (Mode-Dependent)

Step 9: End the Experience

4. Full Example (SwiftUI)

5. Best Practices

6. Next Steps