Live streaming (Agora)

Overview

Nosmai moderates a live stream by tapping each captured video frame and running it through the SDK on-device, in real time. It works with any streaming SDK that exposes raw frames, such as Agora, LiveKit or WebRTC. Nothing about the moderation leaves the device; only the streaming SDK's own media uses the network.

The pattern is the same on every platform:

1. Start the Nosmai stream with a listener (mobile) or an analysis loop (web).
2. Tap each captured frame from the streaming SDK's raw-frame callback.
3. Push the frame to Nosmai (pushFrame on mobile, analyzeImage on web).
4. Act on the per-frame verdict (SAFE / UNSAFE): blur, cut the stream, warn the broadcaster, etc.

You do not need to change how you capture, publish or render the stream. You only add a read-only tap on the captured frames. Sample every Nth frame (about 3/sec is plenty) and do the work off the capture thread so the preview stays smooth.

Example apps

Full, working Agora + Nosmai examples for every platform live here:

https://github.com/nosmai/moderation-agora-examples

android/           Android example (Kotlin)
ios/               iOS example (Swift)
web/               Web example (JavaScript, Vite)
flutter_example/   Flutter example (Dart)

Each app streams the broadcaster's camera with Agora and moderates every frame with Nosmai, showing a live SAFE / UNSAFE verdict.

iOS (Swift + Agora)

Start the Nosmai stream, become Agora's AgoraVideoFrameDelegate, request BGRA frames, and push each captured buffer.

// 1. Start Nosmai (self conforms to NosmaiListener)
try NosmaiSDK.initialize(licenseKey: "NOSMAI-XXXX", models: [.objectDetection, .nsfw])
NosmaiSDK.startStream(listener: self)

// 2. Ask Agora for upright BGRA frames
func getVideoFormatPreference() -> AgoraVideoFormat { .cvPixelBGRA }
func getRotationApplied() -> Bool { true }

// 3. Push each captured frame (every 5th is plenty)
func onCapture(_ videoFrame: AgoraOutputVideoFrame, sourceType: AgoraVideoSourceType) -> Bool {
  if frameCount % 5 == 0, let buffer = videoFrame.pixelBuffer {
    NosmaiSDK.pushFrame(buffer, rotationDegrees: 0)   // read-only; stream is untouched
  }
  return true
}

// 4. Verdicts arrive on the listener
func nosmaiOnResult(_ result: NosmaiResult) { /* result.isUnsafe, result.nsfw, ... */ }

Android (Kotlin + Agora)

Register an IVideoFrameObserver, convert each I420 frame to a bitmap off the capture thread, and push it.

// 1. Start Nosmai
NosmaiSDK.startStream(object : NosmaiListener {
  override fun onResult(result: NosmaiResult) { /* result.isUnsafe, result.nsfw, ... */ }
})

// 2. Tap Agora's captured frames (read-only)
rtcEngine.registerVideoFrameObserver(object : IVideoFrameObserver {
  override fun onCaptureVideoFrame(sourceType: Int, frame: VideoFrame?): Boolean {
    if (frameCount % 5 == 0) worker.execute {   // off the capture thread, drop-if-busy
      val bitmap = i420ToBitmap(frame!!.buffer.toI420())
      NosmaiSDK.pushFrame(bitmap, 0)
    }
    return true
  }
  override fun getVideoFrameProcessMode() = IVideoFrameObserver.PROCESS_MODE_READ_ONLY
  override fun getVideoFormatPreference() = IVideoFrameObserver.VIDEO_PIXEL_I420
  // ... other overrides
})

Web (JavaScript + Agora)

Read the Agora local track into a <video> element and analyze it on a loop.

import AgoraRTC from "agora-rtc-sdk-ng";
import { NosmaiModeration } from "@nosmai/moderation-web";

await NosmaiModeration.initialize("NOSMAI-XXXX");
await NosmaiModeration.initializeNsfw({ modelUrl: "/models/nsm_nsfw.onnx" });
await NosmaiModeration.initializeDetector({ modelUrl: "/models/nsm_detector.onnx" });

const cameraTrack = await AgoraRTC.createCameraVideoTrack();
await client.publish([cameraTrack]);
video.srcObject = new MediaStream([cameraTrack.getMediaStreamTrack()]);
await video.play();

setInterval(async () => {
  const r = await NosmaiModeration.analyzeImage(video);   // any CanvasImageSource
  render(r.isUnsafe ? "UNSAFE" : "SAFE");
}, 1000);

Flutter (Agora bridge)

The moderation_agora_bridge_flutter package does the native frame plumbing for you, so your existing Agora code changes by one line. It ships no SDK of its own; results come from nosmai_moderation_sdk.

Install it alongside your existing agora_rtc_engine and nosmai_moderation_sdk:

dependencies:
  moderation_agora_bridge_flutter:
    git:
      url: https://github.com/nosmai/moderation_agora_bridge_flutter.git

// 1. Nosmai, as usual, then start live detection on external frames
await NosmaiModeration.initialize('NOSMAI-XXXX', models: [NosmaiModel.objectDetection, NosmaiModel.nsfw]);
await NosmaiLive.startExternal();
final sub = NosmaiLive.results().listen((r) { /* r.isUnsafe, r.nsfw, ... */ });

// 2. Create the Agora engine from the bridge's shared handle (the only changed line)
final handle = await ModerationAgoraBridge.getNativeHandle(agoraAppId: appId);
final engine = createAgoraRtcEngine(sharedNativeHandle: handle);

// 3. ... your ordinary Agora setup (initialize / join / preview) is unchanged ...

// On camera switch: await engine.switchCamera(); await ModerationAgoraBridge.notifyCameraSwitch();
// Teardown:        await NosmaiLive.stop(); await ModerationAgoraBridge.disposeNative();

Notes

• The frame tap is read-only: Nosmai never modifies the outgoing stream. Only the per-frame verdict is returned.
• Push a sampled subset of frames (about 3/sec). The detector samples again internally, so pushing every frame just wastes a copy.
• Do the frame conversion off the capture thread and drop a frame if a previous one is still processing, so a slow device never stalls the preview.
• The same tap works for any raw-frame source (a token server, screen share, or another WebRTC SDK), not just Agora.