Effects Off-screen rendering
Effects / Concepts / Off-screen rendering

Off-screen rendering and streaming with Nosmai Effects

Updated August 27, 2026

Learn when a recording, video-call, streaming, or custom camera system needs the effected image instead of a normal Nosmai preview.

When to use it

Use off-screen rendering when Nosmai does not own the final display or media transport. Common cases include a custom camera source, a WebRTC encoder, a video-call SDK, or a server upload pipeline that already consumes native frames.

Nosmai produces the effected image and passes it to another system instead of displaying it directly. That system might record it, send it into a video call, or upload it. Standard camera-preview screens do not need this workflow.

An encoder prepares video for recording or transmission. WebRTC is a common real-time audio and video technology used by calling apps.

Supported native paths

PlatformInput or outputFormat
Android external inputCamera2 OES, preview-backed YUV, or standalone I420SDK-provided SurfaceTexture; YUV planes with strides; direct planar I420
Android camera outputCPU callback or shared GPU textureSDK frame metadata or EGL-shared texture
iOS external inputSubmitted pixel or sample bufferskCVPixelFormatType_32BGRA
iOS camera outputProcessed live-frame callbackCVPixelBufferRef
FlutterMaintained provider bridgeNative GPU texture on Android and CVPixelBuffer on iOS
React NativeBounded analysis sampling onlyBase64 i420, rgba8888, bgra8888, or nv12

React Native sampling is limited to 1 through 5 FPS and is not a live-streaming transport. The public Flutter API does not accept arbitrary external I420 or pixel-buffer input.

Choose input separately from output

Owning the camera, supplying frames, displaying a preview, and exporting processed output are separate responsibilities. On native Android, the host Camera2 owner writes to Nosmai's OES surface or submits YUV planes. An exported texture is the processed 2D result, not that external camera-input texture. The standalone I420 methods use a different CPU-plane input/output contract.

On iOS, either use the normal Nosmai camera with the Core live-output callback, or supply app-owned BGRA buffers through native off-screen processing. Do not start two capture sessions for the same camera. A maintained Flutter provider bridge keeps this routing native; do not shuttle every frame through Dart.

For React Native, analysis sampling is not a replacement for a native RTC adapter. A custom RTC source must accept the documented texture or buffer type and keep ownership until the consumer is finished.

Frame ownership

Real-time output needs clear ownership:

  1. Keep at most one expensive operation in progress.
  2. Keep the newest waiting frame instead of building a queue.
  3. Keep a consistent monotonic timestamp mapping for the downstream encoder; convert units explicitly and do not assume every API preserves source PTS.
  4. Retain an iOS output buffer only when the consumer outlives the callback.
  5. Wait for every Android texture before sampling, then complete its GPU read; release unused frames exactly once using the texture ID, not the fence handle.
  6. Clear callbacks before releasing the camera or graphics context.

Preview and processed output

Android provides PREVIEW_ONLY, STREAMING_ONLY, and DUAL_OUTPUT render modes. iOS provides a live processed-frame callback while the normal camera is running. Dual output costs more than preview alone, so measure the combined effect and encoder load on target hardware.

setOesInputFrameInfo and preview-backed processYuvFrame use Android rotation-mode IDs, while standalone I420 processing uses degrees. Do not share one unconverted rotation variable across those APIs. See the Android custom-pipeline guide for dimensions, camera orientation, mirror policy, and fence/release examples. The iOS guide covers BGRA format checks, buffer retention, and output-callback ownership.

Streaming compatibility

ConsumerStatusRoute
AgoraVerifiedNative and Flutter guides
LiveKitVerifiedNative and Flutter guides
Raw WebRTCVerified on native Android and iOSNative guides and examples
Other video SDKsIntegration-dependentUse a custom video source that accepts the documented native format

Do not describe an untested provider as verified. Format, orientation, timestamp, and buffer ownership must be validated with the exact provider version used by the application.

Next steps

References

Nosmai

An AR and AI company. Founded on the idea that the next decade of software will be visual, and that the tools for it should be within reach of anyone building.

Newsletter

Product updates and release notes. No spam.

© 2026 nosmai, inc · all rights reserved