# Nosmai

> Nosmai provides two on-device mobile SDKs. Nosmai Effects adds real-time camera filters, beauty, makeup, AR face effects, backgrounds, screen capture and live streaming. Nosmai Moderation flags NSFW imagery and moderates toxic text. Both run fully on-device, so no frame or message ever leaves the phone, and both are available for iOS, Android, Flutter and React Native. The concise index is at https://nosmai.com/llms.txt. This file concatenates the full documentation for both products, Effects first, then Moderation.

---

# Nosmai Effects

> On-device camera SDK documentation for real-time filters, beauty, makeup,
> face effects, backgrounds, capture and live streaming on iOS, Android and
> Flutter. This corpus contains public integration guidance only.

---

# Introduction
Source: https://docs.nosmai.com/docs/effects/introduction/

## What is Nosmai Effects?

Nosmai Effects is a camera SDK for adding real-time visual effects to mobile applications. It processes camera frames on the device and displays the result in a live preview. The same processed output can also be used for photos, video recording, and live streaming.

The SDK handles the difficult parts of a camera effects experience, including:

- real-time frame processing
- GPU-accelerated rendering
- face detection and face tracking
- filter and effect management
- beauty and makeup placement
- background segmentation and replacement
- camera lifecycle management
- processed output for recording and live streaming

Your application remains responsible for its screens, buttons, navigation, product rules, and user experience. Nosmai provides the camera and effects capabilities that those screens control.

## What you can build

| Capability | Examples |
| --- | --- |
| Camera preview | Front and back camera preview with real-time processing |
| Color filters | LUT looks, brightness, contrast, hue, saturation, RGB, and white balance |
| Beauty | Skin smoothing, skin whitening, sharpening, and teeth whitening |
| Makeup | Lipstick, eyeshadow, blusher, eyelashes, and eyebrows |
| Face shaping | Face slimming, eye size, nose, chin, jaw, lips, forehead, and brow controls |
| Eye effects | Eye color adjustment and face-tracked eye effects |
| AR effects | Face masks, stickers, particles, animated overlays, and 3D content |
| Backgrounds | Blur, solid color, image, video, and packaged background effects |
| Capture | Photos and videos with active effects included |
| Live streaming | Processed camera output for services such as Agora |
| Cloud filters | Browse, download, cache, and apply filters without shipping every filter inside the app |

## How the SDK is used

A basic Nosmai integration has four steps:

1. Create a project and license key in the [Nosmai Console](https://console.nosmai.com/).
2. Add the Nosmai SDK package for your platform.
3. Initialize the SDK and show the camera preview.
4. Apply a built-in feature or a `.nosmai` filter.

The SDK automatically handles face detection when an active feature needs it. An application does not need to start a separate face detector before applying lipstick, face shaping, or a face-tracked effect.

## Two types of visual features

Nosmai provides built-in features and external `.nosmai` packages.

### Built-in features

Built-in features are part of the SDK and are controlled through direct methods. They are suitable for settings that an application wants to adjust continuously with sliders, buttons, or saved presets.

Examples include:

- skin smoothing
- skin whitening
- lipstick
- eyeshadow
- blusher
- eyelashes
- eyebrows
- face shaping
- eye color
- brightness and contrast

### External `.nosmai` packages

A `.nosmai` package is a protected filter file that contains the resources and instructions for one visual experience. The application passes the package path to `applyEffect`, and the SDK reads the package type and applies it correctly.

The supported package types are:

| Type | Intended use |
| --- | --- |
| `filter` | Color grading, LUT effects, and full-frame visual filters |
| `effect` | AR effects, face masks, stickers, particles, and 3D content |
| `beauty_effect` | Packaged makeup and face-mesh beauty effects |
| `background` | Packaged background replacement effects |

Applications do not need separate apply methods for these four package types. Use `applyEffect` for all of them.

## Local and cloud filters

Filters can reach the application in two ways.

### Local filters

Local filters are included in the application bundle. They are available immediately and do not require a download before use.

Use local filters when:

- a filter is required for the first camera session
- the application must work without a network connection
- the filter library is small
- a specific branded effect must always be available

### Cloud filters

Cloud filters are listed through the Nosmai cloud catalog and downloaded when required. Once downloaded, a cloud filter is stored locally and uses the same `applyEffect` method as a bundled filter.

Use cloud filters when:

- the filter library is large
- new filters must be published without releasing a new application version
- filters are seasonal or temporary
- reducing the initial application download size is important

Cloud catalog requests and downloads need a network connection. Applying an already downloaded filter does not upload camera frames.

## On-device processing

Camera frame processing happens on the user's device. The SDK uses device hardware acceleration where available to keep the preview responsive.

This provides three important benefits:

- **Privacy:** camera frames do not need to be uploaded for filters, beauty, face tracking, or background processing.
- **Low latency:** the effect is rendered directly on the device, without waiting for a response from a remote image-processing service.
- **Predictable operation:** active effects can continue to work when the network is unavailable, subject to license validation and any assets that still need to be downloaded.

> [!NOTE]
> The license service and cloud filter catalog use the network. This is separate from camera frame processing. Nosmai does not need to upload every camera frame to apply an effect.

## Supported platforms

| Platform | Current package | Minimum requirement |
| --- | --- | --- |
| iOS | [`NosmaiCameraSDK` 3.0.0](https://github.com/nosmai/camera-sdk-ios) | iOS 15.0+, physical arm64 device |
| Android | [Nosmai Android AAR 3.0.0](https://github.com/nosmai/camera-sdk-android) | Android API 21+, arm64-v8a device |
| Flutter | [`nosmai_camera_sdk` 3.0.6](https://pub.dev/packages/nosmai_camera_sdk) | Flutter 3.22+, iOS 15.0+ or Android API 21+ |

The current release is intended for physical mobile devices. Camera processing and device hardware acceleration should be tested on real iOS and Android devices before release.

The iOS release is a dynamic `nosmai.framework`, distributed through the `NosmaiCameraSDK` CocoaPod and as a verified ZIP from GitHub. The Flutter package resolves this iOS pod automatically. On Android, Flutter applications must download the proprietary AAR separately and add it to the application module. The AAR is intentionally not included in the pub.dev archive.

React Native and Web are not part of the current SDK documentation set. They should only be documented when an officially supported package is available.

## Official SDK repositories

Use the official repositories for package releases, example applications, change history, and platform-specific issue reports:

| Platform | Repository |
| --- | --- |
| Android | [nosmai/camera-sdk-android](https://github.com/nosmai/camera-sdk-android) |
| iOS | [nosmai/camera-sdk-ios](https://github.com/nosmai/camera-sdk-ios) |
| Flutter | [nosmai/nosmai_camera_sdk_flutter](https://github.com/nosmai/nosmai_camera_sdk_flutter) |

Always use the documentation that matches the installed SDK version. A repository default branch or older README may describe a previous release.

## Performance expectations

Actual frame rate depends on:

- device GPU and processor
- camera resolution
- target frame rate
- number and complexity of active features
- face detection requirements
- background segmentation
- recording or live streaming at the same time
- device temperature and power-saving state

Nosmai is designed for real-time use, but applications should still test low-end and high-end target devices. A stable 30 FPS experience is often a better default than requesting 60 FPS on hardware that cannot sustain it.

## What the SDK does not decide

Nosmai provides camera and effects technology. Your application still decides:

- which filters are shown to each user
- whether a feature is free or paid
- which effect is selected by default
- when recording starts and stops
- how captured media is stored or uploaded
- how live-stream access is authenticated
- how camera and microphone permissions are explained
- how failures are presented to the user

## Before continuing

You will need:

- a Nosmai account
- a project in the [Nosmai Console](https://console.nosmai.com/)
- a valid license key for the application
- the final Android package name or iOS bundle identifier
- a physical supported device
- camera permission
- microphone permission if the application records audio or streams with audio
- internet access for initial license verification

Continue with the [Quickstart](/docs/effects/quickstart) to install the SDK and display your first camera preview.

Use [Releases and compatibility](/docs/effects/releases-and-compatibility) when selecting native and Flutter versions for a production application.

---

# Quickstart
Source: https://docs.nosmai.com/docs/effects/quickstart/

## What you will build

This guide covers the minimum setup required to:

1. obtain a Nosmai license key
2. add the SDK to an application
3. request the required permissions
4. initialize Nosmai
5. display a live camera preview
6. apply a simple built-in effect

Choose the section for your platform. The platform guides provide the complete production setup, lifecycle handling, camera controls, and error handling.

## Before you start

### Create a project

Open the [Nosmai Console](https://console.nosmai.com/), sign in, and create a project for the application.

Use the exact application identifier that will be present in the installed app:

| Platform | Application identifier |
| --- | --- |
| Android | Package name, such as `com.example.cameraapp` |
| iOS | Bundle identifier, such as `com.example.cameraapp` |
| Flutter | Android package name and iOS bundle identifier for the two native applications |

Copy the license key issued for the project. Nosmai license keys use the following format:

```text
NOSMAI-<project-key>
```

Do not use a key issued for a different package name, bundle identifier, or platform.

### Use a physical device

Use an arm64 iPhone or an `arm64-v8a` Android device. A simulator or emulator can help with layout development, but it is not a reliable environment for camera effects, face tracking, recording, or performance testing.

### Keep the first launch online

The SDK verifies the license with the Nosmai service. Keep the test device connected to the internet during the first successful launch.

## Flutter

Flutter is the shortest path to a complete cross-platform preview because `NosmaiCameraPreview` owns the native camera view on both platforms.

### 1. Add the package

The Flutter plugin, example application, and release history are available in the [Nosmai Flutter SDK repository](https://github.com/nosmai/nosmai_camera_sdk_flutter).

Add the current package to `pubspec.yaml`:

```yaml title="pubspec.yaml"
dependencies:
  flutter:
    sdk: flutter
  nosmai_camera_sdk: ^3.0.6
```

Run:

```sh
flutter pub get
```

### 2. Configure iOS

The Flutter plugin resolves the native `NosmaiCameraSDK` dependency through CocoaPods. Do not copy an iOS framework into the Flutter application manually.

Set the minimum iOS version in the application Podfile:

```ruby title="ios/Podfile"
platform :ios, '15.0'
```

Install the native pod and open the generated workspace when working in Xcode:

```sh
cd ios
pod install --repo-update
```

Add the required permission descriptions:

```xml title="ios/Runner/Info.plist"
<key>NSCameraUsageDescription</key>
<string>This app uses the camera for real-time filters and effects.</string>
<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone when recording video or streaming.</string>
<key>NSPhotoLibraryAddUsageDescription</key>
<string>This app saves captured photos and videos to your library.</string>
```

Camera permission is required for preview. Microphone permission is only required when audio is recorded or streamed. Photo library permission is only required when the app saves media to the user's library.

### 3. Configure Android

The proprietary Android SDK is distributed separately and is not included in the pub.dev package.

1. Download `nosmai-sdk-3.0.0.aar` and `SHA256SUMS` from the [Android SDK v3.0.0 release](https://github.com/nosmai/camera-sdk-android/releases/tag/v3.0.0).
2. Run `shasum -a 256 -c SHA256SUMS` in the download directory.
3. Rename the verified AAR to `nosmai-release.aar`.
4. Place it at `android/app/libs/nosmai-release.aar`.

Add the local AAR repository to the Flutter application's Android project:

```gradle title="android/build.gradle"
allprojects {
    repositories {
        google()
        mavenCentral()
        flatDir { dirs "${rootProject.projectDir}/app/libs" }
    }
}
```

Add the AAR to the application module:

```gradle title="android/app/build.gradle"
dependencies {
    implementation files('libs/nosmai-release.aar')
}

android {
    defaultConfig {
        minSdk 21

        ndk {
            abiFilters "arm64-v8a"
        }
    }
}
```

Add the required permissions:

```xml title="android/app/src/main/AndroidManifest.xml"
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.INTERNET" />

    <application>
        ...
    </application>
</manifest>
```

The Flutter Dart package and native Android AAR are versioned separately. Flutter package `3.0.6` is compatible with Android native SDK `3.0.0`.

### 4. Initialize the SDK

Initialize once before opening a camera screen:

```dart title="lib/main.dart"
import 'package:flutter/material.dart';
import 'package:nosmai_camera_sdk/nosmai_camera_sdk.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  final initialized = await NosmaiFlutter.initialize(
    'NOSMAI-YOUR-LICENSE-KEY',
  );

  if (!initialized) {
    throw StateError('Nosmai SDK initialization failed');
  }

  runApp(const MyApp());
}
```

Do not initialize Nosmai every time a widget rebuilds. Initialize it once at application startup or through a single application-level service.

### 5. Display the camera

```dart title="lib/camera_screen.dart"
import 'package:flutter/material.dart';
import 'package:nosmai_camera_sdk/nosmai_camera_sdk.dart';

class CameraScreen extends StatelessWidget {
  const CameraScreen({super.key});

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Stack(
        fit: StackFit.expand,
        children: [
          NosmaiCameraPreview(
            onInitialized: () {
              debugPrint('Nosmai camera is ready');
            },
            onError: (error) {
              debugPrint('Nosmai camera error: $error');
            },
          ),
        ],
      ),
    );
  }
}
```

`NosmaiCameraPreview` creates the platform camera view and starts processing when the native preview is ready.

### 6. Apply a built-in effect

Apply subtle skin smoothing after the camera is ready:

```dart
final nosmai = NosmaiFlutter.instance;
await nosmai.applySkinSmoothing(0.4);
```

Remove built-in beauty effects when required:

```dart
await nosmai.removeAllBeautyEffects();
```

> [!NOTE]
> Built-in effect ranges are documented per method. Do not assume every beauty control uses the same numeric range.

## iOS

The iOS SDK provides `NosmaiCore` as the high-level entry point. It exposes camera and effects objects after initialization.

### 1. Add the framework

Get the framework, examples, and release information from the [Nosmai iOS SDK repository](https://github.com/nosmai/camera-sdk-ios).

The recommended installation uses CocoaPods:

```ruby title="Podfile"
platform :ios, '15.0'

target 'CameraApp' do
  use_frameworks!
  pod 'NosmaiCameraSDK', '3.0.0'
end
```

Run `pod install --repo-update`, then open the generated `.xcworkspace`.

For manual installation, download `nosmai.framework.zip` and `SHA256SUMS` from the [iOS SDK v3.0.0 release](https://github.com/nosmai/camera-sdk-ios/releases/tag/v3.0.0). Verify the archive with `shasum -a 256 -c SHA256SUMS`, unzip it, add `nosmai.framework` to the Xcode project, and set it to **Embed & Sign** for the application target.

Confirm:

- deployment target is iOS 15.0 or later
- the app target includes the framework
- the framework is embedded in the final application
- the target uses a physical arm64 device for camera testing

### 2. Add permission descriptions

```xml title="Info.plist"
<key>NSCameraUsageDescription</key>
<string>This app uses the camera for real-time filters and effects.</string>
<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone when recording video or streaming.</string>
<key>NSPhotoLibraryAddUsageDescription</key>
<string>This app saves captured photos and videos to your library.</string>
```

### 3. Initialize and start the preview

The following Objective-C example creates a preview view, initializes the SDK, attaches the camera, and starts capture:

```objc title="CameraViewController.m"
#import <nosmai/Nosmai.h>

@interface CameraViewController ()
@property(nonatomic, strong) UIView *cameraPreview;
@end

@implementation CameraViewController

- (void)viewDidLoad {
    [super viewDidLoad];

    self.cameraPreview = [[UIView alloc] initWithFrame:self.view.bounds];
    self.cameraPreview.autoresizingMask =
        UIViewAutoresizingFlexibleWidth | UIViewAutoresizingFlexibleHeight;
    [self.view addSubview:self.cameraPreview];

    [[NosmaiCore shared]
        initializeWithAPIKey:@"NOSMAI-YOUR-LICENSE-KEY"
                  completion:^(BOOL success, NSError *error) {
        if (!success) {
            NSLog(@"Nosmai initialization failed: %@", error.localizedDescription);
            return;
        }

        NosmaiCameraConfig *config = [[NosmaiCameraConfig alloc] init];
        config.position = NosmaiCameraPositionFront;
        config.sessionPreset = AVCaptureSessionPresetHigh;
        config.frameRate = 30;

        [[[NosmaiCore shared] camera] updateConfiguration:config];
        [[[NosmaiCore shared] camera] attachToView:self.cameraPreview];

        BOOL started = [[[NosmaiCore shared] camera] startCapture];
        if (!started) {
            NSLog(@"Nosmai camera could not start");
        }
    }];
}

@end
```

The initialization completion is delivered on the main thread. Attach and start the camera only after initialization succeeds.

### 4. Apply a built-in effect

```objc
NosmaiEffectsEngine *effects = [[NosmaiCore shared] effects];
[effects applySkinSmoothing:0.4f];
```

## Android

The Android SDK provides `NosmaiSDK` for initialization and processing, `NosmaiPreviewView` for display, and `NosmaiEffects` for external `.nosmai` packages.

The native Android SDK accepts camera frames from the application. The complete sample includes a Camera2 helper that demonstrates the recommended camera connection. Use that helper or connect an existing Camera2 source to `NosmaiPreviewView`.

### 1. Add the AAR

Get the AAR, sample application, and release information from the [Nosmai Android SDK repository](https://github.com/nosmai/camera-sdk-android).

Place the supplied Android SDK AAR in:

```text
app/libs/nosmai-sdk-3.0.0.aar
```

Download `SHA256SUMS` with the AAR and verify it before integration:

```sh
shasum -a 256 -c SHA256SUMS
```

Reference it from the application module:

```kotlin title="app/build.gradle.kts"
dependencies {
    implementation(files("libs/nosmai-sdk-3.0.0.aar"))
}
```

The current Android SDK requires:

- minimum API 21
- Java 11 compatibility
- an `arm64-v8a` device

### 2. Add permissions

```xml title="app/src/main/AndroidManifest.xml"
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    <uses-permission android:name="android.permission.INTERNET" />

    <application>
        ...
    </application>
</manifest>
```

Request camera permission at runtime before starting the camera. Request microphone permission before recording or streaming audio.

### 3. Initialize once

Initialize from the application class or another application-level owner:

```java title="NosmaiApplication.java"
import android.app.Application;
import com.nosmai.effect.api.NosmaiSDK;

public final class NosmaiApplication extends Application {
    @Override
    public void onCreate() {
        super.onCreate();
        NosmaiSDK.initialize(this, "NOSMAI-YOUR-LICENSE-KEY");
    }
}
```

Register the application class:

```xml title="AndroidManifest.xml"
<application
    android:name=".NosmaiApplication"
    ...>
</application>
```

### 4. Create the processing preview

```java title="CameraActivity.java"
import android.os.Bundle;
import android.widget.FrameLayout;
import androidx.appcompat.app.AppCompatActivity;
import com.nosmai.effect.api.NosmaiPreviewView;
import com.nosmai.effect.api.NosmaiSDK;

public final class CameraActivity extends AppCompatActivity {
    private NosmaiPreviewView previewView;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_camera);

        FrameLayout container = findViewById(R.id.preview_container);
        previewView = new NosmaiPreviewView(this);
        container.addView(
            previewView,
            new FrameLayout.LayoutParams(
                FrameLayout.LayoutParams.MATCH_PARENT,
                FrameLayout.LayoutParams.MATCH_PARENT
            )
        );

        NosmaiSDK.startProcessing(previewView);

        // Connect Camera2 after camera permission is granted.
        // The Android platform guide contains the complete camera input example.
    }

    @Override
    protected void onResume() {
        super.onResume();
        if (previewView != null) {
            previewView.onResume();
        }
    }

    @Override
    protected void onPause() {
        if (previewView != null) {
            previewView.onPause();
        }
        super.onPause();
    }

    @Override
    protected void onDestroy() {
        // Stop the Camera2 source before stopping Nosmai processing.
        NosmaiSDK.stopProcessing();
        super.onDestroy();
    }
}
```

Starting processing prepares the preview and effects. The application must then connect its Camera2 source. Do not start more than one camera source for the same preview.

### 5. Apply a built-in effect

```java
import com.nosmai.effect.api.NosmaiBeauty;

NosmaiBeauty.applySkinSmoothing(0.4f);
```

## Apply a `.nosmai` package

After the camera preview works, all platforms use one apply method for external packages.

**Flutter**

```dart
final applied = await NosmaiFlutter.instance.applyEffect(filter.path);
```

**iOS**

```objc
[[[NosmaiCore shared] effects]
    applyEffect:filterPath
      completion:^(BOOL success, NSError *error) {
    if (!success) {
        NSLog(@"Effect failed: %@", error.localizedDescription);
    }
}];
```

**Android**

```java
NosmaiEffects.applyEffect(filterPath, new NosmaiEffects.EffectCallback() {
    @Override
    public void onSuccess() {
        // Update the selected state in the application UI.
    }

    @Override
    public void onError(String message) {
        // Show or record the failure.
    }
});
```

The value passed to `applyEffect` must identify a valid local `.nosmai` package. Cloud packages must be downloaded before they can be applied.

## Verify the integration

Before adding a large filter library, verify this basic flow:

1. the application starts without an initialization error
2. the operating system asks for camera permission
3. the front camera preview appears
4. the preview orientation is correct
5. the front preview uses the intended mirror setting
6. the built-in test effect changes the preview
7. sending the app to the background releases or pauses the camera
8. returning to the app restores the preview
9. leaving the camera screen turns off the operating system camera indicator

## Common first-run problems

| Symptom | Check |
| --- | --- |
| Initialization fails | Confirm the key, package name, bundle identifier, internet connection, and project status |
| Camera is black | Confirm runtime camera permission and that the camera source starts after the preview exists |
| Android install fails | Confirm the device supports `arm64-v8a` |
| iOS framework does not load | Confirm `NosmaiCameraSDK` is installed, the workspace is open, and the deployment target is iOS 15.0+ |
| Flutter Android cannot find Nosmai classes | Confirm the verified AAR exists at `android/app/libs/nosmai-release.aar` and is added to the app module |
| Flutter preview does not return after navigation | Keep initialization at application level and let `NosmaiCameraPreview` own native view disposal |
| Effect does not appear | Confirm the file exists, the package is valid, and `applyEffect` reports success |

## Next steps

- [Authentication](/docs/effects/authentication) explains license keys, project identity, verification, caching, and production handling.
- [Core concepts](/docs/effects/concepts) explains camera input, processing, effect types, built-in beauty, local filters, cloud filters, and output.

The dedicated Android, iOS, and Flutter guides will add complete platform-specific integration, lifecycle, capture, recording, and troubleshooting instructions.

---

# Authentication
Source: https://docs.nosmai.com/docs/effects/authentication/

## Overview

Nosmai Effects requires a valid license key. The key identifies the Nosmai project, the application, the platform, and the features that the application is allowed to use.

Create and manage projects in the [Nosmai Console](https://console.nosmai.com/).

A license key uses the following format:

```text
NOSMAI-<project-key>
```

Initialize the SDK with this key before using the camera, filters, beauty features, recording, or processed live output.

## Create a project

The normal setup is:

1. Sign in to the [Nosmai Console](https://console.nosmai.com/).
2. Create a project for the application.
3. Add the Android package name or iOS bundle identifier.
4. Select the required platform and product capabilities.
5. Copy the generated license key.
6. Initialize the SDK with that key.

Use the application's final identifier. A key issued for a temporary identifier may stop working after the application is renamed.

## Application identity

Nosmai validates the installed application identity.

### Android

Android uses the application ID from the app module:

```kotlin title="app/build.gradle.kts"
android {
    defaultConfig {
        applicationId = "com.example.cameraapp"
    }
}
```

The value configured in the Nosmai Console must match the installed application's package name.

Do not use the SDK library namespace as the application identifier. The license belongs to the consuming application.

### iOS

iOS uses the bundle identifier configured for the application target:

```text
com.example.cameraapp
```

The value configured in the Nosmai Console must match the bundle identifier in the signed application.

### Flutter

A Flutter application has two native identities:

- the Android package name
- the iOS bundle identifier

Configure both applications correctly in the Nosmai Console. Do not assume that one platform key can automatically be used by the other platform unless the console project explicitly issues and supports that configuration.

## Initialize once

Initialize Nosmai once for the current application process.

**Flutter**

```dart
final initialized = await NosmaiFlutter.initialize(
  'NOSMAI-YOUR-LICENSE-KEY',
);

if (!initialized) {
  // Do not open the Nosmai camera screen.
}
```

**iOS**

```objc
[[NosmaiCore shared]
    initializeWithAPIKey:@"NOSMAI-YOUR-LICENSE-KEY"
              completion:^(BOOL success, NSError *error) {
    if (!success) {
        NSLog(@"Nosmai initialization failed: %@", error.localizedDescription);
    }
}];
```

**Android**

```java
NosmaiSDK.initialize(
    applicationContext,
    "NOSMAI-YOUR-LICENSE-KEY"
);
```

Do not initialize the SDK from a list item, widget build method, frame callback, or every camera-screen appearance.

## What verification checks

License verification can check:

- license key format
- project status
- application package name or bundle identifier
- platform
- license validity
- enabled product capabilities
- license response integrity

If the supplied key belongs to another application, the cached license for that other key is not reused.

## Network behavior

The device needs access to the Nosmai licensing service for the first successful verification.

After a successful verification:

The SDK may use securely stored license state for limited offline operation,
according to the active SDK version and commercial plan. When that offline
period is no longer available, the device must reconnect and verify again.

The cache is tied to the license key. Supplying a different key does not make the previous key's cached result valid.

> [!NOTE]
> The exact license policy may change by SDK version or commercial plan. Applications should handle license status changes instead of assuming that cached access is permanent.

## License verification and camera frames

License verification sends application and license information to the Nosmai licensing service. It does not require uploading every camera frame.

Real-time filters, beauty, face tracking, and background processing run on the device. Cloud filter listing and filter downloads use separate network requests.

## Temporary network failures

A temporary DNS failure, timeout, or unavailable network does not always mean the license key is invalid.

Treat these cases differently:

| Situation | Meaning |
| --- | --- |
| Invalid key format | The supplied key is malformed |
| Package or bundle mismatch | The key belongs to another application identity |
| Expired or disabled project | The project is no longer authorized |
| DNS failure or timeout | The licensing service could not be reached |
| Valid offline license state | The SDK can continue without a new online response |
| Offline access unavailable | The device must reconnect before licensed use continues |

Do not show "invalid license" to a user when the actual failure is a temporary network error. Record the technical error and present a retry action.

## Development and production keys

Use separate projects or keys for development and production.

| Environment | Recommended identifier | Purpose |
| --- | --- | --- |
| Development | Development package name or bundle identifier | Local development and internal testing |
| Staging | Staging application identifier | QA, release candidates, and automated testing |
| Production | Store application identifier | Public App Store or Play Store release |

This separation prevents:

- a test build using production access by accident
- an internal package name failing against a production key
- unclear usage reporting
- development changes affecting a released application

## Store keys safely

A mobile license key must be included in the installed application so the SDK can initialize. It should not be treated like a backend password that can never reach a device.

Still follow these rules:

- do not commit production keys to a public repository
- do not place keys in screenshots, tutorials, issue reports, or public sample apps
- use build configuration or environment files for internal development
- keep development and production keys separate
- rotate a key through the Nosmai Console if it is exposed unexpectedly
- never send the key to analytics as an event value
- never print the full key in release logs

The SDK verifies the key against the application identity and the license response. Possessing the text value alone should not authorize an unrelated application.

## Example build configuration

### Android

Expose the key through a build configuration field:

```kotlin title="app/build.gradle.kts"
android {
    defaultConfig {
        buildConfigField(
            "String",
            "NOSMAI_LICENSE_KEY",
            "\"${project.findProperty("NOSMAI_LICENSE_KEY") ?: ""}\""
        )
    }
}
```

Use it at startup:

```java
NosmaiSDK.initialize(
    getApplicationContext(),
    BuildConfig.NOSMAI_LICENSE_KEY
);
```

Do not commit the local property containing the real production value.

### iOS

Use an `.xcconfig` value:

```text title="Config/Secrets.xcconfig"
NOSMAI_LICENSE_KEY = NOSMAI-YOUR-LICENSE-KEY
```

Map it into `Info.plist` through a build setting and read it at startup. Keep the real secrets file outside public source control.

### Flutter

Use a compile-time environment value:

```sh
flutter run \
  --dart-define=NOSMAI_LICENSE_KEY=NOSMAI-YOUR-LICENSE-KEY
```

Read and validate it:

```dart
const licenseKey = String.fromEnvironment('NOSMAI_LICENSE_KEY');

if (licenseKey.isEmpty) {
  throw StateError('NOSMAI_LICENSE_KEY is missing');
}

final initialized = await NosmaiFlutter.initialize(licenseKey);
```

Build automation must supply the correct key for the selected environment.

## Handle initialization failure

Do not continue into the camera experience as though initialization succeeded.

A production application should:

1. stop the loading state
2. record the technical error without the full key
3. distinguish a connection problem from a rejected license where possible
4. show a retry action for temporary failures
5. prevent paid or licensed features from being used without authorization
6. send the user back to a safe screen if the camera experience cannot start

## Retry behavior

Retry after:

- network connectivity returns
- a temporary DNS failure
- a timeout
- the application returns to the foreground after a failed startup

Do not retry continuously in a tight loop. Use a user action or a limited delay between attempts.

On iOS, `NosmaiCore` provides license status and retry methods. Flutter and Android should use their platform status callbacks or initialize again only through the application's single SDK owner.

## Changing the active key

Do not switch keys while the SDK is actively processing camera frames.

For a real environment change:

1. stop camera capture
2. stop recording or streaming
3. detach the preview
4. clean up the SDK
5. initialize again with the new key

Normal screen navigation does not require changing or reinitializing the key.

## Production checklist

Before release, confirm:

- the production package name or bundle identifier matches the console project
- the production key is supplied by release automation
- the test key is not present in the release build
- the app has a clear initialization failure state
- temporary connection failures can be retried
- full keys are not written to logs
- the first-launch network requirement is covered by testing
- offline behavior is tested after a successful online verification
- the project includes every feature used by the application

For product information and account access, visit [nosmai.com](https://nosmai.com/) and the [Nosmai Console](https://console.nosmai.com/).

---

# Core concepts
Source: https://docs.nosmai.com/docs/effects/concepts/

## The basic flow

A Nosmai camera experience has four parts:

```text
Camera input
    |
    v
Nosmai processing
    |
    v
Preview and active effects
    |
    v
Photo, recording, or live-stream output
```

The application supplies or starts the camera, Nosmai processes each frame, and the processed result is displayed or delivered to the requested output.

## Camera input

Camera input is the original frame before a visual effect is applied.

Nosmai can work with:

- the SDK-managed camera on iOS
- the SDK-managed platform camera view in Flutter
- Camera2 input connected to `NosmaiPreviewView` on Android
- external video frames for custom camera, video, or streaming integrations

The application should have one active owner for a camera source. Starting two camera sources for the same preview can cause a black screen, duplicated work, incorrect orientation, or camera access failures.

## Processing

Processing is the work that turns an input frame into the final visual frame.

Depending on the active features, this can include:

- orientation and mirroring
- color adjustment
- face detection
- face landmark tracking
- skin and makeup rendering
- face shaping
- AR effect rendering
- background segmentation
- composition of the final frame

Nosmai automatically enables the work required by the active feature. A simple color filter does not need the same face analysis as lipstick or face shaping.

## Preview

The preview is the live processed camera image shown inside the application.

| Platform | Preview API |
| --- | --- |
| Flutter | `NosmaiCameraPreview` |
| iOS | Attach `NosmaiCore.shared.camera` to a `UIView` |
| Android | `NosmaiPreviewView` |

The preview should have a stable size and remain attached while camera processing is active. Recreating it repeatedly can force camera and graphics resources to be rebuilt.

## Start, pause, stop, and cleanup

These actions have different meanings.

| Action | Purpose |
| --- | --- |
| Start | Begin camera capture and frame processing |
| Pause | Temporarily release or pause camera use while keeping reusable SDK state |
| Resume | Restore a paused camera session |
| Stop | End the current processing session |
| Cleanup | Release SDK resources when the application is truly finished with the SDK |

Use pause and resume for temporary application backgrounding. Use stop and view detachment when leaving a camera screen. Use full cleanup only when the SDK will not be needed again without a new initialization.

Do not call full cleanup on every widget rebuild or every short navigation event.

## Orientation and mirroring

Camera sensor orientation, device orientation, and preview orientation are separate values.

The final preview must account for:

- front or back camera
- camera sensor orientation
- portrait or landscape display
- application rotation support
- front-camera mirror preference

Front preview mirroring is usually a user-interface choice. Recording and live-stream output may need a different mirror setting from the local preview.

Apply mirroring in one place. Applying it twice can cancel the visual flip or make face-tracked effects appear reversed.

## Built-in features

Built-in features are SDK methods that can be adjusted directly while the camera runs.

They are grouped into several categories.

### Beauty

- skin smoothing
- skin whitening
- sharpening
- teeth whitening

### Makeup

- lipstick
- eyeshadow
- blusher
- eyelashes
- eyebrows

### Face shaping

- lips
- face slimming
- eye size
- nose
- chin
- brow
- brow thickness
- jaw
- mouth width
- forehead

### Color controls

- brightness
- contrast
- RGB
- hue
- saturation
- white balance
- grayscale

Built-in controls are useful when the application needs a slider, intensity value, or custom preset.

## External `.nosmai` packages

A `.nosmai` file is a protected package containing a complete visual effect. The package includes a manifest that tells the SDK what type of effect it contains.

Use:

```text
applyEffect(path)
```

for every supported package type.

### Package types

| Manifest type | What it is for | Replacement behavior |
| --- | --- | --- |
| `filter` | LUT, color grade, and full-frame filter | Replaces the previous regular filter |
| `effect` | AR mask, sticker, particle effect, or 3D effect | Replaces the previous AR or beauty effect |
| `beauty_effect` | Packaged makeup or face-mesh beauty effect | Replaces the previous AR or beauty effect |
| `background` | Packaged background replacement | Replaces the previous background package |

`effect` and `beauty_effect` share the same active AR position. Applying one replaces the other.

The application does not choose this position manually. The SDK reads the internal package type.

## Active features and coexistence

Some features can remain active together, while others replace the previous feature in the same category.

A common external-package combination is:

```text
regular filter
+ AR or beauty effect
+ background
```

Important rules:

- one regular `.nosmai` filter is active at a time
- one `.nosmai` `effect` or `beauty_effect` is active at a time
- one `.nosmai` background package is active at a time
- applying a new package replaces the current package in the same category

Built-in beauty, makeup, reshape, color, and hair controls are an alternative mode to an external `effect` or `beauty_effect`. Applying either AR-slot package clears those built-in controls. Applying one of those built-in controls while an AR-slot package is active clears the AR package. This rule is the same on Android, iOS, and Flutter.

A regular external `filter` can remain active with built-in controls. Built-in controls can also remain active with a manual background. External background packages and AR effects follow the background policy described in [Filters and effects](/docs/effects/filters-and-effects).

The SDK provides active-state methods and listeners so the application can keep selected buttons and filter sheets synchronized with the native state. Treat listener state as the final result after an asynchronous apply or clear operation.

## Local filter discovery

Production local filters use three files:

```text
filter_name.nosmai
filter_name_manifest.json
filter_name_preview.png
```

The external manifest and preview allow the application to show a filter list without opening and reading every protected package.

Use production local discovery for released applications.

## Debug filter discovery

Debug discovery scans loose `.nosmai` files that may not yet have an external manifest or preview image.

It is intended for:

- filter development
- internal testing
- quickly testing a new package

It is not the recommended production catalog method. Debug discovery may need additional file inspection and does not provide the same catalog quality as complete production entries.

## Cloud filters

Cloud filters separate the catalog from the application release.

The cloud flow is:

```text
Request filter metadata
    |
    v
Show names and previews
    |
    v
Download the selected package
    |
    v
Use its local path with applyEffect
```

Cloud filter requests can include:

- page
- limit
- filter type
- catalog version
- whether all pages should be fetched

Version 2 is the default catalog version in the current SDK, so an application can request the standard list without supplying a version every time.

## Built-in features compared with `.nosmai` packages

| Requirement | Recommended choice |
| --- | --- |
| A slider that changes skin smoothing continuously | Built-in method |
| A reusable lipstick style with direct intensity control | Built-in makeup method |
| A complete themed AR experience | `.nosmai` effect |
| A branded package containing several authored visual assets | `.nosmai` effect |
| A downloadable seasonal filter | Cloud `.nosmai` package |
| A simple color adjustment controlled by the app | Built-in color method |
| A packaged LUT distributed through the catalog | `.nosmai` filter |

The application does not need to convert every built-in setting into a `.nosmai` package. If a product combines the two systems, verify the selected state after every external package change and design the interface around the coexistence supported by the target platforms.

## Face detection

Face detection and tracking are enabled internally when an active feature needs them.

Face-aware features include:

- makeup
- face shaping
- eye color
- face masks
- face-tracked 2D and 3D effects
- some background and beauty effects

The application normally does not need to:

- load a face model
- run a separate face detector
- send landmarks to each built-in makeup method
- choose a detection interval

The SDK owns these internal details so Android, iOS, and Flutter behavior can remain consistent.

## Background processing

Background effects separate a person from the camera background.

Supported background experiences include:

- blur
- solid color
- still image
- looping video
- packaged `.nosmai` background

Background processing is more expensive than a simple color filter because it analyzes the image before composing the final frame. Test background effects on the lowest-end device supported by the application.

## Photo capture

A captured photo should contain the same active effects visible in the preview.

Before capture:

- wait for the preview to be ready
- wait for the selected effect to finish applying
- avoid switching the camera at the same moment
- request photo library permission only if the application saves to the library

Capture and saving are separate actions. The SDK can produce the image, while the application decides whether to save, upload, edit, or discard it.

## Video recording

Recording receives processed frames, so active effects appear in the output video.

Recording may require:

- microphone permission for audio
- an output location
- enough available storage
- a supported video size
- stopping the recording before leaving the camera screen

Applying a very expensive effect while recording can increase device load. Test effect switching during recording if the user interface allows it.

## Live streaming

Live streaming uses the processed output rather than the unmodified camera frame.

The general flow is:

```text
Camera
    |
    v
Nosmai effects
    |
    v
Processed frame
    |
    v
Streaming provider
```

Nosmai does not replace channel authentication, tokens, user roles, or the streaming provider's session management. It supplies the processed visual frame that the provider publishes.

Use the guide that matches the application:

- [Native live streaming](../guide/native-live-streaming.md) for Android and
  iOS applications.
- [Flutter live streaming with Agora](../guide/flutter-live-streaming-agora.md)
  for Flutter applications using the Nosmai Agora bridge.

## Performance and feature cost

Not every visual feature has the same cost.

| Feature | Typical relative cost |
| --- | --- |
| Brightness, contrast, or LUT | Low |
| Simple full-frame overlay | Low to medium |
| Face-tracked makeup | Medium |
| Face shaping | Medium |
| Complex AR effect | Medium to high |
| Background segmentation | High |
| Recording with effects | Additional output cost |
| Live streaming with effects | Additional output and encoding cost |

The exact result depends on the device and effect design.

For a stable experience:

- start with 30 FPS
- use a reasonable camera resolution
- avoid unnecessary simultaneous outputs
- do not apply the same effect repeatedly
- prevent rapid repeated camera switching
- wait for one filter change to finish before starting another
- test recording and streaming separately and together
- test thermal behavior during a long session

## State ownership

Use one application-level owner for SDK initialization and one screen-level owner for the active camera view.

A useful responsibility split is:

| Owner | Responsibility |
| --- | --- |
| Application service | License initialization and global SDK access |
| Camera screen | Preview, camera controls, and screen lifecycle |
| Filter controller | Filter lists, selection, download, apply, and remove |
| Recording controller | Start, stop, elapsed time, and output handling |
| Streaming controller | Channel connection, token, role, and publishing |

This prevents multiple screens from trying to initialize, stop, or clean up the same SDK resources at the same time.

## Error boundaries

Handle errors at the operation that can fail:

- initialization can fail because of identity, license, or connectivity
- camera start can fail because of permission or hardware access
- effect apply can fail because of a missing or invalid package
- cloud listing can fail because of connectivity
- download can fail because of storage or network issues
- recording can fail because of permissions, storage, or encoder setup
- streaming can fail because of provider authentication or channel setup

Do not represent every failure as a camera failure. Clear error categories make support and retry behavior easier.

## Recommended integration order

Build the application in this order:

1. initialize the SDK
2. display a stable camera preview
3. implement pause, resume, stop, and navigation
4. apply one built-in effect
5. apply one local `.nosmai` package
6. add filter listing and active-state UI
7. add photo capture
8. add recording
9. add cloud filters
10. add background effects
11. add live streaming
12. run long-session and low-end-device tests

This order makes it easier to identify whether an issue belongs to camera setup, effect application, output, or application lifecycle.

Continue with [Filters and effects](/docs/effects/filters-and-effects) to
understand package types, listing, application, active state, removal, and
cloud downloads.

---

# Filters and effects
Source: https://docs.nosmai.com/docs/effects/filters-and-effects/

## Overview

Nosmai supports visual content through protected `.nosmai` packages. Every
package declares one of four types:

- `filter`
- `effect`
- `beauty_effect`
- `background`

The type tells the SDK what the package does and which active position it
occupies.

The application does not need to select that position manually. It passes the
package path to `applyEffect(path)`. Nosmai reads the package type and applies
it correctly.

The simplest rule is:

```text
List a package
    |
    v
Obtain its local path
    |
    v
Call applyEffect(path)
    |
    v
Read the active state
```

## The four package types

### Filter

A `filter` changes the overall appearance of the camera image.

Common examples:

- color grading
- LUT filters
- brightness or tone styles stored in a package
- black-and-white looks
- cinematic color styles
- full-frame shader effects
- screen overlays that do not need face tracking

A filter normally affects the full camera frame. It usually does not need to
follow a face.

Only one external `filter` package is active at a time. Applying another
`filter` replaces the previous filter.

A filter can normally remain active with:

- one `effect` or `beauty_effect`
- one `background`
- built-in beauty and makeup

Example:

```text
Warm color filter
    +
Face glasses effect
    +
Background replacement
```

These can be visible together because they have different responsibilities.

### Effect

An `effect` adds an interactive or tracked visual experience.

Common examples:

- face masks
- glasses
- hats
- 2D face overlays
- 3D models
- animated face effects
- face mesh effects
- screen distortion
- animated particles
- effects that react to face movement

An effect may use face tracking, animation, a 3D model, a full-frame shader, or
background processing. Its exact requirements are stored inside the package.

Only one external face effect is active at a time.

`effect` and `beauty_effect` share the same active position. Therefore:

- applying an `effect` replaces the current `beauty_effect`
- applying a `beauty_effect` replaces the current `effect`

This replacement is intentional. It prevents two face packages from competing
for the same tracked face resources.

### Beauty effect

A `beauty_effect` is a packaged beauty or makeup look.

Common examples:

- a complete makeup preset
- packaged lipstick and blusher
- packaged eye makeup
- a face-mesh beauty look
- a beauty look containing textures and authored strengths
- a coordinated makeup style distributed as one `.nosmai` file

A `beauty_effect` is different from a normal `effect` because its catalog
category is beauty. Applications can display it in a dedicated Beauty section.

It is also different from built-in beauty.

| Packaged beauty effect | Built-in beauty |
| --- | --- |
| Stored in a `.nosmai` package | Compiled into the SDK |
| Listed with local or cloud packages | Controlled through direct methods |
| Applied with `applyEffect(path)` | Applied with methods such as `applySkinSmoothing` or `applyLipstick` |
| Has package metadata and preview | Uses application-defined controls |
| Occupies the same position as an `effect` | Managed through built-in beauty controls |

The package type must be exactly:

```text
beauty_effect
```

Do not use `beauty`, `beauty-effect`, or a display label inside the package
manifest.

Only one `effect` or `beauty_effect` package can be active at the same time.

### Background

A `background` package changes or replaces the area behind the subject.

Common examples:

- background blur
- an authored image background
- an animated background
- a virtual room
- a background shader
- a packaged background replacement

A `background` package is different from a manual background configured with a
direct API.

| Background package | Manual background |
| --- | --- |
| Distributed as a `.nosmai` file | Created by the application |
| Listed as type `background` | Not part of the package catalog |
| Applied with `applyEffect(path)` | Applied with background configuration methods |
| Contains authored package settings | Uses color, image, video, or blur values supplied by the app |

Only one external `background` package is active at a time. Applying another
background package replaces the previous one.

## Type comparison

| Type | Main purpose | Usually tracks a face | Replacement rule |
| --- | --- | --- | --- |
| `filter` | Full-frame color or visual style | No | Replaces the previous `filter` |
| `effect` | AR, tracked, animated, or interactive effect | Often | Replaces the current `effect` or `beauty_effect` |
| `beauty_effect` | Packaged makeup or beauty look | Yes | Replaces the current `effect` or `beauty_effect` |
| `background` | Packaged background visual | No, but may identify the subject | Replaces the previous `background` |

## What can be active together

The package type controls replacement.

| Active combination | Result |
| --- | --- |
| `filter` and `effect` | Both can remain active |
| `filter` and `beauty_effect` | Both can remain active |
| `filter` and `background` | Both can remain active |
| `beauty_effect` and `background` | Both can remain active |
| `effect` and `beauty_effect` | The latest package replaces the previous one |
| built-in beauty, makeup, reshape, color, or hair and `effect` | The latest mode clears the other mode |
| built-in beauty, makeup, reshape, color, or hair and `beauty_effect` | The latest mode clears the other mode |
| built-in controls and a regular external `filter` | Both can remain active |
| two `filter` packages | The latest filter replaces the previous filter |
| two `background` packages | The latest background replaces the previous background |

Some effects include their own background or explicitly control background
behavior. In that case, the effect package decides whether another background
can remain active.

The application should not manually remove the previous package before applying
another package of the same type. Applying the new package already performs the
replacement.

When a user switches between an AR-slot package and built-in beauty, update the interface from the apply completion and active-state listener. The SDK performs the required clear automatically in both directions.

## Package type and source are different

Do not confuse package type with package source.

Package type describes what the package does:

```text
filter
effect
beauty_effect
background
```

Package source describes where the application obtained it:

```text
local
cloud
debug
```

For example:

- a local `filter`
- a cloud `filter`
- a local `beauty_effect`
- a cloud `beauty_effect`

Local and cloud copies use the same apply method after a local file path is
available.

## Built-in features are not `.nosmai` packages

Built-in features use direct SDK methods.

Examples:

- skin smoothing
- skin whitening
- lipstick
- eyeshadow
- blusher
- eyelash
- eyebrow
- face slimming
- eye enlargement
- nose slimming
- brightness
- contrast
- sharpening
- white balance

These features are not returned by the local or cloud `.nosmai` package lists.

For example, Flutter built-in beauty is applied directly:

```dart
await NosmaiFlutter.instance.applySkinSmoothing(0.35);
await NosmaiFlutter.instance.setFaceSlimLevel(0.15);
```

A packaged beauty preset is applied by path:

```dart
await NosmaiFlutter.instance.applyEffect(
  beautyPackage.path,
);
```

## Package manifest type

Every `.nosmai` package contains an internal manifest. The canonical field is
`type`.

Example:

```json
{
  "type": "beauty_effect",
  "schemaVersion": "2.0"
}
```

Use one exact value:

```text
filter
effect
beauty_effect
background
```

The filename does not decide the type. A file named
`cinematic_beauty.nosmai` is not a beauty effect unless its internal manifest
declares `beauty_effect`.

The application should not open or edit the package. Nosmai validates
and reads it when needed.

## Local production packages

A production local entry contains:

```text
<name>.nosmai
<name>_manifest.json
<name>_preview.png
```

The external manifest supplies catalog information that can be read without
opening the protected package:

```json
{
  "id": "soft_glam",
  "displayName": "Soft Glam",
  "description": "A balanced makeup look",
  "type": "beauty_effect",
  "version": "1.0",
  "author": "Nosmai"
}
```

The external type should match the internal package type.

Recommended Flutter layout:

```text
assets/
  nosmai_filters/
    soft_glam/
      soft_glam.nosmai
      soft_glam_manifest.json
      soft_glam_preview.png
```

Recommended Android layout:

```text
app/src/main/assets/
  Nosmai_Filters/
    soft_glam/
      soft_glam.nosmai
      soft_glam_manifest.json
      soft_glam_preview.png
```

Recommended iOS layout:

```text
nosmai_filters/
  soft_glam/
    soft_glam.nosmai
    soft_glam_manifest.json
    soft_glam_preview.png
```

An incomplete production entry may be skipped. Keep the package, external
manifest, and preview together.

## List local packages

### Flutter

Get all local packages grouped by type:

```dart
final groups =
    await NosmaiFlutter.instance.getAllLocalFilters();

final filters = groups['filter'] ?? const [];
final effects = groups['effect'] ?? const [];
final beautyEffects =
    groups['beauty_effect'] ?? const [];
final backgrounds =
    groups['background'] ?? const [];
```

Get one local type:

```dart
final filters =
    await NosmaiFlutter.instance.getLocalFilters();

final effects =
    await NosmaiFlutter.instance.getLocalEffects();

final beautyEffects =
    await NosmaiFlutter.instance.getLocalBeautyEffects();

final backgrounds =
    await NosmaiFlutter.instance.getLocalBackgrounds();
```

`getLocalFilters()` returns the production local catalog. Use the typed methods
when the interface shows separate tabs.

### Android

Get every local package:

```java
List<NosmaiFilterInfo> packages =
        NosmaiEffects.getFilters();
```

Get one type:

```java
List<NosmaiFilterInfo> effects =
        NosmaiEffects.getFilters(
                NosmaiFilterInfo.Type.EFFECT
        );
```

Android types are:

```text
FILTER
EFFECT
BEAUTY_EFFECT
BACKGROUND
```

### iOS

Get every local package:

```objc
NosmaiSDK *sdk = [NosmaiSDK sharedInstance];
NSArray<NosmaiFilterInfo *> *packages =
    [sdk getFilters];
```

Get one type:

```objc
NSArray<NosmaiFilterInfo *> *beautyEffects =
    [sdk getFiltersOfType:
        NosmaiFilterTypeBeautyEffect];
```

iOS types are:

```text
NosmaiFilterTypeFilter
NosmaiFilterTypeEffect
NosmaiFilterTypeBeautyEffect
NosmaiFilterTypeBackground
```

## Debug package discovery

Debug discovery is for development and testing.

It can find loose `.nosmai` files that do not yet have a complete production
manifest and preview.

Do not use the debug method as the production catalog.

### Flutter

Get every debug package:

```dart
final packages =
    await NosmaiFlutter.instance.getDebugFilters();
```

Get one debug type:

```dart
final effects =
    await NosmaiFlutter.instance.getDebugFilters(
  type: NosmaiLocalFilterType.effect,
);
```

Available Flutter debug types are:

```text
NosmaiLocalFilterType.filter
NosmaiLocalFilterType.effect
NosmaiLocalFilterType.beautyEffect
NosmaiLocalFilterType.background
```

### Android

```java
NosmaiEffects.getDebugFilters(
        NosmaiFilterInfo.Type.EFFECT,
        (filters, error) -> {
            if (error != null) {
                return;
            }

            // Update the development filter list.
        }
);
```

### iOS

```objc
[sdk getDebugFiltersOfType:
          NosmaiFilterTypeEffect
                  completion:^(
    NSArray<NosmaiFilterInfo *> *filters,
    NSError *error
) {
    if (error != nil) {
        return;
    }

    // Update the development filter list.
}];
```

Debug discovery may take longer than production listing because missing catalog
metadata can require the SDK to inspect the package.

## Apply any package

Use one apply method for all four package types.

### Flutter

```dart
final success =
    await NosmaiFlutter.instance.applyEffect(
  selected.path,
);

if (!success) {
  // Keep the previous selected state.
}
```

### Android

```java
NosmaiEffects.applyEffect(
        selected,
        new NosmaiEffects.EffectCallback() {
            @Override
            public void onSuccess() {
                // The package is active.
            }

            @Override
            public void onError(String message) {
                // Keep the previous selected state.
            }
        }
);
```

A path can also be applied:

```java
NosmaiEffects.applyEffect(
        selected.getPath(),
        callback
);
```

### iOS

```objc
[sdk applyEffectInfo:selected
          completion:^(
    BOOL success,
    NSError *error
) {
    if (!success) {
        // Keep the previous selected state.
        return;
    }

    // The package is active.
}];
```

The apply operation is asynchronous. Do not mark the package selected before
the success result.

## Why the method is named applyEffect

`applyEffect` is the unified package method. Its name does not mean that it only
accepts a package whose type is `effect`.

It accepts:

- `filter`
- `effect`
- `beauty_effect`
- `background`

Flutter also keeps `applyFilter(path)` as a compatibility alias. New code
should use `applyEffect(path)` for every `.nosmai` package.

## Active selection state

The application should use SDK state to keep selected cells, check marks, and
active labels correct.

A reliable screen flow is:

1. subscribe to active-state changes
2. read the current state once
3. build selected UI from that state
4. update the UI after future state changes
5. cancel the subscription when the screen closes

Do not keep a separate list of assumed active packages. A new package can
replace an older package automatically.

### Flutter

Read the initial state:

```dart
final initial =
    await NosmaiFlutter.instance.getActiveEffects();
```

Listen for changes:

```dart
late final StreamSubscription<NosmaiActiveEffects>
    activeEffectsSubscription;

void observeActiveEffects() {
  activeEffectsSubscription =
      NosmaiFlutter.instance
          .onActiveEffectsChanged
          .listen((state) {
    final hasFilter = state.hasFilter;
    final hasEffect = state.hasEffect;
    final hasBeautyEffect =
        state.hasBeautyEffectPackage;
    final hasBackground =
        state.hasBackgroundPackage;

    final filterPath =
        state.activeFilterPath;
    final effectPath =
        state.activeEffectPath;
    final backgroundPath =
        state.activeBackgroundPath;

    // Update selected cells from these values.
  });
}
```

Cancel when the owner closes:

```dart
await activeEffectsSubscription.cancel();
```

The active face package appears in `activeEffectPath`. Its metadata tells the
application whether it is an `effect` or a `beauty_effect`.

Typed convenience methods are also available:

```dart
final activeFilter =
    await NosmaiFlutter.instance.getActiveFilterInfo();

final activeEffect =
    await NosmaiFlutter.instance.getActiveEffectInfo();

final activeBeauty =
    await NosmaiFlutter.instance
        .getActiveBeautyEffectInfo();
```

### Android

Keep the listener instance so it can be removed:

```java
private final
NosmaiEffectsEngine.PipelineStateListener
        activeStateListener =
        state -> updateSelectedItems(state);

@Override
protected void onStart() {
    super.onStart();

    NosmaiEffects.addPipelineStateListener(
            activeStateListener
    );

    updateSelectedItems(
            NosmaiEffects.getCurrentPipelineState()
    );
}

@Override
protected void onStop() {
    NosmaiEffects.removePipelineStateListener(
            activeStateListener
    );

    super.onStop();
}
```

Do not register an anonymous listener that cannot be removed later.

### iOS

Use the effects delegate:

```objc
@interface CameraViewController ()
    <NosmaiEffectsDelegate>
@end

- (void)viewDidLoad {
    [super viewDidLoad];
    [NosmaiCore shared].effects.delegate = self;
}

- (void)nosmaiEffectsDidChangePipelineState:
    (NosmaiPipelineState *)state {
    [self updateSelectedItems:state];
}

- (void)dealloc {
    if ([NosmaiCore shared].effects.delegate == self) {
        [NosmaiCore shared].effects.delegate = nil;
    }
}
```

Read the current state when the screen first opens:

```objc
NosmaiPipelineState *state =
    [[NosmaiSDK sharedInstance]
        currentPipelineState];
```

## Remove packages

Remove the position that the user intends to clear.

### Flutter

```dart
await NosmaiFlutter.instance.clearFilter();
```

This clears the external `filter`.

```dart
await NosmaiFlutter.instance.clearAREffect();
```

This clears the active `effect` or `beauty_effect`.

Remove a selected item by its type:

```dart
await NosmaiFlutter.instance.removeEffect(
  selected,
);
```

Remove all external packages:

```dart
await NosmaiFlutter.instance.removeAllFilters();
```

Clear external packages and other active visual state:

```dart
await NosmaiFlutter.instance.clearAll();
```

### Android

```java
NosmaiEffects.clearFilter();
NosmaiEffects.clearAREffect();
NosmaiEffects.removeEffect(selected);
NosmaiEffects.removeEffect();
```

### iOS

```objc
[sdk clearFilter];
[sdk clearAREffect];
[sdk removeEffectInfo:selected];
[sdk removeAllFilters];
```

Use typed removal when the interface has one common Remove button for mixed
package types.

## Cloud packages

Cloud listing returns package metadata and preview information. The protected
package is downloaded when the user selects it.

The flow is:

```text
Get cloud list
    |
    v
Show metadata and preview
    |
    v
Download selected package
    |
    v
Receive local path
    |
    v
Call applyEffect(localPath)
```

### Cloud type values

Cloud request values differ slightly from internal package types.

| Package type | Flutter cloud enum | API value |
| --- | --- | --- |
| `filter` | `NosmaiCloudFilterType.filter` | `filter` |
| `effect` | `NosmaiCloudFilterType.effects` | `effects` |
| `beauty_effect` | `NosmaiCloudFilterType.beautyEffect` | `beauty_effect` |
| `background` | `NosmaiCloudFilterType.background` | `bg` |

Use the enum instead of typing these strings in Flutter.

### Flutter cloud listing

Get all available cloud packages:

```dart
final packages =
    await NosmaiFlutter.instance.getCloudFilters();
```

Get the first page of beauty effects:

```dart
final beautyEffects =
    await NosmaiFlutter.instance.getCloudFilters(
  filterType:
      NosmaiCloudFilterType.beautyEffect,
  page: 1,
  limit: 20,
);
```

Cloud compatibility version 2 is selected automatically. The application does
not need to pass a version for the normal request.

Read pagination:

```dart
final pagination =
    NosmaiFlutter.instance.lastPaginationInfo;

if (pagination?.hasNextPage == true) {
  final nextPage =
      await NosmaiFlutter.instance.getCloudFilters(
    filterType:
        NosmaiCloudFilterType.beautyEffect,
    page: pagination!.currentPage + 1,
    limit: 20,
  );
}
```

### Download and apply in Flutter

```dart
final result =
    await NosmaiFlutter.instance
        .downloadCloudFilter(selected.cloudIdentifier);

if (result['success'] == true) {
  final localPath =
      result['localPath'] as String;

  final applied =
      await NosmaiFlutter.instance
          .applyEffect(localPath);
}
```

Do not pass a cloud preview URL to `applyEffect`. Apply the downloaded local
package path.

## Recommended filter interface

Use separate tabs or sections:

```text
Filters
Effects
Beauty
Backgrounds
```

Map them as:

| Interface label | Package type |
| --- | --- |
| Filters | `filter` |
| Effects | `effect` |
| Beauty | `beauty_effect` |
| Backgrounds | `background` |

For each item, show:

- preview image
- display name
- download state for cloud items
- download progress when needed
- applying state
- selected state from the SDK
- a retry action after a temporary failure

Do not show internal paths, package headers, or package internals to the user.

## Apply-state handling

Use these interface states:

```text
idle
downloading
applying
selected
failed
```

Recommended behavior:

1. Disable repeated taps while one cloud download is active.
2. Show download progress.
3. Call `applyEffect` after the local path is available.
4. Mark the package selected only after success.
5. Confirm the final selected state from the SDK.
6. Restore the previous selected item if apply fails.

Do not call clear and apply together to replace a package. Apply the new
package and let its type perform the correct replacement.

## Face and background requirements

The application does not need to manually start face tracking for a package.

Nosmai reads the package requirements when it is applied. Depending on the
package, the SDK can enable:

- face tracking
- face landmarks
- animation timing
- background subject detection
- background coexistence behavior

These behaviors depend on correct package metadata created by the filter
author.

## Performance guidance

For a smooth camera:

- keep package textures reasonably sized
- avoid repeated apply calls for the already selected item
- wait for an apply result before starting another expensive operation
- use production listing instead of debug discovery in a released app
- download cloud files before applying them
- avoid showing several animated previews at full resolution
- test face effects with rapid head movement
- test background packages on the lowest supported device
- test filters during recording and streaming
- test a long session for heat and memory use

A color `filter` is usually cheaper than a tracked `effect`,
`beauty_effect`, or subject-aware `background`.

## Error handling

Package apply can fail because:

- Nosmai is not initialized
- the path is empty
- the local file does not exist
- the package is incomplete
- the package cannot be opened or validated
- the internal manifest is invalid
- the package type is invalid
- a required visual resource cannot be created
- the device does not support the required operation
- the license does not include the required feature

Keep technical details in logs. Show a short message such as:

```text
This effect could not be applied. Please try again.
```

Do not change selected UI before success.

## Summary

Use these rules:

1. `filter` changes the overall frame appearance.
2. `effect` provides AR, tracking, animation, or interactive visuals.
3. `beauty_effect` is a packaged beauty or makeup look.
4. `background` controls the area behind the subject.
5. `effect` and `beauty_effect` replace each other.
6. Different package types can normally remain active together.
7. Use `applyEffect(path)` for every `.nosmai` package.
8. Use production methods for released applications.
9. Use debug discovery only during development.
10. Download cloud packages before applying their local path.
11. Build selected UI from the SDK active state.
12. Remove listeners when their screen closes.

---

# Off-screen rendering
Source: https://docs.nosmai.com/docs/effects/off-screen-rendering/

# Off-screen rendering

Off-screen rendering lets Nosmai process a frame and produce a filtered result
without requiring that result to be drawn directly in the standard camera
preview.

This is useful when your application needs to:

- Process frames supplied by another camera or video source.
- Send processed frames to a live-streaming service.
- Record or encode processed frames using a custom media system.
- Run effects in a native video-processing component without a visible preview.
- Display the preview while also sending the same processed result elsewhere.

Off-screen rendering does not mean that Android or iOS can continue unrestricted
camera processing after the application enters the background. Operating-system
camera, microphone, and background-execution rules still apply.

## Choose the correct flow

Nosmai supports more than one off-screen use case. Choose the flow that matches
who owns the camera input.

| Requirement | Input owner | Recommended flow |
| --- | --- | --- |
| Process external video frames without the Nosmai camera | Your application | Android external I420 or iOS external `CVPixelBuffer` processing |
| Show the Nosmai camera preview and send processed output elsewhere | Nosmai | Android `DUAL_OUTPUT` or iOS live frame output |
| Send the Nosmai camera output without showing a local preview | Nosmai | Android `STREAMING_ONLY` |
| Record the normal Nosmai camera result | Nosmai | Use the standard recording APIs |
| Stream from Flutter through Agora | Nosmai and the bridge | Use the Nosmai Agora bridge |

Do not feed frames from the Nosmai camera back into the external-input API. That
would process the same frame twice, increase latency, and may apply an effect
twice.

## Frame ownership

Real-time rendering works best when each input frame has one clear owner.

Follow these rules:

1. Keep at most one expensive off-screen operation in progress for a live
   source.
2. When processing falls behind, discard an old waiting frame and keep the
   newest frame.
3. Do not build an unlimited queue of camera frames.
4. Do not reuse or release an input buffer until Nosmai has finished reading it.
5. Consume a returned native output buffer before submitting another frame,
   unless your integration explicitly copies or retains that output.
6. Use the input timestamp when sending the processed result to an encoder or
   streaming service.

These rules keep the preview responsive and prevent old results from appearing
after the user's face has already moved.

## Android external frames

Android accepts planar I420 input for direct external-frame processing.

### Requirements

- Initialize `NosmaiSDK` before initializing external processing.
- Supply Y, U, and V as direct `ByteBuffer` objects.
- Use planar I420, not NV21, NV12, RGBA, or an Android `Bitmap`.
- Pass the actual stride for every plane.
- Use rotation `0`, `90`, `180`, or `270`.
- Keep the input dimensions stable during a processing session.
- Use `NosmaiSDK.ThreadMode.GL_THREAD` unless you manage the current GL context
  yourself.

`CALLER_THREAD` requires a compatible GL context to be current on the calling
thread. It is intended for advanced native integrations. `GL_THREAD` is the
safer default for normal applications.

### Initialize external processing

```java
int width = 1280;
int height = 720;

boolean ready = NosmaiSDK.initializeExternalFramePipeline(width, height);
if (!ready) {
    throw new IllegalStateException("Nosmai external processing is unavailable");
}

NosmaiSDK.setExternalFrameMode(true);
NosmaiSDK.setExternalFrameThreadMode(NosmaiSDK.ThreadMode.GL_THREAD);
```

Initialize once for the active dimensions. If the video source changes
resolution, stop external processing and initialize it again with the new
dimensions.

### Process and return a new I420 frame

```java
NosmaiSDK.ProcessedI420Frame output =
        NosmaiSDK.processExternalI420Sync(
                yBuffer,
                uBuffer,
                vBuffer,
                width,
                height,
                yStride,
                uStride,
                vStride,
                rotation,
                mirror
        );

if (output != null) {
    videoConsumer.onI420Frame(
            output.yBuffer,
            output.uBuffer,
            output.vBuffer,
            output.width,
            output.height,
            output.timestampNs
    );
}
```

This call is synchronous. The returned Y, U, and V buffers refer to native
output storage. Consume or copy them before processing the next frame.

### Process writable I420 planes in place

Use in-place processing when the downstream video system already owns writable
I420 planes and expects the result in those same buffers.

```java
boolean processed = NosmaiSDK.processExternalI420InPlace(
        yBuffer,
        uBuffer,
        vBuffer,
        width,
        height,
        yStride,
        uStride,
        vStride,
        rotation,
        mirror
);

if (processed) {
    videoConsumer.onI420Frame(
            yBuffer,
            uBuffer,
            vBuffer,
            width,
            height,
            timestampNs
    );
}
```

The planes must be writable and large enough for their supplied strides.
Nosmai blocks until the processed result has been copied back into the input
planes.

### Apply a `.nosmai` filter

Initialize external processing before applying the package:

```java
boolean applied = NosmaiSDK.applyEffect(filterPath);
```

Use a tested `.nosmai` package intended for this input mode. External I420
processing uses a focused single-filter flow, so validate every package and
required feature before shipping it in an external video integration.

### Stop external processing

Stop the source first so no new frame arrives during cleanup.

```java
NosmaiSDK.clearAllEffects();
NosmaiSDK.setFrameCallback(null);
NosmaiSDK.setExternalFrameMode(false);
```

If the application is closing the SDK completely, continue with the normal
Nosmai cleanup call used by your application.

## Android camera output

When Nosmai owns the camera, use a render mode instead of the external I420
input methods.

```java
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.DUAL_OUTPUT);
```

The available modes are:

| Mode | Local preview | External processed output |
| --- | --- | --- |
| `PREVIEW_ONLY` | Yes | No |
| `STREAMING_ONLY` | No | Yes |
| `DUAL_OUTPUT` | Yes | Yes |

A blank local view is expected in `STREAMING_ONLY`. Use `DUAL_OUTPUT` when the
user must see the camera while publishing or recording through another system.

When a portrait streaming consumer requires physically rotated output planes,
enable portrait off-screen output before registering the frame consumer:

```java
NosmaiSDK.setPortraitOffscreenOutput(true);
```

Disable it when the stream stops. Do not also rotate the same frame in the
encoder unless that encoder expects additional rotation.

### CPU frame callback

```java
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.DUAL_OUTPUT);
NosmaiSDK.setFrameCallback(frame -> {
    externalConsumer.onFrame(
            frame.pixelBuffer,
            frame.width,
            frame.height,
            frame.format,
            frame.timestampNs
    );
});
```

The CPU callback is compatible with systems that cannot consume a shared GPU
texture. It costs more because frame data must be transferred from GPU memory
to CPU memory.

Remove the callback when output is no longer needed:

```java
NosmaiSDK.setFrameCallback(null);
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.PREVIEW_ONLY);
```

### GPU texture output

For high-performance live streaming, prefer the Nosmai Agora bridge. It creates
the required shared EGL setup and handles processed textures without a
per-frame CPU image conversion.

Low-level Android integrations must:

1. Provide Agora's EGL share context through
   `NosmaiSDK.setAgoraShareContext(...)` before Nosmai creates its GL context.
2. Register `NosmaiSDK.setTextureFrameCallback(...)` only after the streaming
   consumer is ready.
3. Wait for the supplied fence before sampling a texture.
4. Always call `NosmaiSDK.releaseStreamSlot(texId)`, including error and dropped
   frame paths.
5. Clear the callback before leaving the channel or destroying either GL
   context.

Failure to release a stream slot can stop future texture delivery. Registering
the shared context too late can produce black remote frames.

See
[Flutter live streaming with Agora](../guide/flutter-live-streaming-agora.md)
for the supported Flutter integration.

### External Android surface

`NosmaiSDK.setRenderSurface(...)` is a compatibility fallback for drawing the
processed result into another Android `Surface`. It converts callback data
through CPU memory and a `Bitmap`, so it is not the preferred route for a
high-frame-rate camera experience.

Embed the normal `NosmaiPreviewView` when possible. If a fallback surface is
required, remove it with:

```java
NosmaiSDK.clearRenderSurface();
```

This releases the surface and restores `PREVIEW_ONLY`.

## iOS external frames

iOS accepts external `CVPixelBufferRef` or `CMSampleBufferRef` input.

### Requirements

- Initialize `NosmaiSDK` before initializing off-screen processing.
- Supply `kCVPixelFormatType_32BGRA` pixel buffers.
- Keep width and height stable during the active session.
- Keep an asynchronously submitted input buffer alive until completion.
- Treat output callbacks as background-thread callbacks.
- Retain an output pixel buffer if it must remain valid after the callback
  returns.

Convert camera formats such as bi-planar YUV to BGRA before calling the direct
off-screen API.

### Initialize and receive processed output

```objective-c
NosmaiSDK *sdk = [NosmaiSDK sharedInstance];

[sdk setCVPixelBufferCallback:^(CVPixelBufferRef output,
                                double timestamp) {
    CVPixelBufferRetain(output);

    dispatch_async(encoderQueue, ^{
        [videoConsumer consumePixelBuffer:output timestamp:timestamp];
        CVPixelBufferRelease(output);
    });
}];

BOOL ready = [sdk initializeOffscreenWithWidth:width height:height];
if (!ready) {
    NSLog(@"Nosmai off-screen processing is unavailable");
} else {
    [sdk setProcessingMode:NosmaiProcessingModeOffscreen];
}
```

`initializeOffscreenWithWidth:height:` prepares the required resources. Setting
`NosmaiProcessingModeOffscreen` explicitly also stops an already-running
internal camera before external frames begin.

### Process a `CVPixelBuffer`

```objective-c
BOOL accepted = [sdk processFrame:inputPixelBuffer mirror:NO];
if (!accepted) {
    // Drop this result and continue with the next live frame.
}
```

An accepted frame can still be skipped briefly while Nosmai replaces an active
filter. Treat the output callback as the source of completed processed frames.

### Process a `CMSampleBuffer`

```objective-c
BOOL accepted = [sdk processSampleBuffer:sampleBuffer mirror:NO];
```

Nosmai reads the image buffer from the sample buffer and processes it through
the same off-screen flow.

### Process asynchronously

Retain the input until the completion callback because the work continues
after the method returns.

```objective-c
CVPixelBufferRetain(inputPixelBuffer);

[sdk processFrameAsync:inputPixelBuffer
                 mirror:NO
             completion:^(BOOL success, NSError *error) {
    CVPixelBufferRelease(inputPixelBuffer);

    if (!success) {
        NSLog(@"Frame processing failed: %@", error.localizedDescription);
    }
}];
```

The completion callback reports whether processing succeeded. The processed
image still arrives through `setCVPixelBufferCallback:`.

### Read processing metrics

```objective-c
NSDictionary *metrics = [sdk getProcessingMetrics];

NSNumber *fps = metrics[@"currentFPS"];
NSNumber *averageTime = metrics[@"averageProcessingTime"];
NSNumber *processed = metrics[@"framesProcessed"];
NSNumber *dropped = metrics[@"framesDropped"];
```

The metrics also include `lastProcessingTime`. Use them during development to
identify frame backlog or an unsuitable input resolution.

### Return to the Nosmai camera

Stop the external source, clear its output callback, and switch back to live
mode:

```objective-c
[sdk setCVPixelBufferCallback:nil];
[sdk setProcessingMode:NosmaiProcessingModeLive];
```

Use `NosmaiProcessingModeHybrid` only when the application genuinely needs the
internal camera and external frame input at the same time. Running two active
sources increases resource use and requires careful ownership testing.

## iOS camera output

When Nosmai owns the camera and another native component needs the processed
result, use the high-level live frame callback:

```objective-c
[NosmaiCore shared].liveFrameStreamCallback =
    ^(CVPixelBufferRef pixelBuffer, double timestamp) {
        // Forward the processed frame to the native consumer.
    };
```

The callback runs on a background processing thread. Retain the pixel buffer if
the consumer uses it after the callback returns. Do not update UIKit directly
from this callback.

Stop delivery when the consumer is removed:

```objective-c
[NosmaiCore shared].liveFrameStreamCallback = nil;
```

Clearing an unused callback conserves processing and memory resources.

## Flutter support

The public Flutter API manages the Nosmai camera, native preview, photo capture,
video recording, filters, beauty, and lifecycle. It does not currently expose
a Dart method for submitting arbitrary I420 or `CVPixelBuffer` frames.

For Flutter:

- Use `NosmaiCameraPreview` for the standard processed camera preview.
- Use the normal capture and recording methods for media output.
- Use the Nosmai Agora bridge for processed live streaming.
- Use native Android or iOS integration when your application must supply its
  own external video frames.

Do not call private method-channel handlers from application code. Internal
native frame methods are reserved for maintained Nosmai integrations and may
change independently from the public Dart API.

## Rotation and mirroring

Rotation and mirroring describe how the input image should be interpreted
before effects are placed.

On Android:

- Pass the frame rotation as `0`, `90`, `180`, or `270`.
- Pass the real Y, U, and V strides.
- Set `mirror` to match the source image, not only the camera position.

On iOS:

- Convert the source to upright BGRA before submission when possible.
- Set `mirror:YES` only when the supplied pixels are already mirrored.

Do not mirror the frame once before Nosmai and again in the encoder or remote
renderer. Double mirroring produces reversed makeup and face effects.

Test front camera, back camera, portrait, landscape, and device rotation
separately.

## Performance guidance

### Prefer GPU texture output for live streaming

Shared texture output avoids full-frame GPU-to-CPU transfer. It is the preferred
Android route when the streaming SDK supports a shared EGL context.

Android external I420 processing and CPU frame callbacks include memory copies
or GPU readback. They are useful for compatibility, but they cost more than
shared texture delivery.

### Use a stable input size

Do not change frame dimensions from one frame to the next. Recreate external
processing when the source resolution changes.

Start with a practical live resolution such as 720p at 30 FPS. Increase it only
after testing face effects, beauty, recording, and streaming together on
mid-range devices.

### Keep the newest frame

If the next frame arrives while a previous frame is still processing, keep the
newest waiting frame and discard the older waiting frame. This is preferable to
showing a smooth but delayed result.

### Avoid unnecessary conversions

- On Android, keep external input and output in I420 when the video consumer
  supports I420.
- On iOS, keep direct off-screen input in BGRA.
- Do not convert every frame through `Bitmap`, `UIImage`, JPEG, or PNG.
- Do not copy a native output unless the receiving component needs longer
  ownership.

## Cleanup order

Use this order when stopping an off-screen session:

1. Stop the camera, decoder, or other frame producer.
2. Prevent new frames from entering Nosmai.
3. Clear the frame or texture callback.
4. Release any outstanding texture slot or retained pixel buffer.
5. Disable external processing or restore preview-only mode.
6. Clear effects if they are no longer required.
7. Release the SDK only when the application no longer needs it.

This order prevents a late frame from accessing a released surface, buffer, or
GL context.

## Troubleshooting

### Processing returns `null` or `false`

Check that:

- Nosmai initialization completed successfully.
- External processing was initialized for the current dimensions.
- Android buffers are direct I420 buffers with valid strides.
- iOS input is a BGRA `CVPixelBuffer`.
- The SDK is not being released on another thread.

### Output is rotated or mirrored

Verify the input's real pixel orientation. Do not use the visible UI orientation
as a substitute for the frame rotation. Confirm that mirroring is applied only
once.

### Output is delayed

Remove any unbounded queue. Allow only one frame to process and keep at most one
new waiting frame. Reduce input resolution if processing still takes longer
than the frame interval.

### Remote video is black

For Android texture output, register the shared EGL context before Nosmai
initialization, wait for the supplied fence, and release every stream slot.
For Flutter Agora integration, use the maintained bridge setup described in
[Flutter live streaming with Agora](../guide/flutter-live-streaming-agora.md).

### Memory increases during streaming

Confirm that every retained iOS pixel buffer is released and every Android
texture ID is returned through `releaseStreamSlot`. Also clear callbacks when
the stream stops.

## Test checklist

Before releasing an off-screen integration, test:

- Front and back camera input.
- Portrait and landscape orientation.
- Mirrored and non-mirrored input.
- No face, one face, and rapid face movement.
- Filter apply, replace, and remove.
- Repeated start and stop.
- App pause and resume.
- Camera or video-source resolution changes.
- Recording while effects are active.
- Live streaming with local and remote video.
- Network interruption during streaming.
- At least one mid-range Android device and one older supported iPhone.
- Stable memory and frame rate during a long session.

## Next steps

- Read [Core concepts](concepts.md) for the complete processing model.
- Read [Filters and effects](filters-and-effects.md) for package types and
  replacement rules.
- Read
  [Flutter live streaming with Agora](../guide/flutter-live-streaming-agora.md)
  for Flutter live streaming.
- Read [Native live streaming](../guide/native-live-streaming.md) when Nosmai
  owns the camera in an Android or iOS application.
- Read [Errors and troubleshooting](../reference/errors.md) for common runtime
  failures.

---

# Beauty and makeup
Source: https://docs.nosmai.com/docs/effects/beauty-and-makeup/

## Overview

Nosmai provides built-in beauty controls that can be adjusted while the camera
preview is running. These controls are separate from downloadable `.nosmai`
packages and are intended for interactive controls such as sliders, color
selectors, and style lists.

The built-in beauty features are divided into four groups:

| Group | Features |
| --- | --- |
| Skin and detail | Skin smoothing, skin whitening, sharpening, and teeth whitening |
| Makeup | Lipstick, eyeshadow, blusher, eyelashes, and eyebrows |
| Face shaping | Face slim, eye size, nose slim, and other supported shape controls |
| Eye color | Iris color and color intensity |

Built-in beauty can be combined with a regular `filter` package. A packaged
`beauty_effect` is different: it is a complete authored beauty look stored in a
`.nosmai` file. See [Filters and effects](/docs/effects/filters-and-effects) for
the package types and replacement rules.

## Before applying beauty

Complete these steps first:

1. Initialize the SDK with a valid license key.
2. Show the Nosmai camera preview.
3. Wait until the preview is ready.
4. Apply the required beauty values.

Do not apply a saved beauty preset before initialization has completed. The
native SDK can queue some operations, but waiting for a ready preview gives the
application predictable state and clearer error handling.

Beauty effects that follow facial regions become visible when a face is
detected. If the face leaves the camera view, the SDK hides those visual layers.
The selected settings remain available and become visible again after the face
is detected.

## Strength values

Most beauty and makeup controls use a value from `0.0` to `1.0`:

- `0.0` means disabled or invisible.
- `0.5` means medium strength.
- `1.0` means maximum strength.

Start with low values. Maximum strength is useful for testing, but it often
looks artificial in a production camera.

Suggested starting values:

| Control | Suggested starting value |
| --- | --- |
| Skin smoothing | `0.25` to `0.40` |
| Skin whitening | `0.05` to `0.15` |
| Sharpening | `0.10` to `0.20` |
| Teeth whitening | `0.15` to `0.30` |
| Lipstick | `0.45` to `0.70` |
| Eyeshadow | `0.20` to `0.40` |
| Blusher | `0.15` to `0.35` |
| Eyelashes | `0.30` to `0.55` |
| Eyebrows | `0.20` to `0.40` |
| Face slim | `0.10` to `0.25` |
| Eye size | `0.05` to `0.15` |
| Nose slim | `0.05` to `0.15` |

These are starting points, not required values. Test the final preset on
different face shapes, skin tones, lighting conditions, camera distances, and
device classes.

## Skin and detail controls

### Flutter

Apply the controls after `NosmaiCameraPreview` reports that it is ready:

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.applySkinSmoothing(0.35);
await nosmai.applySkinWhitening(0.10);
await nosmai.applySharpening(0.15);
await nosmai.applyTeethWhitening(0.20);
```

Remove the built-in skin and color controls:

```dart
await nosmai.removeBuiltInFilters();
```

### Android

Use `NosmaiBeauty`:

```java
import com.nosmai.effect.api.NosmaiBeauty;

NosmaiBeauty.applySkinSmoothing(0.35f);
NosmaiBeauty.applySkinWhitening(0.10f);
NosmaiBeauty.applySharpen(0.15f);
NosmaiBeauty.applyTeethWhitening(0.20f);
```

Remove the slider-based beauty controls:

```java
NosmaiBeauty.removeAllBeautyFilters();
```

To clear both slider-based beauty and built-in makeup, use:

```java
NosmaiBeauty.clearAllBeautyFilters();
```

### iOS

Use the effects engine owned by `NosmaiCore`:

```objc
NosmaiEffectsEngine *effects = [NosmaiCore shared].effects;

[effects applySkinSmoothing:0.35f];
[effects applySkinWhitening:0.10f];
[effects applySharpening:0.15f];
[effects applyTeethWhitening:0.20f];
```

Remove the built-in skin and color controls:

```objc
[[NosmaiCore shared].effects removeBuiltInFilters];
```

## Makeup styles

Nosmai provides three styles for each makeup feature.

| Feature | Style 0 | Style 1 | Style 2 |
| --- | --- | --- | --- |
| Lipstick | Classic | Matte | Natural |
| Eyeshadow | Smokey | Shimmer | Natural |
| Blusher | Round | Contour | Natural |
| Eyelashes | Natural | Dramatic | Wispy |
| Eyebrows | Natural | Bold | Arched |

Applying one makeup feature does not remove the others. Lipstick, eyeshadow,
blusher, eyelashes, and eyebrows can remain active together.

### Flutter makeup

Flutter exposes named style enums and an intensity value:

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.applyLipstick(
  style: NosmaiLipstickStyle.matte,
  intensity: 0.60,
);

await nosmai.applyEyeshadow(
  style: NosmaiEyeshadowStyle.natural,
  intensity: 0.30,
);

await nosmai.applyBlusher(
  style: NosmaiBlusherStyle.natural,
  intensity: 0.25,
);

await nosmai.applyEyelash(
  style: NosmaiEyelashStyle.natural,
  intensity: 0.40,
);

await nosmai.applyEyebrow(
  style: NosmaiEyebrowStyle.natural,
  intensity: 0.30,
);
```

Available Flutter style values:

```dart
NosmaiLipstickStyle.classic
NosmaiLipstickStyle.matte
NosmaiLipstickStyle.natural

NosmaiEyeshadowStyle.smokey
NosmaiEyeshadowStyle.shimmer
NosmaiEyeshadowStyle.natural

NosmaiBlusherStyle.round
NosmaiBlusherStyle.contour
NosmaiBlusherStyle.natural

NosmaiEyelashStyle.natural
NosmaiEyelashStyle.dramatic
NosmaiEyelashStyle.wispy

NosmaiEyebrowStyle.natural
NosmaiEyebrowStyle.bold
NosmaiEyebrowStyle.arched
```

Change an active layer without loading a new style:

```dart
await nosmai.setLipstickIntensity(0.45);
await nosmai.setEyeshadowIntensity(0.25);
await nosmai.setBlusherIntensity(0.20);
await nosmai.setEyelashIntensity(0.35);
await nosmai.setEyebrowIntensity(0.25);
```

Check whether a layer is active:

```dart
final lipstickActive = await nosmai.hasLipstick();
final eyeshadowActive = await nosmai.hasEyeshadow();
final blusherActive = await nosmai.hasBlusher();
final eyelashActive = await nosmai.hasEyelash();
final eyebrowActive = await nosmai.hasEyebrow();
```

Remove one layer or all makeup:

```dart
await nosmai.removeLipstick();
await nosmai.removeEyeshadow();
await nosmai.removeBlusher();
await nosmai.removeEyelash();
await nosmai.removeEyebrow();

await nosmai.removeAllMakeup();
```

### Android makeup

Android accepts a style constant and RGB components from `0.0` to `1.0`.

```java
NosmaiBeauty.applyLipstickStyle(
    NosmaiBeauty.LIPSTICK_MATTE,
    0.62f,
    0.12f,
    0.18f
);

NosmaiBeauty.applyEyeshadowStyle(
    NosmaiBeauty.EYESHADOW_NATURAL,
    0.42f,
    0.30f,
    0.48f
);

NosmaiBeauty.applyBlusherStyle(
    NosmaiBeauty.BLUSHER_NATURAL,
    0.95f,
    0.38f,
    0.42f
);

NosmaiBeauty.applyEyelashStyle(
    NosmaiBeauty.EYELASH_NATURAL
);
```

The Android eyebrow method uses style values `0`, `1`, and `2` for Natural,
Bold, and Arched:

```java
int naturalEyebrowStyle = 0;

NosmaiBeauty.applyEyebrowStyle(
    naturalEyebrowStyle,
    0.16f,
    0.11f,
    0.08f
);
```

Set the intensity of an active layer:

```java
NosmaiBeauty.setMakeupIntensity(
    NosmaiBeauty.MAKEUP_LIPSTICK,
    0.60f
);

NosmaiBeauty.setMakeupIntensity(
    NosmaiBeauty.MAKEUP_EYESHADOW,
    0.30f
);
```

Check or remove one layer:

```java
boolean active = NosmaiBeauty.isMakeupActive(
    NosmaiBeauty.MAKEUP_LIPSTICK
);

NosmaiBeauty.removeMakeup(
    NosmaiBeauty.MAKEUP_LIPSTICK
);
```

Remove all built-in makeup:

```java
NosmaiBeauty.removeMakeup(NosmaiBeauty.MAKEUP_ALL);
```

Use `clearBuiltInMakeup()` when the screen is being reset and the complete
built-in makeup manager should be released:

```java
NosmaiBeauty.clearBuiltInMakeup();
```

### iOS makeup

iOS provides style enums and color indexes:

```objc
NosmaiEffectsEngine *effects = [NosmaiCore shared].effects;

[effects applyLipstickWithStyle:NosmaiLipstickStyleMatte
                     colorIndex:0];

[effects applyEyeshadowWithStyle:NosmaiEyeshadowStyleNatural
                      colorIndex:0];

[effects applyBlusherWithStyle:NosmaiBlusherStyleNatural
                    colorIndex:0];

[effects applyEyelashWithStyle:NosmaiEyelashStyleNatural];

[effects applyEyebrowWithStyle:NosmaiEyebrowStyleNatural
                    colorIndex:0];
```

Read the available colors before presenting a color list:

```objc
NSArray<NosmaiMakeupColor *> *colors =
    [NosmaiCore shared].effects.lipstickColors;

for (NosmaiMakeupColor *color in colors) {
    NSLog(@"%@, %.2f, %.2f, %.2f",
          color.name,
          color.r,
          color.g,
          color.b);
}
```

Change active intensities:

```objc
[effects setLipstickIntensity:0.60f];
[effects setEyeshadowIntensity:0.30f];
[effects setBlusherIntensity:0.25f];
[effects setEyelashIntensity:0.40f];
[effects setEyebrowIntensity:0.30f];
```

Check and remove individual layers:

```objc
if (effects.hasLipstick) {
    [effects removeLipstick];
}

[effects removeEyeshadow];
[effects removeBlusher];
[effects removeEyelash];
[effects removeEyebrow];
```

Remove all makeup:

```objc
[effects removeAllMakeup];
```

The `NosmaiSDKBeauty.h` category also provides custom RGB overloads for native
iOS applications that need an exact makeup color.

## Face shaping

Face shaping changes facial geometry instead of adding color. Keep the default
values low and let the user adjust each control separately.

### Flutter

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.setFaceSlimLevel(0.20);
await nosmai.setEyeSizeLevel(0.10);
await nosmai.setNoseSlimLevel(0.10);
```

Reset all face shaping:

```dart
await nosmai.removeAllMorphing();
```

### Android

```java
NosmaiBeauty.applyMorphFaceSlim(0.20f);
NosmaiBeauty.applyMorphEyeSize(0.10f);
NosmaiBeauty.applyMorphNoseSlim(0.10f);
```

Reset these controls:

```java
NosmaiBeauty.applyMorphFaceSlim(0.0f);
NosmaiBeauty.applyMorphEyeSize(0.0f);
NosmaiBeauty.applyMorphNoseSlim(0.0f);
```

Android also exposes chin, lip size, and jawline controls:

```java
NosmaiBeauty.applyMorphChinSize(0.0f);
NosmaiBeauty.applyMorphLipSize(0.10f);
NosmaiBeauty.applyMorphJawline(0.10f);
```

`applyMorphChinSize` accepts values from `-1.0` to `1.0`. The other controls in
this example use `0.0` to `1.0`.

### iOS

```objc
NosmaiEffectsEngine *effects = [NosmaiCore shared].effects;

[effects setFaceSlimLevel:0.20f];
[effects setEyeSizeLevel:0.10f];
[effects setNoseSlimLevel:0.10f];
```

iOS also provides eyebrow position, chin size, lip size, and jawline controls:

```objc
[effects setEyebrowPositionLevel:0.0f];
[effects setChinSizeLevel:0.0f];
[effects setLipSizeLevel:0.10f];
[effects setJawlineLevel:0.10f];
```

Reset all face shaping:

```objc
[effects removeAllMorphing];
```

## Eye color

Eye color changes the visible iris color. Use a moderate intensity so natural
iris detail remains visible.

### Flutter

```dart
import 'package:flutter/material.dart';

final nosmai = NosmaiFlutter.instance;

await nosmai.setEyeColor(
  const Color(0xFF4F7A56),
  intensity: 0.45,
);
```

Update or remove it:

```dart
await nosmai.setEyeColorIntensity(0.30);
await nosmai.removeEyeColoring();
```

### Android

Android calls this feature an eye lens:

```java
NosmaiBeauty.applyEyeLens(
    0.31f,
    0.48f,
    0.34f,
    0.45f
);
```

Update, inspect, or remove it:

```java
NosmaiBeauty.setEyeLensIntensity(0.30f);

boolean active = NosmaiBeauty.isEyeLensActive();

NosmaiBeauty.removeEyeLens();
```

### iOS

```objc
NosmaiEffectsEngine *effects = [NosmaiCore shared].effects;

[effects setEyeColorR:0.31f g:0.48f b:0.34f];
[effects setEyeColorIntensity:0.45f];
```

Update, inspect, or remove it:

```objc
[effects setEyeColorIntensity:0.30f];

BOOL active = effects.hasEyeColoring;

[effects removeEyeColoring];
```

## Build a complete beauty preset

A preset should store style choices and numeric values in application state.
Apply it only after the preview is ready.

Flutter example:

```dart
Future<void> applyNaturalPreset() async {
  final nosmai = NosmaiFlutter.instance;

  await nosmai.applySkinSmoothing(0.30);
  await nosmai.applySkinWhitening(0.08);
  await nosmai.applySharpening(0.12);
  await nosmai.applyTeethWhitening(0.18);

  await nosmai.applyLipstick(
    style: NosmaiLipstickStyle.natural,
    intensity: 0.50,
  );
  await nosmai.applyEyeshadow(
    style: NosmaiEyeshadowStyle.natural,
    intensity: 0.22,
  );
  await nosmai.applyBlusher(
    style: NosmaiBlusherStyle.natural,
    intensity: 0.20,
  );
  await nosmai.applyEyelash(
    style: NosmaiEyelashStyle.natural,
    intensity: 0.32,
  );

  await nosmai.setFaceSlimLevel(0.12);
  await nosmai.setEyeSizeLevel(0.06);
  await nosmai.setNoseSlimLevel(0.05);
}
```

Keep the selected values in your application rather than reading them from the
camera preview. This makes it easier to restore the same look after changing
screens, switching cameras, or recreating the preview.

## Reset behavior

Choose the narrowest reset method for the user action.

| User action | Flutter | Android | iOS |
| --- | --- | --- | --- |
| Remove one makeup layer | `removeLipstick()` or the matching method | `removeMakeup(category)` | `removeLipstick` or the matching method |
| Remove all makeup | `removeAllMakeup()` | `removeMakeup(MAKEUP_ALL)` | `removeAllMakeup` |
| Remove face shaping | `removeAllMorphing()` | Set active morph values to `0.0` | `removeAllMorphing` |
| Remove eye color | `removeEyeColoring()` | `removeEyeLens()` | `removeEyeColoring` |
| Remove skin and color controls | `removeBuiltInFilters()` | `removeAllBeautyFilters()` | `removeBuiltInFilters` |
| Remove all beauty groups | `removeAllBeautyEffects()` | `clearAllBeautyFilters()` and `removeEyeLens()` | `removeAllBeautyEffects` |

Do not use a full SDK reset when the user only turns off lipstick. A narrow
reset keeps unrelated filters and camera state unchanged.

## Built-in beauty and `beauty_effect`

Built-in beauty is best when the application needs:

- live sliders
- separate makeup controls
- runtime color choices
- user-created presets
- individual remove buttons

A packaged `beauty_effect` is best when the application needs:

- a complete authored look
- one preview image and one selection item
- assets and settings delivered as one `.nosmai` package
- cloud distribution and versioning

Apply a packaged look with the same `applyEffect(path)` method used for other
`.nosmai` packages. Do not call the built-in makeup methods merely to apply a
packaged `beauty_effect`.

Built-in beauty, makeup, reshape, color, and hair controls do not coexist with an external `effect` or `beauty_effect`. Applying the package clears those built-in controls. Applying a built-in control later clears the active AR package. Regular external `filter` packages and manual backgrounds can remain active with built-in beauty.

Always rebuild the selected UI from the apply result and active-state listener after switching modes. Do not leave a built-in slider or AR item selected after the SDK reports that it was cleared.

## User interface guidance

Use controls that match the setting:

- Use a slider for strength.
- Use swatches for colors.
- Use a horizontal style list for Classic, Matte, and Natural choices.
- Use a visible Off choice for every feature.
- Update the selected state only after an asynchronous call succeeds.
- Keep the last selected value when the user changes only the style.
- Debounce very frequent slider changes if the application sends more updates
  than the display can present.

For a natural default look, avoid enabling every feature at medium or maximum
strength. A small number of subtle controls usually looks better.

## Performance guidance

Beauty rendering is designed for live use, but the final workload depends on
the number of active features, camera resolution, recording, streaming, and the
device GPU.

Follow these rules:

1. Keep the camera at 30 FPS unless the target devices have been tested at a
   higher rate.
2. Do not repeatedly reapply the same makeup style for every camera frame.
3. Apply the style once, then use its intensity method for slider updates.
4. Avoid rebuilding the camera preview while changing beauty settings.
5. Keep image assets at the resolution required by the effect.
6. Test beauty while recording and live streaming, not only in preview.
7. Test rapid style changes and clear actions.
8. Release the camera when its screen closes.

## Test cases

Test every production preset with:

- one face centered
- the face near each screen edge
- fast head movement
- head rotation to the left and right
- mouth open and closed
- eyes open, blinking, and partly closed
- glasses and facial hair
- bright, dim, and mixed lighting
- front and back camera where supported
- camera switching
- leaving and reopening the camera screen
- recording
- live streaming
- repeated apply and remove actions

Watch for color outside the intended facial region, delayed attachment, sharp
edges, visible movement, stale effects after the face disappears, and a drop in
preview frame rate.

## Troubleshooting

### The feature is selected but not visible

Confirm that:

1. the preview is ready
2. a face is visible
3. the intensity is greater than `0.0`
4. the selected style is supported
5. the feature was not removed by a later clear action

### Makeup looks too strong

Reduce the layer intensity before changing the style. Also reduce skin
whitening and sharpening because strong global controls can make makeup edges
more noticeable.

### Makeup follows the face with visible delay

Test without recording or streaming, verify the camera remains near the target
frame rate, and reduce unnecessary repeated UI updates. If the delay appears
only on particular devices, collect the device model, OS version, active
features, camera resolution, and frame-rate logs.

### An effect remains after pressing Clear

Use the reset method for the correct group. Skin controls, makeup, face shaping,
and eye color have separate remove methods. Use the combined beauty reset only
when the user intends to remove all four groups.

## Production checklist

Before release:

- initialize Nosmai before applying a preset
- wait for preview readiness
- keep all normal strength values within `0.0` to `1.0`
- provide an Off choice for each feature
- use individual intensity methods for slider updates
- preserve selected settings in application state
- handle camera switching and screen reopening
- verify each remove action
- test no-face behavior
- test multiple skin tones and face shapes
- test low and high lighting
- test recording and live streaming
- test supported low-end and high-end devices
- disable debug logging in production

## Next steps

- Read [Filters and effects](/docs/effects/filters-and-effects) to understand
  packaged `beauty_effect` files.
- Read [Errors and troubleshooting](/docs/effects/errors) for initialization,
  camera, filter, and performance failures.
- Read
  [Flutter live streaming with Agora](flutter-live-streaming-agora.md) before
  publishing the processed camera preview.

---

# Cloud filters
Source: https://docs.nosmai.com/docs/effects/cloud-filters/

## Overview

Nosmai Cloud lets an application show filters, effects, packaged beauty looks, and backgrounds without including every `.nosmai` file in the initial application download.

The catalog returns metadata and preview URLs. The protected `.nosmai` package is downloaded only when it is required. After download, the application applies the returned local file path through the same `applyEffect(path)` method used for bundled packages.

```text
Request catalog metadata
    |
    v
Render names and previews
    |
    v
Download the selected package
    |
    v
Receive and validate a local path
    |
    v
Call applyEffect(localPath)
    |
    v
Confirm selected state from the SDK
```

Camera frames are not uploaded to list, download, or apply a cloud package.

## Requirements

Before requesting cloud filters:

1. Initialize the SDK successfully.
2. Use a license that includes cloud filters.
3. Keep internet access available for catalog requests and downloads.
4. Keep enough application storage available for the downloaded package.
5. Use the cloud catalog version supported by the installed SDK.

Cloud catalog schema `2.0.0` is the current default. It is separate from Android SDK `3.0.0`, iOS SDK `3.0.0`, and Flutter package `3.0.6`.

An already downloaded and cached package can be applied from its local path without downloading it again. License rules still apply.

## Cloud categories

The application should use the platform enum or normalized category where one is available.

| Interface section | Package manifest type | Cloud request value | Flutter enum |
| --- | --- | --- | --- |
| Filters | `filter` | `filter` | `NosmaiCloudFilterType.filter` |
| Effects | `effect` | `effects` | `NosmaiCloudFilterType.effects` |
| Beauty | `beauty_effect` | `beauty_effect` | `NosmaiCloudFilterType.beautyEffect` |
| Backgrounds | `background` | `bg` | `NosmaiCloudFilterType.background` |

Cloud request categories and package manifest types are related but not identical. Do not write a cloud request value into a package manifest. The protected package manifest must use `filter`, `effect`, `beauty_effect`, or `background`.

## Recommended interface state

Track cloud state per filter identifier:

```text
notDownloaded
downloading
downloaded
applying
selected
failed
```

Also track the catalog request separately:

```text
idle
loadingFirstPage
showingCachedData
loadingNextPage
refreshing
failed
```

Recommended behavior:

- Show cached metadata immediately when the platform provides it.
- Refresh the catalog away from the UI thread.
- Keep one current catalog request for each catalog screen or controller.
- Keep one in-flight download for each cloud identifier.
- Ignore repeated taps while that item is downloading or applying.
- Mark an item selected only after apply succeeds.
- Confirm selection from the active-state listener.
- Keep the completed download cached even if its sheet closed during download.
- Do not update a disposed screen when a request finishes.
- Offer retry for temporary network failures.
- Do not delete other categories while processing a scoped or paginated response.

## Flutter

### List all categories

```dart
final nosmai = NosmaiFlutter.instance;
final filters = await nosmai.getCloudFilters();
```

Calling the method without a page requests all available pages for backward compatibility. For a large catalog, request one page at a time.

### List one category with pagination

```dart
final filters = await nosmai.getCloudFilters(
  filterType: NosmaiCloudFilterType.background,
  version: NosmaiCloudFilterVersion.v2,
  page: 1,
  limit: 20,
  fetchAllPages: false,
);

final pagination = nosmai.lastPaginationInfo;
```

Load the next page only when it exists:

```dart
if (pagination?.hasNextPage == true) {
  final nextPage = await nosmai.getCloudFilters(
    filterType: NosmaiCloudFilterType.background,
    page: pagination!.currentPage + 1,
    limit: pagination.itemsPerPage,
    fetchAllPages: false,
  );

  // Merge by cloudIdentifier to avoid duplicate cells.
}
```

Do not start a second next-page request while the first one is still running.

### Download and apply

Use `cloudIdentifier`, not the catalog record `id`, for download and cache operations:

```dart
final filter = filters.first;
final result = await nosmai.downloadCloudFilter(
  filter.cloudIdentifier,
);

final path = (result['path'] ?? result['localPath']) as String?;
if (path == null || path.isEmpty) {
  throw StateError('Cloud filter download returned no local path');
}

final applied = await nosmai.applyEffect(path);
if (!applied) {
  // Keep the previous selected state and offer retry.
}
```

`cloudIdentifier` preserves compatibility with catalogs that expose separate record and downloadable-package identifiers.

Remove the cached download when the user explicitly requests it:

```dart
await nosmai.removeCloudFilter(filter.cloudIdentifier);
```

Removing a cached file is different from clearing an active render slot. If the item is active, remove its visual state through `removeEffect(filter)` or the correct narrow clear method as well.

### Protect a closing screen

An asynchronous request can finish after its sheet or route is disposed. Check ownership before changing widget state:

```dart
if (!context.mounted) return;
```

Keep download futures in a map keyed by `cloudIdentifier`. Reuse the existing future for repeated taps and remove it from the map when it completes.

## Android

Android exposes cloud operations through `NosmaiCloud`.

### Show cached data without freezing the sheet

`NosmaiCloud.cachedList()` is an immediate snapshot intended for rendering existing catalog data. It does not perform a network request.

```java
List<NosmaiCloud.Item> cached = NosmaiCloud.cachedList();
List<NosmaiCloud.Item> visible = new ArrayList<>();

for (NosmaiCloud.Item item : cached) {
    if ("bg".equals(item.category) ||
        "background".equals(item.category)) {
        visible.add(item);
    }
}

renderCloudItems(visible);
```

Filter a cached snapshot by the active tab before rendering it. An All snapshot must not be displayed unchanged inside the Filters or Backgrounds tab.

Do not call a network-backed catalog refresh on the main thread. Refresh through an executor and post only the result back to the UI:

```java
private final ExecutorService cloudExecutor =
        Executors.newSingleThreadExecutor();
private final Handler mainHandler =
        new Handler(Looper.getMainLooper());
private final AtomicLong cloudRequest = new AtomicLong();

private void loadBackgrounds() {
    long request = cloudRequest.incrementAndGet();

    NosmaiCloud.FilterQuery query =
            new NosmaiCloud.FilterQuery();
    query.filterType = "bg";
    query.page = 1;
    query.limit = 20;
    query.fetchAllPages = false;
    query.cleanupRemoved = false;

    cloudExecutor.execute(() -> {
        boolean success = NosmaiCloud.fetch(query);
        List<NosmaiCloud.Item> items = success
                ? NosmaiCloud.list()
                : Collections.emptyList();

        mainHandler.post(() -> {
            if (request != cloudRequest.get() || isFinishing()) {
                return;
            }

            if (success) {
                renderCloudItems(items);
            } else {
                showCloudRetry();
            }
        });
    });
}
```

Increase `cloudRequest` when the user changes tab, closes the sheet, or starts a new refresh. This prevents an old response from replacing a newer interface state.

Keep `cleanupRemoved` false for a category-specific or single-page request. Such a response is not the complete server catalog and must not be used to remove cached items from other categories or pages.

### Download and apply

`NosmaiCloud.download` performs the package transfer away from the calling UI flow. Its callbacks must still be marshalled to the main thread before changing views.

```java
NosmaiCloud.download(
    item.id,
    progress -> mainHandler.post(
        () -> updateDownloadProgress(item.id, progress)
    ),
    (filterId, success, localPath, error) -> {
        if (!success || localPath == null || localPath.isEmpty()) {
            mainHandler.post(() -> showDownloadRetry(filterId));
            return;
        }

        NosmaiEffects.applyEffect(
            localPath,
            new NosmaiEffects.EffectCallback() {
                @Override
                public void onSuccess() {
                    // Confirm final UI from the pipeline listener.
                }

                @Override
                public void onError(String message) {
                    mainHandler.post(
                        () -> showApplyRetry(filterId)
                    );
                }
            }
        );
    }
);
```

Keep a thread-safe set of downloading identifiers. If the set already contains `item.id`, ignore the repeated tap. Remove the identifier in both success and failure paths.

When the owning screen is destroyed, increment the request generation, remove listeners, and stop the executor if it is owned by that screen. A completed package can remain in the SDK cache for the next screen.

## iOS

The iOS effects engine provides asynchronous catalog, progress, download, and apply callbacks.

### Request one category

```objc
NosmaiCloudFilterRequestOptions *options =
    [NosmaiCloudFilterRequestOptions defaultOptions];
options.filterType = @"bg";
options.version = NosmaiCloudFilterVersion2;
options.page = 1;
options.limit = 20;
options.fetchAllPages = NO;
options.cleanupRemovedFilters = NO;

[[NosmaiCore shared].effects
    getCloudFiltersWithOptions:options
                    completion:^(NSArray<NSDictionary *> *filters,
                                 NosmaiCloudFilterPaginationInfo *pagination,
                                 NSError *error) {
        dispatch_async(dispatch_get_main_queue(), ^{
            if (error != nil) {
                [self showCloudRetry];
                return;
            }

            [self renderCloudFilters:filters pagination:pagination];
        });
    }];
```

Use `effects`, `filter`, `beauty_effect`, or `bg` for a scoped request. Use `nil` to request all categories.

### Download and apply

```objc
NSString *filterId = filter[@"filterId"] ?: filter[@"id"];

[[NosmaiCore shared].effects
    downloadCloudFilter:filterId
               progress:^(float progress) {
        dispatch_async(dispatch_get_main_queue(), ^{
            [self updateDownloadProgress:progress filterId:filterId];
        });
    }
             completion:^(BOOL success,
                          NSString *localPath,
                          NSError *error) {
        if (!success || localPath.length == 0) {
            dispatch_async(dispatch_get_main_queue(), ^{
                [self showDownloadError:error filterId:filterId];
            });
            return;
        }

        [[NosmaiCore shared].effects
            applyEffect:localPath
             completion:^(BOOL applied, NSError *applyError) {
                dispatch_async(dispatch_get_main_queue(), ^{
                    [self finishApply:applied error:applyError];
                });
            }];
    }];
```

Keep one in-flight operation per filter identifier. Capture the screen or request owner weakly in production code so a completed callback does not keep a closed sheet alive.

Remove a cached download only when requested:

```objc
BOOL removed =
    [[NosmaiCore shared].effects removeCloudFilter:filterId];
```

## Applying and clearing rules

The downloaded local path uses the normal package policy:

- A new `filter` replaces the active external color filter.
- `effect` and `beauty_effect` share the AR slot and replace each other.
- A new `background` package replaces the previous background package.
- Built-in beauty, makeup, reshape, color, and hair controls are mutually exclusive with `effect` and `beauty_effect`.
- A regular external `filter` can remain active with an AR package, background, or built-in beauty.
- An AR effect that owns its background can replace existing background content.

Use the active-state listener after apply and removal. Do not decide selected UI only from a tap or downloaded filename.

## Error handling

| Failure | Recommended response |
| --- | --- |
| Cloud feature not licensed | Hide or disable cloud entry points and explain the plan requirement |
| First page fails | Show cached data if available and provide Retry |
| Next page fails | Keep existing items and retry only that page |
| Download interrupted | Keep the item unselected and offer Retry |
| Local path is empty | Treat download as failed and do not call `applyEffect` |
| Apply fails | Preserve the previous active selection and show a retry action |
| Sheet closes during work | Let safe cache work finish but ignore UI updates for the disposed owner |
| Repeated tap | Reuse or ignore the existing operation for that identifier |
| Cached file was deleted | Download again and refresh metadata |

Do not show backend payloads, full local paths, access tokens, or license keys to the end user.

## Test checklist

Before release, test:

1. All, Filters, Effects, Beauty, and Background tabs.
2. Empty categories and empty search results.
3. First page, next page, refresh, and end-of-list behavior.
4. Cached data followed by a background refresh.
5. Download success, network loss, retry, and low storage.
6. Rapid repeated taps on one item.
7. Downloads of different items at the same time if the interface allows them.
8. Closing and reopening the sheet during a download.
9. Applying a downloaded item and reusing it after relaunch.
10. Removing the active package without deleting unrelated downloads.
11. Removing a cached download and downloading it again.
12. Package coexistence and replacement rules.
13. Offline launch after the catalog and selected package were cached.
14. Slow networks and large catalogs without blocking the camera preview.

Continue with [Filters and effects](/docs/effects/filters-and-effects) for package slots and [Errors and troubleshooting](/docs/effects/errors) for failure diagnosis.

---

# Native live streaming
Source: https://docs.nosmai.com/docs/effects/native-live-streaming/

# Native live streaming

> This page is for native Android and iOS applications. For Flutter with Agora,
> read [Flutter live streaming with Agora](flutter-live-streaming-agora.md).

## Overview

Native live streaming sends the processed Nosmai camera result to a video
encoder or streaming SDK.

The remote viewer should receive the same effects that appear in the local
preview:

```text
Camera
  -> Nosmai filters and beauty
  -> Processed frame
  -> Native streaming SDK or encoder
  -> Remote viewers
```

Nosmai owns the camera and effect rendering. Your application still owns:

- the streaming provider account and App ID
- channel names and access tokens
- broadcaster and audience roles
- audio publishing
- connection events
- remote-user rendering
- application navigation and session state

Do not start a second local camera from the streaming SDK while Nosmai owns the
camera. Publish the processed Nosmai output instead.

## Choose the platform

| Application | Processed output | Recommended API |
| --- | --- | --- |
| Native Android, broad compatibility | CPU frame | `NosmaiSDK.setFrameCallback(...)` |
| Native Android with Agora shared EGL | GPU texture | `NosmaiSDK.setTextureFrameCallback(...)` |
| Native iOS | `CVPixelBufferRef` | `NosmaiCore.liveFrameStreamCallback` |
| Flutter with Agora | Bridge-managed output | `NosmaiAgoraBridge` |

The Android GPU texture flow provides the lowest copy cost, but it requires
correct EGL context sharing and texture ownership. The CPU callback is easier
to connect to a general encoder but costs more per frame.

## Common lifecycle

Use this order:

1. Create the streaming engine.
2. On Android GPU texture integrations, register the shared EGL context before
   Nosmai creates its GL context.
3. Initialize Nosmai and start its camera preview.
4. Apply filters, beauty, makeup, or background effects.
5. Register the processed frame consumer.
6. Configure the streaming SDK to publish external video.
7. Join the channel and start publishing.
8. On stop, prevent new publishing before clearing callbacks and releasing
   resources.

The streaming SDK must not publish its own raw camera track at the same time.
Doing so can show an unfiltered stream or cause camera ownership conflicts.

## Android

### Select the output mode

Android provides three render modes:

| Mode | Local preview | Processed streaming output |
| --- | --- | --- |
| `PREVIEW_ONLY` | Yes | No |
| `STREAMING_ONLY` | No | Yes |
| `DUAL_OUTPUT` | Yes | Yes |

For a broadcaster who needs to see the local preview:

```java
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.DUAL_OUTPUT);
```

Use `STREAMING_ONLY` only when the application intentionally does not show the
local preview. A blank local Nosmai view is expected in that mode.

### CPU frame output

The CPU callback works with streaming systems that accept copied image or I420
data.

```java
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.DUAL_OUTPUT);

NosmaiSDK.setFrameCallback(frame -> {
    streamingConsumer.pushFrame(
            frame.pixelBuffer,
            frame.width,
            frame.height,
            frame.format,
            frame.timestampNs
    );
});
```

`FrameData.format` uses:

| Value | Format |
| --- | --- |
| `0` | RGBA |
| `1` | I420 |
| `2` | NV21 |

Use `timestampNs` as the video timestamp when the receiving API accepts
nanoseconds. Convert it carefully when the streaming SDK expects milliseconds
or microseconds.

The callback is not the Android UI thread. Do not block it with network
requests, file access, image compression, or UI work. Pass the frame directly
to the encoder and return.

### GPU texture output for Agora

The Android texture route avoids full-frame GPU-to-CPU readback.

Before Nosmai initialization:

```java
NosmaiSDK.setAgoraShareContext(agoraEglContextHandle);
```

After the Agora video consumer is ready:

```java
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.DUAL_OUTPUT);

NosmaiSDK.setTextureFrameCallback(
        (texId, width, height, timestampNs, fence) -> {
    streamingConsumer.pushTexture(
            texId,
            width,
            height,
            timestampNs,
            fence,
            () -> NosmaiSDK.releaseStreamSlot(texId)
    );
});
```

`streamingConsumer` represents the adapter written for the selected streaming
SDK. Its completion callback must run after the encoder has finished using the
texture. If submission fails before ownership is accepted, call
`releaseStreamSlot(texId)` from that failure path.

The consumer must also wait for the supplied EGL fence before sampling the
texture.

Every delivered texture ID must be released, including:

- successful publish
- dropped frame
- encoder error
- channel leave
- application pause

Registering the share context after Nosmai initialization is too late because a
GL context can join the share group only when that context is created. A late
registration can produce a correct local preview and black remote video.

### Portrait output

When the receiving encoder requires physically rotated portrait frames:

```java
NosmaiSDK.setPortraitOffscreenOutput(true);
```

Do not apply the same rotation again in the encoder. Disable portrait output
when the stream stops:

```java
NosmaiSDK.setPortraitOffscreenOutput(false);
```

### Stop Android streaming

Stop the publisher first, then clear the Nosmai consumers:

```java
NosmaiSDK.setTextureFrameCallback(null);
NosmaiSDK.setFrameCallback(null);
NosmaiSDK.setPortraitOffscreenOutput(false);
NosmaiSDK.setRenderMode(NosmaiSDK.RenderMode.PREVIEW_ONLY);
```

Leave the streaming channel and release the streaming engine according to its
own lifecycle. Release Nosmai only when the application no longer needs the
camera.

## iOS

### Receive processed frames

When Nosmai owns the camera, use the high-level live frame callback:

```objective-c
[NosmaiCore shared].liveFrameStreamCallback =
    ^(CVPixelBufferRef pixelBuffer, double timestamp) {
        [streamingConsumer pushPixelBuffer:pixelBuffer
                                 timestamp:timestamp];
    };
```

Assigning the callback enables processed live frame output. Setting it to `nil`
disables that extra output and conserves resources.

The callback runs on a background processing thread. Do not update UIKit
directly from it.

### Pixel buffer ownership

The callback owns the `CVPixelBufferRef` only for the duration of the callback.
If the streaming SDK uses the buffer asynchronously, retain it first and
release it when the consumer finishes:

```objective-c
[NosmaiCore shared].liveFrameStreamCallback =
    ^(CVPixelBufferRef pixelBuffer, double timestamp) {
        CVPixelBufferRetain(pixelBuffer);

        [streamingConsumer pushPixelBuffer:pixelBuffer
                                 timestamp:timestamp
                                completion:^{
            CVPixelBufferRelease(pixelBuffer);
        }];
    };
```

Do not retain every frame without a matching release. That causes continuous
memory growth during a long stream.

### Stop iOS streaming

Stop the publisher, then clear the callback:

```objective-c
[NosmaiCore shared].liveFrameStreamCallback = nil;
```

Leave the channel and release the streaming engine using its documented
lifecycle. Keep Nosmai running if the application returns to the normal camera
preview.

## Apply effects

Apply filters and beauty through the standard native Nosmai APIs. Streaming
does not require a separate filter instance.

The expected order is:

1. Start the Nosmai camera.
2. Apply or update effects through the normal SDK methods.
3. Publish the processed output callback.

When an effect is replaced, one or more frames may be skipped briefly while the
new resources are prepared. Do not queue old frames during that transition.

## Performance

### Keep one current frame

If the encoder is still using a previous frame, drop an older waiting frame and
keep the newest frame. An unlimited queue creates visible delay between face
movement and the remote effect.

### Match the encoder to the output

Configure the encoder for the same:

- width and height
- portrait or landscape orientation
- frame rate
- expected pixel format

A mismatch can cause cropping, stretching, extra conversion, or black output.

Start with 720p at 30 FPS for live effects. Increase resolution only after
testing beauty, AR effects, audio, network publishing, and thermal behavior
together on mid-range devices.

### Avoid extra conversions

Do not convert each frame through:

- Android `Bitmap`
- iOS `UIImage`
- JPEG
- PNG
- Dart byte arrays

Use the native frame type accepted by the encoder. Prefer the Android texture
route when a compatible shared EGL integration is available.

## Troubleshooting

### Local preview works but remote video is black on Android

Check that:

- the shared EGL context was registered before Nosmai initialization
- the streaming SDK is publishing external video, not its own camera
- the texture consumer waits for the supplied fence
- every texture slot is released after use
- the encoder is configured for the supplied texture dimensions

### Remote stream has no effects

The streaming SDK is probably publishing its raw camera track. Disable that
track and publish the processed Nosmai output.

### Stream becomes delayed

Remove frame queues and image conversions. Keep only the newest waiting frame.
Check whether the encoder resolution or bitrate is too high for the device.

### Memory increases on iOS

Verify that every `CVPixelBufferRetain` has one matching
`CVPixelBufferRelease`. Clear `liveFrameStreamCallback` when publishing stops.

### Second stream is black or frozen

Clear all callbacks and outstanding texture ownership during the first stop.
Create or reconnect streaming-specific resources for the new session rather
than reusing a released encoder or native handle.

## Next steps

- Read [Flutter live streaming with Agora](flutter-live-streaming-agora.md) for
  Flutter bridge setup.
- Read [Off-screen rendering](../core-concepts/off-screen-rendering.md) when
  your application supplies its own external video frames.
- Read [Errors and troubleshooting](../reference/errors.md) for general camera
  and lifecycle problems.

---

# Flutter live streaming with Agora
Source: https://docs.nosmai.com/docs/effects/flutter-live-streaming-agora/

# Flutter live streaming with Agora

> This page is for Flutter applications using `nosmai_agora_bridge`. For native
> Android or iOS integration, read
> [Native live streaming](native-live-streaming.md).

## Overview

Nosmai does not tap your stream. It **becomes** the stream. The SDK renders the filtered camera frame on the GPU and hands that frame to Agora as a custom video track, so what viewers see is exactly what the broadcaster sees, effects included.

The frame never round-trips through the CPU. On **Android** the Agora engine and the Nosmai renderer share an EGL **share group**, so Nosmai passes a texture ID straight to Agora's encoder. On **iOS** there is no share group; frames are handed over as `CVPixelBuffer`/IOSurface instead. The Dart API is identical on both.

```
camera -> Nosmai render (beauty, makeup, AR, background) -> Agora custom video track -> viewers
```

The `nosmai_agora_bridge` package does all the native plumbing. Install it beside your existing `agora_rtc_engine` and `nosmai_camera_sdk`.

```yaml title="pubspec.yaml"
dependencies:
  nosmai_camera_sdk: ^3.0.6
  nosmai_agora_bridge:
    git:
      url: https://github.com/nosmai/nosmai_agora_bridge.git
  agora_rtc_engine: ^6.5.3
  permission_handler: ^12.0.1
```

Streaming needs **camera and microphone** permission. Request both before you initialize the engine, and declare `CAMERA` + `RECORD_AUDIO` in `AndroidManifest.xml` and `NSCameraUsageDescription` + `NSMicrophoneUsageDescription` in `Info.plist`.

## Setup order

The ordering rule below is load-bearing. Get it wrong and the local preview looks perfect while every remote viewer sees black.

1. **Call `getNativeHandle` first, before `NosmaiFlutter.initialize`.** This creates the shared Agora engine and stashes Agora's `EGLContext` for Nosmai to consume.
2. **Then initialize Nosmai.** Its GL context is created now and joins the share group.
3. **Create the Agora engine from that same handle.**
4. **Start the preview**, then start streaming.

```dart title="main.dart"
// 1. Register the share context while NO Nosmai GL context exists yet.
await NosmaiAgoraBridge.getNativeHandle(agoraAppId: agoraAppId);

// 2. Now build the Nosmai GL context. It picks up the share context above.
await NosmaiFlutter.initialize(nosmaiLicenseKey);
```

> [!WARNING]
> An EGL context can only join a share group **at creation time**. If `NosmaiFlutter.initialize` runs first, the share context arrives too late, is silently ignored, and Agora's encoder cannot sample Nosmai's output texture, which produces black remote video. iOS needs this call too, for a different reason: it constructs the bridge's native controller, without which streaming falls back to Agora's unfiltered raw camera.

## Create the engine

Build the engine from the bridge's handle rather than a plain one. `getNativeHandle` is idempotent and returns the cached handle.

```dart title="agora_service.dart"
final handle = await NosmaiAgoraBridge.getNativeHandle(agoraAppId: appId);
final engine = createAgoraRtcEngine(sharedNativeHandle: handle);

await engine.initialize(RtcEngineContext(
  appId: appId,
  channelProfile: ChannelProfileType.channelProfileLiveBroadcasting,
));
await engine.enableVideo();
await engine.enableAudio();

// Match the encoder to the frames you actually push: 9:16 portrait.
await engine.setVideoEncoderConfiguration(const VideoEncoderConfiguration(
  dimensions: VideoDimensions(width: 720, height: 1280),
  frameRate: 30,
  bitrate: 2500,
  minBitrate: 1200,
  orientationMode: OrientationMode.orientationModeFixedPortrait,
));
```

> [!NOTE]
> On iOS the bridge builds its engine with `sharedEngineWithConfig:`, a **process singleton**. Always wrap the bridge's handle instead of creating a second engine, or two Dart wrappers end up driving one native engine with conflicting state.

## Start streaming

The bridge sets the external video source, flips Nosmai to dual-output and **joins the channel itself** with `publishCustomVideoTrack: true` and `publishCameraTrack: false`. Do not call `joinChannel` yourself on this path.

```dart
final ok = await NosmaiAgoraBridge.startStreaming(
  channelName: channelName,
  token: (token != null && token.isNotEmpty) ? token : null,
  uid: uid,
);
```

Guard against duplicate starts. If a server event can fire twice, a second `startStreaming` while the first join is still in flight makes Agora reject it and the cleanup path disarms the stream that was about to succeed.

Filters apply exactly as they do off-stream. Anything you set through `NosmaiFlutter.instance` is already in the published frames.

## Teardown

```dart
await NosmaiAgoraBridge.stopStreaming();   // returns Nosmai to preview-only
if (!Platform.isIOS) await engine.leaveChannel();

// only when tearing the whole session down
await engine.release();
await NosmaiAgoraBridge.disposeNative();
```

On iOS `stopStreaming` already leaves the channel on the shared engine, so a second `leaveChannel` fires a spurious callback that can clear bridge state while a restart races. Android's bridge does not leave on your behalf and still needs it.

Keep `disposeNative` symmetric with `engine.release()`. Releasing the engine without clearing the bridge's cached handle leaves a dangling pointer, and the next `getNativeHandle` returns a dead engine.

## Troubleshooting

**Black remote video, local preview fine.** The share group was never established. Confirm `getNativeHandle` runs before `NosmaiFlutter.initialize`, and that the engine was created with `sharedNativeHandle:` rather than a plain `createAgoraRtcEngine()`.

**Second go-live freezes, then goes permanently black.** The Agora texture helper must be created and destroyed **per stream**, not once at init. A helper that survives `stopStreaming` keeps contending the single Nosmai GL worker, so the next go-live's surface release blocks behind a saturated worker. The bridge tears the helper down inside `stopStreaming`. Make sure you call it before starting the next stream rather than only leaving the channel.

**Remote view is zoomed in or softer than the preview.** The encoder is declaring a different aspect than the pushed frames. Nosmai publishes 720x1280 portrait; a 4:3 or adaptive-orientation encoder config makes Agora centre-crop the sides and upscale what is left. Set `orientationModeFixedPortrait` at 720x1280, and note that a `setVideoEncoderConfiguration` call after engine init overrides whatever the bridge configured.

**Stream falls back to the unfiltered camera.** `startStreaming` returned false, or `getNativeHandle` failed and the engine was built plain. Check the return value and fall back deliberately rather than silently.

**Camera switch loses filters on iOS.** Call `NosmaiAgoraBridge.notifyCameraSwitch()` after switching. Android detects the switch itself and the call is a no-op there.

---

# iOS
Source: https://docs.nosmai.com/docs/effects/ios/

## Overview

The iOS SDK provides public APIs for:

- SDK initialization and license handling
- front and back camera preview
- built-in beauty, makeup, and face shaping
- local and cloud `.nosmai` filters
- background blur and replacement
- active-effect state
- photo capture
- processed video recording
- processed output for live streaming

The SDK is written with Objective-C public interfaces and can be used from both Objective-C and Swift applications. The examples below use Objective-C so that every method name matches the public headers exactly.

The official iOS repository contains releases and sample code:

[github.com/nosmai/camera-sdk-ios](https://github.com/nosmai/camera-sdk-ios)

## Requirements

| Requirement | Value |
| --- | --- |
| Minimum iOS version | iOS 15.0 |
| Supported architecture | arm64 |
| Camera framework | AVFoundation |
| Privacy manifest | Included in `nosmai.framework` |
| Recommended camera frame rate | 30 FPS |
| Test environment | Physical iPhone or iPad |

Use a physical arm64 device for camera preview, face tracking, recording, and performance testing. The simulator is not a reliable environment for these features.

## Install

### Option 1: Use CocoaPods

Add the current Nosmai pod to the application:

```ruby title="Podfile"
platform :ios, '15.0'

target 'CameraApp' do
  use_frameworks!
  pod 'NosmaiCameraSDK', '3.0.0'
end
```

Then run:

```sh
pod install --repo-update
```

Open the generated `.xcworkspace` after CocoaPods finishes.

The native framework contains `PrivacyInfo.xcprivacy`. Keep it inside the framework so App Store archives include the SDK privacy manifest.

### Option 2: Add the framework manually

Download `nosmai.framework.zip` and `SHA256SUMS` from the [iOS SDK v3.0.0 release](https://github.com/nosmai/camera-sdk-ios/releases/tag/v3.0.0).

```sh
shasum -a 256 -c SHA256SUMS
```

After the checksum reports `nosmai.framework.zip: OK`:

1. Unzip `nosmai.framework.zip`.
2. Drag `nosmai.framework` into the Xcode project.
3. Select the application target and open **General**.
4. Add the framework under **Frameworks, Libraries, and Embedded Content**.
5. Set it to **Embed & Sign**.
6. Set the deployment target to iOS 15.0 or later.

The current release is a physical-device ARM64 framework and does not include an iOS Simulator slice. Confirm that the framework appears in the final application target, not only in a test target.

Do not remove `PrivacyInfo.xcprivacy` from the manually embedded framework.

## Permissions

Add permission descriptions before starting the camera:

```xml title="Info.plist"
<key>NSCameraUsageDescription</key>
<string>This app uses the camera for real-time filters and effects.</string>

<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone when recording video or streaming.</string>

<key>NSPhotoLibraryAddUsageDescription</key>
<string>This app saves captured photos and videos to your library.</string>
```

| Permission | Required when |
| --- | --- |
| Camera | Showing a camera preview or capturing a photo |
| Microphone | Recording or streaming with audio |
| Photo library add access | Saving a captured photo or video to Photos |

Do not request microphone or photo library permission unless the application uses the related feature.

## Import the SDK

Use the umbrella header:

```objc
#import <nosmai/Nosmai.h>
```

In Swift:

```swift
import nosmai
```

If the framework module name in a specific release is shown differently by Xcode, use the module name included with that release.

## Initialize

Initialize once before opening the camera experience:

```objc title="AppDelegate.m"
#import <nosmai/Nosmai.h>

- (BOOL)application:(UIApplication *)application
    didFinishLaunchingWithOptions:(NSDictionary *)launchOptions {

    [[NosmaiCore shared]
        initializeWithAPIKey:@"NOSMAI-YOUR-LICENSE-KEY"
                  completion:^(BOOL success, NSError *error) {
        if (!success) {
            NSLog(@"Nosmai initialization failed: %@",
                  error.localizedDescription);
            return;
        }

        NSLog(@"Nosmai is ready");
    }];

    return YES;
}
```

Initialization is asynchronous. Open the camera only after the completion reports success.

Do not initialize the SDK from every view controller. Keep one application-level owner and reuse the shared instance.

For production applications, keep the license key outside committed source code. Use the application configuration or another protected configuration source.

## Display the camera preview

Create a view for the preview and attach the Nosmai camera after initialization:

```objc title="CameraViewController.m"
#import <nosmai/Nosmai.h>

@interface CameraViewController ()
@property(nonatomic, strong) UIView *cameraPreviewView;
@end

@implementation CameraViewController

- (void)viewDidLoad {
    [super viewDidLoad];

    self.cameraPreviewView =
        [[UIView alloc] initWithFrame:self.view.bounds];
    self.cameraPreviewView.autoresizingMask =
        UIViewAutoresizingFlexibleWidth |
        UIViewAutoresizingFlexibleHeight;
    [self.view addSubview:self.cameraPreviewView];
}

- (void)startNosmaiCamera {
    NosmaiCore *core = [NosmaiCore shared];

    if (!core.isInitialized || core.camera == nil) {
        return;
    }

    NosmaiCameraConfig *config = [[NosmaiCameraConfig alloc] init];
    config.position = NosmaiCameraPositionFront;
    config.sessionPreset = AVCaptureSessionPresetHigh;
    config.frameRate = 30;
    config.orientation = NosmaiVideoOrientationPortrait;
    config.enableMirroring = YES;

    [core.camera updateConfiguration:config];
    [core.camera attachToView:self.cameraPreviewView];

    if (![core.camera startCapture]) {
        NSLog(@"Nosmai camera could not start");
    }
}

@end
```

Keep the preview view attached while the camera is active. Replacing the view repeatedly causes camera and graphics resources to be recreated.

## Camera controls

### Switch the camera

```objc
BOOL switched = [[NosmaiCore shared].camera switchCamera];
```

Or choose a specific position:

```objc
[[NosmaiCore shared].camera
    switchToPosition:NosmaiCameraPositionBack];
```

The SDK updates the input orientation and active face effects for the selected camera.

### Set the frame rate

```objc
BOOL accepted = [[NosmaiCore shared].camera setFrameRate:30];
```

The device may reject an unsupported frame rate. Use 30 FPS as the normal starting point and test all supported target devices.

### Focus and exposure

The point uses normalized coordinates from `0.0` to `1.0`:

```objc
CGPoint point = CGPointMake(0.5, 0.5);
[[NosmaiCore shared].camera setFocusPointOfInterest:point];
[[NosmaiCore shared].camera setExposurePointOfInterest:point];
```

Reset to automatic behavior:

```objc
[[NosmaiCore shared].camera resetFocusAndExposure];
```

### Zoom

```objc
NosmaiCamera *camera = [NosmaiCore shared].camera;
CGFloat requestedZoom = 2.0;
CGFloat zoom = MIN(requestedZoom, camera.maxZoomFactor);
[camera rampToZoomFactor:zoom withDuration:0.2];
```

### Torch

Check support before enabling it:

```objc
NosmaiCamera *camera = [NosmaiCore shared].camera;

if (camera.hasTorch) {
    [camera toggleTorch];
}
```

The front camera normally does not provide a hardware torch.

## Apply a `.nosmai` filter

Use the same method for `filter`, `effect`, `beauty_effect`, and `background` packages:

```objc
NSString *path =
    [[NSBundle mainBundle] pathForResource:@"heart_face"
                                   ofType:@"nosmai"
                              inDirectory:@"filters"];

[[NosmaiCore shared].effects
    applyEffect:path
     completion:^(BOOL success, NSError *error) {
        if (!success) {
            NSLog(@"Filter failed: %@", error.localizedDescription);
        }
    }];
```

Always use the asynchronous completion result. A valid path does not guarantee that a package can be loaded successfully.

The SDK reads the package type and places it in the correct active category.

## Adjust effect parameters

Some `.nosmai` packages expose adjustable values such as intensity, animation speed, or display text.

Read parameter metadata:

```objc
NSArray<NSDictionary *> *parameters =
    [[NosmaiCore shared].effects getEffectParameters];
```

Set and read a numeric value:

```objc
BOOL updated =
    [[NosmaiCore shared].effects
        setEffectParameter:@"intensity"
                     value:0.70f];

float intensity =
    [[NosmaiCore shared].effects
        getEffectParameterValue:@"intensity"];
```

Set a text value through the shared SDK:

```objc
BOOL updated =
    [[NosmaiSDK sharedInstance]
        setEffectParameter:@"headerText"
               stringValue:@"Hello"];
```

Use the exact parameter names and types returned by the selected package. Unknown names and incompatible value types return `NO`.

## List local filters

After `NosmaiCore` initialization, the shared `NosmaiSDK` instance provides typed local filter information:

```objc
NosmaiSDK *sdk = [NosmaiSDK sharedInstance];
NSArray<NosmaiFilterInfo *> *filters = [sdk getFilters];

for (NosmaiFilterInfo *filter in filters) {
    NSLog(@"%@, type: %@, path: %@",
          filter.displayName,
          filter.typeKey,
          filter.path);
}
```

List one type only:

```objc
NSArray<NosmaiFilterInfo *> *beautyFilters =
    [[NosmaiSDK sharedInstance]
        getFiltersOfType:NosmaiFilterTypeBeautyEffect];
```

Supported values are:

- `NosmaiFilterTypeFilter`
- `NosmaiFilterTypeEffect`
- `NosmaiFilterTypeBeautyEffect`
- `NosmaiFilterTypeBackground`

Apply a typed item:

```objc
NosmaiFilterInfo *filter = filters.firstObject;

[[NosmaiSDK sharedInstance]
    applyEffectInfo:filter
         completion:^(BOOL success, NSError *error) {
        if (!success) {
            NSLog(@"Apply failed: %@", error.localizedDescription);
        }
    }];
```

Remove that item:

```objc
[[NosmaiSDK sharedInstance] removeEffectInfo:filter];
```

### Development-only filter discovery

During development, filters can be discovered even when external manifest and preview files are not present:

```objc
[[NosmaiSDK sharedInstance]
    getDebugFiltersOfType:NosmaiFilterTypeEffect
               completion:^(NSArray<NosmaiFilterInfo *> *filters,
                            NSError *error) {
        if (error != nil) {
            NSLog(@"Discovery failed: %@", error.localizedDescription);
            return;
        }

        NSLog(@"Found %lu effects", (unsigned long)filters.count);
    }];
```

Use `getFilters` and `getFiltersOfType:` for the production catalog. The debug methods may inspect package metadata and are intended for development tools.

## Built-in beauty

Built-in beauty can remain active with a regular color filter.

### Makeup

```objc
NosmaiEffectsEngine *effects = [NosmaiCore shared].effects;

[effects applyLipstickWithStyle:NosmaiLipstickStyleMatte
                     colorIndex:0];
[effects setLipstickIntensity:0.65f];

[effects applyEyeshadowWithStyle:NosmaiEyeshadowStyleNatural
                      colorIndex:0];
[effects setEyeshadowIntensity:0.35f];

[effects applyBlusherWithStyle:NosmaiBlusherStyleNatural
                    colorIndex:0];
[effects setBlusherIntensity:0.30f];
```

Apply eyelashes and eyebrows:

```objc
[effects applyEyelashWithStyle:NosmaiEyelashStyleNatural];
[effects setEyelashIntensity:0.45f];

[effects applyEyebrowWithStyle:NosmaiEyebrowStyleNatural
                    colorIndex:0];
[effects setEyebrowIntensity:0.35f];
```

Remove one feature:

```objc
[effects removeLipstick];
```

Remove all makeup:

```objc
[effects removeAllMakeup];
```

### Face shaping

```objc
[effects setFaceSlimLevel:0.20f];
[effects setEyeSizeLevel:0.10f];
[effects setNoseSlimLevel:0.10f];
[effects setJawlineLevel:0.15f];
```

Keep default values subtle. Strong values can look unnatural and can expose tracking limits during quick movement.

Remove face shaping:

```objc
[effects removeAllMorphing];
```

Remove makeup, face shaping, and eye coloring together:

```objc
[effects removeAllBeautyEffects];
```

## Background effects

### Blur

```objc
NosmaiBackgroundSegmentationConfig *config =
    [[NosmaiBackgroundSegmentationConfig alloc] init];
config.mode = NosmaiBackgroundSegmentationModeBlur;
config.blurStrength = 50.0f;

[[NosmaiCore shared].effects setBackgroundSegmentation:config];
```

### Solid color

```objc
NosmaiBackgroundSegmentationConfig *config =
    [[NosmaiBackgroundSegmentationConfig alloc] init];
config.mode = NosmaiBackgroundSegmentationModeColor;
config.replacementColor = [UIColor colorWithRed:0.08
                                          green:0.08
                                           blue:0.10
                                          alpha:1.0];

[[NosmaiCore shared].effects setBackgroundSegmentation:config];
```

### Image

```objc
NosmaiBackgroundSegmentationConfig *config =
    [[NosmaiBackgroundSegmentationConfig alloc] init];
config.mode = NosmaiBackgroundSegmentationModeImage;
config.replacementImage = [UIImage imageNamed:@"studio_background"];

[[NosmaiCore shared].effects setBackgroundSegmentation:config];
```

### Video

```objc
NSURL *url =
    [[NSBundle mainBundle] URLForResource:@"background_loop"
                           withExtension:@"mp4"];

NosmaiBackgroundSegmentationConfig *config =
    [[NosmaiBackgroundSegmentationConfig alloc] init];
config.mode = NosmaiBackgroundSegmentationModeVideo;
config.replacementVideoURL = url;

[[NosmaiCore shared].effects setBackgroundSegmentation:config];
```

Clear the active background:

```objc
[[NosmaiCore shared].effects clearBackgroundSegmentation];
```

Background processing is more demanding than a basic color filter. Test it together with recording or streaming on the lowest supported device.

## Cloud filters

Request the cloud catalog:

```objc
NosmaiCloudFilterRequestOptions *options =
    [NosmaiCloudFilterRequestOptions defaultOptions];
options.page = 1;
options.limit = 20;
options.version = NosmaiCloudFilterVersion2;
options.filterType = @"beauty_effect";

[[NosmaiCore shared].effects
    getCloudFiltersWithOptions:options
                    completion:^(NSArray<NSDictionary *> *filters,
                                 NosmaiCloudFilterPaginationInfo *pagination,
                                 NSError *error) {
        if (error != nil) {
            NSLog(@"Cloud catalog failed: %@", error.localizedDescription);
            return;
        }

        NSLog(@"Loaded %lu filters", (unsigned long)filters.count);
    }];
```

Valid cloud categories are:

- `effects`
- `filter`
- `bg`
- `beauty_effect`

Download and apply a selected item:

```objc
NSString *filterId = @"CLOUD-FILTER-ID";

[[NosmaiCore shared].effects
    downloadCloudFilter:filterId
               progress:^(float progress) {
        NSLog(@"Download: %.0f%%", progress * 100.0f);
    }
             completion:^(BOOL success,
                          NSString *localPath,
                          NSError *error) {
        if (!success) {
            NSLog(@"Download failed: %@", error.localizedDescription);
            return;
        }

        [[NosmaiCore shared].effects
            applyEffect:localPath
             completion:nil];
    }];
```

Use the version value supported by the installed SDK. The current public version constant is `NosmaiCloudFilterVersion2`.

## Observe active state

The SDK posts one notification whenever a filter, effect, beauty feature, or background changes:

```objc
[[NSNotificationCenter defaultCenter]
    addObserver:self
       selector:@selector(nosmaiStateChanged:)
           name:NosmaiPipelineStateDidChangeNotification
         object:nil];
```

Read the complete state:

```objc
- (void)nosmaiStateChanged:(NSNotification *)notification {
    NosmaiPipelineState *state =
        notification.userInfo[NosmaiPipelineStateUserInfoKey];

    BOOL hasFilter = state.activeFilterPath != nil;
    BOOL hasEffect = state.activeEffectPath != nil;
    BOOL hasBackground = state.backgroundActive;

    NSLog(@"filter=%d effect=%d background=%d",
          hasFilter,
          hasEffect,
          hasBackground);
}
```

Remove the observer when the owning object is released if the application supports an iOS version or observer style that requires manual removal.

## Capture a photo

Capture the processed image:

```objc
[[NosmaiCore shared]
    capturePhoto:^(UIImage *image, NSError *error) {
        if (image == nil) {
            NSLog(@"Photo capture failed: %@",
                  error.localizedDescription);
            return;
        }

        UIImageWriteToSavedPhotosAlbum(
            image,
            nil,
            nil,
            nil
        );
    }];
```

Request photo library access before saving to Photos. Capturing into memory only requires camera access.

## Record video

Start recording:

```objc
[[NosmaiCore shared]
    startRecordingWithCompletion:^(BOOL success, NSError *error) {
        if (!success) {
            NSLog(@"Recording could not start: %@",
                  error.localizedDescription);
        }
    }];
```

Stop recording and receive the temporary file URL:

```objc
[[NosmaiCore shared]
    stopRecordingWithCompletion:^(NSURL *videoURL, NSError *error) {
        if (videoURL == nil) {
            NSLog(@"Recording failed: %@",
                  error.localizedDescription);
            return;
        }

        NSLog(@"Recorded video: %@", videoURL.path);
    }];
```

The application decides whether to save, upload, move, or delete the returned file.

## Lifecycle

### When the application enters the background

```objc
- (void)applicationDidEnterBackground:(UIApplication *)application {
    [[NosmaiCore shared] pause];
}
```

### When the application becomes active

```objc
- (void)applicationDidBecomeActive:(UIApplication *)application {
    [[NosmaiCore shared] resume];
}
```

### When leaving the camera screen

Stop capture and detach the preview:

```objc
- (void)viewDidDisappear:(BOOL)animated {
    [super viewDidDisappear:animated];

    NosmaiCamera *camera = [NosmaiCore shared].camera;
    [camera stopCapture];
    [camera detachFromView];
}
```

When returning, attach the new preview view and call `startCapture` again.

### Full cleanup

```objc
[[NosmaiCore shared] cleanup];
```

Use full cleanup only when the application is finished with the SDK and accepts that initialization will be required again. Do not call it for a short navigation transition.

## Clear active features

Choose the narrowest method that matches the user action:

```objc
NosmaiEffectsEngine *effects = [NosmaiCore shared].effects;

[effects clearFilter];
[effects clearAREffect];
[effects clearBackgroundSegmentation];
[effects removeAllBeautyEffects];
```

Reset every active visual feature:

```objc
[[NosmaiCore shared].effects clearAll];
```

## Release checklist

Before distributing the application:

1. Use a license key issued for the final bundle identifier.
2. Test first launch with network access.
3. Test front and back camera switching.
4. Test leaving and returning to the camera screen.
5. Test background and foreground transitions.
6. Test camera and microphone permission denial.
7. Test local and cloud filter failure handling.
8. Test recording while beauty or background effects are active.
9. Test on the oldest and slowest supported physical device.
10. Disable SDK debug logging in the release build:

```objc
[[NosmaiCore shared] setDebugLoggingEnabled:NO];
```

## Common issues

### Initialization fails

Confirm that:

- the device has internet access
- the license key is complete
- the bundle identifier matches the Nosmai Console project
- the SDK is not already initialized with a different key

### The preview is blank

Confirm that:

- camera permission was granted
- initialization completed successfully
- the preview view has a non-zero size
- `attachToView:` was called before `startCapture`
- no other camera session is using the device camera

### A filter does not appear

Confirm that:

- the path points to an existing `.nosmai` file
- the apply completion reports success
- the package was built for the installed SDK version
- the license includes the required feature

### The camera does not return after navigation

Stop capture and detach the old preview when leaving. Attach the new preview and start capture when returning. Do not keep an old hidden preview view attached.

### Performance drops

Test one feature at a time. Background replacement, face-tracked makeup, recording, and streaming each add work. Use 30 FPS as the starting target and avoid stacking features that are not visible.

---

# Android
Source: https://docs.nosmai.com/docs/effects/android/

## Overview

The Android SDK provides public APIs for:

- SDK initialization and license handling
- real-time processed preview
- front and back camera input
- built-in beauty, makeup, and face shaping
- local and cloud `.nosmai` filters
- active-effect state
- background effects
- processed video recording
- processed output for live streaming

The official Android repository contains releases and a complete sample application:

[github.com/nosmai/camera-sdk-android](https://github.com/nosmai/camera-sdk-android)

## Requirements

| Requirement | Value |
| --- | --- |
| Minimum Android version | API 21 |
| Supported device architecture | `arm64-v8a` |
| Native page size | 16 KB compatible |
| Java compatibility | Java 11 |
| Camera API | Camera2 |
| Test environment | Physical Android device |

The SDK does not run on an x86 or x86_64 emulator. Use a physical arm64 Android device for camera, face tracking, recording, and performance testing.

## Install

### 1. Download the AAR

Download the current AAR from the [Android SDK releases](https://github.com/nosmai/camera-sdk-android/releases).

For SDK `3.0.0`, download both `nosmai-sdk-3.0.0.aar` and `SHA256SUMS`, then verify the artifact:

```sh
shasum -a 256 -c SHA256SUMS
```

Place it in the application module:

```text
app/
  libs/
    nosmai-sdk-3.0.0.aar
```

### 2. Add the dependency

```kotlin title="app/build.gradle.kts"
android {
    compileSdk = 35

    defaultConfig {
        minSdk = 21

        ndk {
            abiFilters += "arm64-v8a"
        }
    }

    compileOptions {
        sourceCompatibility = JavaVersion.VERSION_11
        targetCompatibility = JavaVersion.VERSION_11
    }
}

dependencies {
    implementation(files("libs/nosmai-sdk-3.0.0.aar"))
}
```

If the downloaded release uses a different filename, use that exact filename in the Gradle dependency.

### 3. Prevent duplicate native libraries

If another dependency includes the same shared C++ runtime, add:

```kotlin title="app/build.gradle.kts"
android {
    packaging {
        jniLibs {
            pickFirsts += "lib/arm64-v8a/libc++_shared.so"
        }
    }
}
```

Only add a `pickFirst` rule when Gradle reports a duplicate native-library error. Do not add broad packaging exclusions.

## Permissions

Add the required permissions:

```xml title="app/src/main/AndroidManifest.xml"
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />

    <uses-permission android:name="android.permission.RECORD_AUDIO" />

    <application
        android:name=".NosmaiApplication"
        ...>
    </application>
</manifest>
```

| Permission | Required when |
| --- | --- |
| `CAMERA` | Showing a camera preview |
| `INTERNET` | License verification and cloud filters |
| `RECORD_AUDIO` | Recording or streaming with microphone audio |

Request camera and microphone permission at runtime. Do not request microphone permission if the application never records or streams audio.

## Initialize

Initialize Nosmai once from the application class.

```java title="NosmaiApplication.java"
package com.example.cameraapp;

import android.app.Application;
import com.nosmai.effect.api.NosmaiSDK;

public final class NosmaiApplication extends Application {
    @Override
    public void onCreate() {
        super.onCreate();

        NosmaiSDK.initialize(
            this,
            BuildConfig.NOSMAI_LICENSE_KEY
        );
    }
}
```

Register the class:

```xml title="AndroidManifest.xml"
<application
    android:name=".NosmaiApplication"
    ...>
</application>
```

Check the state when needed:

```java
if (!NosmaiSDK.isInitialized()) {
    // Do not open the camera experience yet.
}
```

Do not initialize from every activity, fragment, or composable. Keep initialization under one application-level owner.

## Add the preview

Create a container:

```xml title="res/layout/activity_camera.xml"
<?xml version="1.0" encoding="utf-8"?>
<FrameLayout
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/preview_container"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />
```

Create `NosmaiPreviewView` after camera permission is granted:

```java title="CameraActivity.java"
private NosmaiPreviewView previewView;

private void createNosmaiPreview() {
    FrameLayout container = findViewById(R.id.preview_container);

    previewView = new NosmaiPreviewView(this);
    container.addView(
        previewView,
        new FrameLayout.LayoutParams(
            FrameLayout.LayoutParams.MATCH_PARENT,
            FrameLayout.LayoutParams.MATCH_PARENT
        )
    );

    NosmaiSDK.startProcessing(previewView);
}
```

`startProcessing` prepares the view for camera frames and effects. It does not grant camera permission.

## Connect Camera2

The native Android SDK accepts Camera2 input from the application. The official sample includes a `Camera2Helper` that handles camera selection, frame size, sensor orientation, camera close, and background-thread ownership.

Use the helper from the official sample or provide equivalent Camera2 code in the host application.

The required application responsibilities are:

1. create one Camera2 source
2. connect it to `NosmaiPreviewView`
3. tell the SDK whether the front or back camera is active
4. provide the camera orientation
5. start the camera after the Nosmai preview is ready
6. stop the camera before removing the screen

### Recommended direct camera connection

The sample application uses the optimized direct camera connection when the device supports it. The public Nosmai setup is:

```java
previewView.enableOesInput(true);

previewView.addOnOesReadyListener(surfaceTexture -> {
    cameraHelper.setInputMode(Camera2Helper.InputMode.OES);
    cameraHelper.setOesPreviewSurfaceTexture(surfaceTexture);

    cameraHelper.setPreviewSizeCallback((width, height) -> {
        int rotation = calculateFrameRotation();
        previewView.setOesInputFrameInfo(width, height, rotation);
    });

    previewView.setCameraOrientation(
        cameraHelper.isFrontCamera(),
        cameraHelper.getSensorOrientation()
    );

    NosmaiSDK.setCameraFacing(cameraHelper.isFrontCamera());
    cameraHelper.startCamera();
});
```

`Camera2Helper` and `calculateFrameRotation` in this example come from the official sample application, not from the public SDK AAR.

Keep the sample's device fallback behavior. If direct input cannot start on a device, reconnect using the sample's normal Camera2 frame input instead of leaving the preview black.

## Front-camera mirroring

Tell the SDK which camera is active:

```java
NosmaiSDK.setCameraFacing(isFrontCamera);
```

Set the preview mirror preference:

```java
NosmaiSDK.setMirrorX(isFrontCamera);
```

Back-camera preview should normally not be mirrored.

Do not mirror the Camera2 frame and the Nosmai preview at the same time. Applying the same horizontal flip twice can make the preview look unchanged while face effects use the wrong direction.

## Switch camera

Stop or reconfigure the current Camera2 session before opening the other camera.

```java
private boolean isSwitchingCamera;

private void switchCamera() {
    if (isSwitchingCamera) {
        return;
    }

    isSwitchingCamera = true;

    cameraHelper.stopCamera();
    isFrontCamera = !isFrontCamera;

    setupCameraHelper(isFrontCamera);

    NosmaiSDK.setCameraFacing(isFrontCamera);
    NosmaiSDK.setMirrorX(isFrontCamera);

    cameraHelper.startCamera();
    isSwitchingCamera = false;
}
```

Prevent rapid repeated taps while a switch is already running.

## Apply built-in beauty

Use `NosmaiBeauty` for built-in controls.

```java
import com.nosmai.effect.api.NosmaiBeauty;

NosmaiBeauty.applySkinSmoothing(0.4f);
NosmaiBeauty.applySkinWhitening(0.2f);
NosmaiBeauty.applySharpen(0.15f);
NosmaiBeauty.applyTeethWhitening(0.3f);
```

Skin smoothing, whitening, sharpening, and teeth whitening use values from `0.0` to `1.0`.

Remove built-in beauty and makeup:

```java
NosmaiBeauty.clearAllBeautyFilters();
```

## Apply makeup

```java
NosmaiBeauty.applyLipstickStyle(
    NosmaiBeauty.LIPSTICK_MATTE,
    0.62f,
    0.12f,
    0.18f
);

NosmaiBeauty.setMakeupIntensity(
    NosmaiBeauty.MAKEUP_LIPSTICK,
    0.65f
);
```

Remove one makeup category:

```java
NosmaiBeauty.removeMakeup(NosmaiBeauty.MAKEUP_LIPSTICK);
```

Remove all built-in makeup:

```java
NosmaiBeauty.clearBuiltInMakeup();
```

## Apply a local `.nosmai` package

Use `applyEffect` for `filter`, `effect`, `beauty_effect`, and `background` packages:

```java
NosmaiEffects.applyEffect(
    filterPath,
    new NosmaiEffects.EffectCallback() {
        @Override
        public void onSuccess() {
            // Update selected state after success.
        }

        @Override
        public void onError(String errorMessage) {
            // Present or record the error.
        }
    }
);
```

Applying a package is asynchronous. Do not mark a filter as selected before `onSuccess`.

Remove an external package:

```java
NosmaiEffects.removeEffect(filterInfo);
```

Use the typed `NosmaiFilterInfo` when available so the SDK can remove the correct category.

## Adjust effect parameters

Some `.nosmai` packages expose adjustable numeric or text values.

Set a numeric value:

```java
boolean updated = NosmaiEffects.setEffectParameter(
    "intensity",
    0.7f
);
```

Read a numeric value:

```java
float intensity =
    NosmaiEffects.getEffectParameterValue("intensity");
```

Set a text value:

```java
boolean updated = NosmaiEffects.setEffectParameter(
    "headerText",
    "Hello"
);
```

Use the exact parameter names and types authored in the selected package. Unknown names and incompatible value types return `false`.

## List local filters

Production filters use a `.nosmai` file, external manifest, and preview image.

```java
List<NosmaiFilterInfo> all = NosmaiEffects.getFilters();

List<NosmaiFilterInfo> effects = NosmaiEffects.getFilters(
    NosmaiFilterInfo.Type.EFFECT
);
```

Development-only loose packages can be scanned asynchronously:

```java
NosmaiEffects.getDebugFilters(
    NosmaiFilterInfo.Type.FILTER,
    (filters, error) -> {
        if (error != null) {
            return;
        }

        // Show development filters.
    }
);
```

Do not use debug discovery as the production catalog.

## Observe active state

Listen for state changes instead of guessing which item remained active:

```java
NosmaiEffects.PipelineStateListener listener = state -> {
    NosmaiFilterInfo activeFilter = NosmaiEffects.getActiveFilterInfo();
    NosmaiFilterInfo activeEffect = NosmaiEffects.getActiveEffectInfo();

    // Update the application's selected buttons.
};

NosmaiEffects.addPipelineStateListener(listener);
```

Remove the listener when its screen or controller is destroyed:

```java
NosmaiEffects.removePipelineStateListener(listener);
```

## Background effects

Apply a manual background configuration through `NosmaiEffects`:

```java
NosmaiBackgroundSegmentationConfig config =
    new NosmaiBackgroundSegmentationConfig();

config.mode = NosmaiBackgroundSegmentationConfig.Mode.BLUR;
config.blurStrength = 55.0f;

NosmaiEffects.setBackgroundSegmentation(config);
```

Clear it:

```java
NosmaiEffects.clearBackgroundSegmentation();
```

Use the exact constructors and mode names from the installed SDK version. Background configuration classes can gain new modes between releases.

## Cloud filters

Use `NosmaiCloud.cachedList()` to paint an existing catalog immediately. Run `NosmaiCloud.fetch(...)` away from the main thread so opening a cloud-filter sheet never blocks the camera interface.

```java
List<NosmaiCloud.Item> cached = NosmaiCloud.cachedList();

NosmaiCloud.FilterQuery query = new NosmaiCloud.FilterQuery();
query.filterType = "effects";
query.page = 1;
query.limit = 20;
query.fetchAllPages = false;
query.cleanupRemoved = false;

cloudExecutor.execute(() -> {
    boolean success = NosmaiCloud.fetch(query);
    List<NosmaiCloud.Item> items = success
            ? NosmaiCloud.list()
            : Collections.emptyList();

    mainHandler.post(() -> {
        // Confirm this screen or request still owns the result.
        // Then render items or show Retry.
    });
});
```

Download the selected identifier and apply only the returned local path:

```java
NosmaiCloud.download(
    item.id,
    (filterId, success, localPath, error) -> {
        if (success && localPath != null && !localPath.isEmpty()) {
            NosmaiEffects.applyEffect(
                localPath,
                new NosmaiEffects.EffectCallback() {
                    @Override
                    public void onSuccess() {
                        // Confirm selection from the pipeline listener.
                    }

                    @Override
                    public void onError(String message) {
                        // Keep the previous selection and offer Retry.
                    }
                }
            );
        }
    }
);
```

Deduplicate repeated taps by cloud identifier and ignore UI updates after the owning sheet closes. See [Cloud filters](/docs/effects/cloud-filters) for category values, pagination, progress callbacks, cache behavior, and complete edge-case handling.

## Record processed video

Create an output path in application storage, then start recording:

```java
File output = new File(
    getExternalFilesDir(null),
    "nosmai_" + System.currentTimeMillis() + ".mp4"
);

NosmaiSDK.startRecording(
    previewView,
    output.getAbsolutePath(),
    new NosmaiSDK.RecordingCallback() {
        @Override
        public void onStarted(boolean success, String error) {
            if (!success) {
                // Recording did not start.
            }
        }
    }
);
```

Stop and wait for finalization:

```java
NosmaiSDK.stopRecording(new NosmaiSDK.RecordingCallback() {
    @Override
    public void onCompleted(
        String outputPath,
        boolean success,
        String error
    ) {
        if (success) {
            // The MP4 is finalized at outputPath.
        }
    }
});
```

Do not leave the camera screen while recording is still being finalized.

Native Android photo capture is owned by the application's camera implementation in the current SDK. Flutter exposes a higher-level `capturePhoto` method through its platform view.

## Lifecycle

Use the following ownership:

| Event | Action |
| --- | --- |
| Camera screen opens | Create preview, start Nosmai processing, then start Camera2 |
| App goes to background | Stop Camera2 and pause the preview view |
| App returns | Resume the preview view and reconnect Camera2 |
| Camera screen closes | Stop Camera2, stop processing, remove the preview |
| Application truly finishes SDK use | Call full cleanup |

Example:

```java
@Override
protected void onResume() {
    super.onResume();

    if (previewView != null) {
        previewView.onResume();
    }

    if (hasCameraPermission()) {
        cameraHelper.startCamera();
    }
}

@Override
protected void onPause() {
    if (cameraHelper != null) {
        cameraHelper.stopCamera();
    }

    if (previewView != null) {
        previewView.onPause();
    }

    super.onPause();
}

@Override
protected void onDestroy() {
    if (cameraHelper != null) {
        cameraHelper.stopCamera();
    }

    NosmaiSDK.stopProcessing();

    if (previewView != null && previewView.getParent() != null) {
        ((ViewGroup) previewView.getParent()).removeView(previewView);
    }

    previewView = null;
    super.onDestroy();
}
```

Do not call `NosmaiSDK.cleanup()` every time a camera activity briefly pauses. Full cleanup requires a later initialization before the SDK can be used again.

## Release configuration

The AAR includes consumer rules for its public SDK classes. If release-only failures occur, confirm:

- the current AAR is used
- consumer ProGuard rules are merged
- JNI methods have not been renamed or removed
- the final APK or AAB contains `lib/arm64-v8a/libnosmai.so`
- all protected SDK assets are present

Distribute an Android App Bundle when publishing to Google Play. Devices without the supported architecture will not receive an incompatible split.

## Common problems

| Problem | What to check |
| --- | --- |
| `UnsatisfiedLinkError` | AAR version, Java API version, native library packaging, and ABI |
| Black preview | Runtime permission, camera source, preview readiness, and activity lifecycle |
| Preview stays black after camera switch | Stop the old Camera2 session completely before opening the new one |
| Camera indicator stays on | Stop Camera2 when leaving the screen |
| Effect appears reversed | Camera-facing value and single mirror owner |
| Filter apply crashes in release only | Consumer ProGuard rules and matching AAR classes/native library |
| Recording does not start | Preview readiness, output path, storage, and current recording state |
| Effect selection UI is wrong | Update it from apply callbacks and active-state listener |
| Emulator cannot install | Use an arm64 physical device |

## Production checklist

- initialize once at application level
- request camera permission before starting Camera2
- use one camera source per preview
- stop the old camera before switching
- stop Camera2 before removing the screen
- apply external packages asynchronously
- remove active-state listeners
- finalize recording before navigation
- test front and back camera orientation
- test background and foreground transitions
- test repeated camera-screen navigation
- test rapid filter switching
- test a low-end and high-end arm64 device
- test a long recording or streaming session

---

# Flutter
Source: https://docs.nosmai.com/docs/effects/flutter/

## Overview

The Flutter package provides one Dart API for the Nosmai Android and iOS SDKs.

It supports:

- native processed camera preview
- front and back camera switching
- built-in beauty and makeup
- local and cloud `.nosmai` filters
- active-effect state
- background color, image, and video replacement
- photo capture
- processed video recording
- gallery save
- native error and license events

The official Flutter repository contains the package, example application, releases, and issue tracker:

[github.com/nosmai/nosmai_camera_sdk_flutter](https://github.com/nosmai/nosmai_camera_sdk_flutter)

## Requirements

| Requirement | Value |
| --- | --- |
| Flutter | 3.22.0 or later |
| Dart | 3.0.0 or later |
| Android | API 21 or later, arm64-v8a device |
| iOS | iOS 15.0 or later, arm64 device |
| Test environment | Physical Android or iOS device |

Use a physical device for camera preview, face tracking, recording, and performance testing.

## Install

Add the current package:

```yaml title="pubspec.yaml"
dependencies:
  flutter:
    sdk: flutter
  nosmai_camera_sdk: ^3.0.6
```

Run:

```sh
flutter pub get
```

The Flutter package resolves its native iOS dependency through CocoaPods. The proprietary Android AAR is distributed separately and is intentionally not included in the pub.dev package.

## Configure Android

### Add the native Android SDK

1. Download `nosmai-sdk-3.0.0.aar` and `SHA256SUMS` from the [Android SDK v3.0.0 release](https://github.com/nosmai/camera-sdk-android/releases/tag/v3.0.0).
2. Verify the artifact with `shasum -a 256 -c SHA256SUMS`.
3. Rename the AAR to `nosmai-release.aar`.
4. Place it at `android/app/libs/nosmai-release.aar`.

Add the local repository to Groovy projects:

```gradle title="android/build.gradle"
allprojects {
    repositories {
        google()
        mavenCentral()
        flatDir { dirs "${rootProject.projectDir}/app/libs" }
    }
}
```

Kotlin DSL projects use:

```kotlin title="android/build.gradle.kts"
allprojects {
    repositories {
        google()
        mavenCentral()
        flatDir { dirs("${rootProject.projectDir}/app/libs") }
    }
}
```

Add the AAR to the Flutter application module:

```gradle title="android/app/build.gradle"
dependencies {
    implementation files('libs/nosmai-release.aar')
}
```

Kotlin DSL application modules use:

```kotlin title="android/app/build.gradle.kts"
dependencies {
    implementation(files("libs/nosmai-release.aar"))
}
```

Do not place the AAR inside the Flutter package cache or plugin source. The consuming application owns this commercial binary.

### Minimum version

Confirm the application uses API 21 or later:

```kotlin title="android/app/build.gradle.kts"
android {
    defaultConfig {
        minSdk = 21

        ndk {
            abiFilters += "arm64-v8a"
        }
    }
}
```

### Permissions

Add the permissions used by the application:

```xml title="android/app/src/main/AndroidManifest.xml"
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.INTERNET" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />

    <application
        android:label="Camera App"
        android:name="${applicationName}"
        android:icon="@mipmap/ic_launcher">
        ...
    </application>
</manifest>
```

| Permission | Required when |
| --- | --- |
| `CAMERA` | Showing the camera preview |
| `INTERNET` | License verification and cloud filters |
| `RECORD_AUDIO` | Recording or streaming with microphone audio |

Request camera and microphone permission at runtime. Do not request microphone permission if the application does not record or stream audio.

## Configure iOS

The plugin installs `NosmaiCameraSDK` through CocoaPods. Do not copy `nosmai.framework` into the Flutter application manually.

The plugin and native framework supply their SDK privacy manifests. The consuming application remains responsible for declaring its own data collection and required-reason API usage accurately in App Store Connect and any app-level privacy manifest.

### Minimum version

Set iOS 15.0 or later:

```ruby title="ios/Podfile"
platform :ios, '15.0'

target 'Runner' do
  use_frameworks! :linkage => :static
  use_modular_headers!

  flutter_install_all_ios_pods(
    File.dirname(File.realpath(__FILE__))
  )
end
```

After changing native dependencies, run:

```sh
cd ios
pod install --repo-update
```

### Permissions

Add the permission descriptions used by the application:

```xml title="ios/Runner/Info.plist"
<key>NSCameraUsageDescription</key>
<string>This app uses the camera for real-time filters and effects.</string>

<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone when recording video or streaming.</string>

<key>NSPhotoLibraryUsageDescription</key>
<string>This app accesses the photo library when selecting or saving media.</string>

<key>NSPhotoLibraryAddUsageDescription</key>
<string>This app saves captured photos and videos to the photo library.</string>
```

Camera permission is required for preview. Microphone and photo library permissions are only required when the related features are used.

## Initialize

Initialize once before showing `NosmaiCameraPreview`:

```dart title="lib/main.dart"
import 'package:flutter/material.dart';
import 'package:nosmai_camera_sdk/nosmai_camera_sdk.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  final initialized = await NosmaiFlutter.initialize(
    'NOSMAI-YOUR-LICENSE-KEY',
  );

  if (!initialized) {
    throw StateError('Nosmai initialization failed');
  }

  runApp(const CameraApp());
}
```

Do not initialize in a widget `build` method. Use one application-level initialization and reuse `NosmaiFlutter.instance`.

For production applications, read the license key from application configuration instead of committing it in source code.

## Display the camera preview

`NosmaiCameraPreview` creates the native camera view for the current platform:

```dart title="lib/camera_screen.dart"
import 'package:flutter/material.dart';
import 'package:nosmai_camera_sdk/nosmai_camera_sdk.dart';

class CameraScreen extends StatefulWidget {
  const CameraScreen({super.key});

  @override
  State<CameraScreen> createState() => _CameraScreenState();
}

class _CameraScreenState extends State<CameraScreen> {
  bool _cameraReady = false;
  String? _cameraError;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Stack(
        fit: StackFit.expand,
        children: [
          NosmaiCameraPreview(
            onInitialized: () {
              if (!mounted) return;
              setState(() {
                _cameraReady = true;
                _cameraError = null;
              });
            },
            onError: (error) {
              if (!mounted) return;
              setState(() {
                _cameraReady = false;
                _cameraError = error;
              });
            },
          ),
          if (!_cameraReady && _cameraError == null)
            const Center(child: CircularProgressIndicator()),
          if (_cameraError != null)
            Center(child: Text(_cameraError!)),
        ],
      ),
    );
  }
}
```

The widget starts native processing when the preview is ready. Do not call `startProcessing` for the same preview unless building a custom camera flow.

Keep one `NosmaiCameraPreview` active for a camera screen. Multiple active native preview widgets can compete for the same camera.

## Camera controls

### Switch the camera

```dart
final nosmai = NosmaiFlutter.instance;
final switched = await nosmai.switchCamera();
```

The method ignores rapid repeated taps while a camera switch is already running. Disable the switch button until the returned future completes.

### Configure the starting camera

For a custom camera flow:

```dart
await NosmaiFlutter.instance.configureCamera(
  position: NosmaiCameraPosition.front,
);
```

The standard preview already starts with the native default configuration.

### Flash and torch

Check support before showing the control:

```dart
final nosmai = NosmaiFlutter.instance;

if (await nosmai.hasTorch()) {
  await nosmai.setTorchMode(NosmaiTorchMode.on);
}
```

The front camera normally does not have a hardware torch.

## Built-in beauty

Apply built-in beauty after the preview reports that it is ready:

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.applySkinSmoothing(0.35);
await nosmai.applySkinWhitening(0.10);
await nosmai.applySharpening(0.15);
await nosmai.applyTeethWhitening(0.20);
```

Use subtle defaults. Strong values can look unnatural and can make small tracking movement more visible.

Remove built-in color and skin controls:

```dart
await nosmai.removeBuiltInFilters();
```

## Makeup

### Lipstick

```dart
await NosmaiFlutter.instance.applyLipstick(
  style: NosmaiLipstickStyle.matte,
  intensity: 0.65,
);
```

Update or remove it:

```dart
await NosmaiFlutter.instance.setLipstickIntensity(0.45);
await NosmaiFlutter.instance.removeLipstick();
```

### Eyeshadow, blusher, eyelashes, and eyebrows

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.applyEyeshadow(
  style: NosmaiEyeshadowStyle.natural,
  intensity: 0.30,
);

await nosmai.applyBlusher(
  style: NosmaiBlusherStyle.natural,
  intensity: 0.25,
);

await nosmai.applyEyelash(
  style: NosmaiEyelashStyle.natural,
  intensity: 0.40,
);

await nosmai.applyEyebrow(
  style: NosmaiEyebrowStyle.natural,
  intensity: 0.30,
);
```

Remove all makeup:

```dart
await nosmai.removeAllMakeup();
```

## Face shaping

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.setFaceSlimLevel(0.20);
await nosmai.setEyeSizeLevel(0.10);
await nosmai.setNoseSlimLevel(0.10);
```

Remove face shaping:

```dart
await nosmai.removeAllMorphing();
```

Remove all makeup, face shaping, and eye coloring:

```dart
await nosmai.removeAllBeautyEffects();
```

## Color controls

Examples:

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.applyBrightnessFilter(0.05);
await nosmai.applyContrastFilter(1.05);
await nosmai.applyRGBFilter(
  red: 1.02,
  green: 1.00,
  blue: 0.98,
);
await nosmai.applyWhiteBalance(
  temperature: 6500,
  tint: 0,
);
```

Use the documented range of each method. The same value does not represent the same strength for every control.

## Add local filters

Production filters use one folder per filter:

```text
assets/
  nosmai_filters/
    glam_lips/
      glam_lips.nosmai
      glam_lips_manifest.json
      glam_lips_preview.png
```

Declare each folder:

```yaml title="pubspec.yaml"
flutter:
  assets:
    - assets/nosmai_filters/glam_lips/
    - assets/nosmai_filters/soft_blush/
```

Each production entry must contain:

1. one `.nosmai` package
2. one external manifest JSON file
3. one preview image

The external files allow the application to show names, types, and previews without opening the protected package.

## List local filters

Get all production filters:

```dart
final filters =
    await NosmaiFlutter.instance.getLocalFilters();
```

Get them grouped by package type:

```dart
final grouped =
    await NosmaiFlutter.instance.getAllLocalFilters();

final regularFilters = grouped['filter'] ?? [];
final effects = grouped['effect'] ?? [];
final beautyEffects = grouped['beauty_effect'] ?? [];
final backgrounds = grouped['background'] ?? [];
```

Or request one type directly:

```dart
final nosmai = NosmaiFlutter.instance;

final effects = await nosmai.getLocalEffects();
final beauty = await nosmai.getLocalBeautyEffects();
final backgrounds = await nosmai.getLocalBackgrounds();
```

Force a refresh after adding or downloading files:

```dart
final filters = await NosmaiFlutter.instance.getLocalFilters(
  forceRefresh: true,
);
```

## Development-only filter discovery

For development, loose `.nosmai` files can be placed in one asset directory:

```text
assets/
  filters/
    effect_one.nosmai
    effect_two.nosmai
```

Declare the directory:

```yaml title="pubspec.yaml"
flutter:
  assets:
    - assets/filters/
```

Get every development filter:

```dart
final filters =
    await NosmaiFlutter.instance.getDebugFilters();
```

Get one type:

```dart
final effects = await NosmaiFlutter.instance.getDebugFilters(
  type: NosmaiLocalFilterType.effect,
);
```

Pass no `type` to get every supported package type. Use the manifest-based local methods for a production filter catalog.

## Apply and remove filters

Use `applyEffect` for all `.nosmai` package types:

```dart
final NosmaiFilter filter = filters.first;
final applied =
    await NosmaiFlutter.instance.applyEffect(filter.path);
```

The SDK reads the package type and applies it in the correct active category.

Remove the selected package without clearing unrelated active features:

```dart
await NosmaiFlutter.instance.removeEffect(filter);
```

Use the narrow clearing methods when the application has separate controls:

```dart
final nosmai = NosmaiFlutter.instance;

await nosmai.clearFilter();
await nosmai.clearAREffect();
await nosmai.clearBackgroundSegmentation();
await nosmai.removeAllBeautyEffects();
```

Clear every active visual feature:

```dart
await nosmai.clearAll();
```

## Adjust effect parameters

Some `.nosmai` packages expose adjustable values such as intensity, animation speed, or display text.

Read the parameters provided by the active package:

```dart
final nosmai = NosmaiFlutter.instance;
final parameters = await nosmai.getEffectParameters();

for (final parameter in parameters) {
  final name = parameter.name;
  final type = parameter.type;
  final currentValue = parameter.currentValue;

  // Build controls supported by this package.
}
```

Set a numeric value:

```dart
final updated = await NosmaiFlutter.instance.setEffectParameter(
  'intensity',
  0.70,
);
```

Read its current value:

```dart
final intensity =
    await NosmaiFlutter.instance.getEffectParameterValue(
  'intensity',
);
```

Set a text value:

```dart
final updated =
    await NosmaiFlutter.instance.setEffectParameterString(
  'headerText',
  'Hello',
);
```

Use the exact parameter names and types returned by the active package. A package can reject unknown names, unsupported types, or values outside its authored range.

## Observe active filters

Read the current state when a screen opens:

```dart
final state =
    await NosmaiFlutter.instance.getActiveEffects();

final selectedFilter = state.activeFilter;
final selectedEffect = state.activeEffect;
final backgroundActive = state.hasBackground;
```

Listen for later changes:

```dart
import 'dart:async';

StreamSubscription<NosmaiActiveEffects>? activeEffectsSubscription;

void observeNosmaiState() {
  activeEffectsSubscription =
      NosmaiFlutter.instance.onActiveEffectsChanged.listen((state) {
    final activeFilterPath = state.activeFilterPath;
    final activeEffectPath = state.activeEffectPath;

    // Update the selected filter controls.
  });
}
```

Cancel the subscription when its owner is disposed:

```dart
await activeEffectsSubscription?.cancel();
```

Use this state instead of assuming that a button tap always leaves the requested package active. A new package in the same category can replace the previous one.

## Background replacement

### Solid color

```dart
await NosmaiFlutter.instance.setBackgroundSegmentation(
  NosmaiBackgroundSegmentationConfig.color(
    const Color(0xFF202124),
  ),
);
```

### Image

Load image bytes and pass them to the SDK:

```dart
import 'dart:typed_data';
import 'package:flutter/services.dart';

final data =
    await rootBundle.load('assets/backgrounds/studio.jpg');
final bytes = Uint8List.sublistView(data);

await NosmaiFlutter.instance.setBackgroundSegmentation(
  NosmaiBackgroundSegmentationConfig.image(bytes),
);
```

### Video

```dart
await NosmaiFlutter.instance.setBackgroundSegmentation(
  NosmaiBackgroundSegmentationConfig.video(
    '/absolute/path/background.mp4',
  ),
);
```

Clear it:

```dart
await NosmaiFlutter.instance.clearBackgroundSegmentation();
```

Background replacement requires more device work than a color filter. Test it together with recording or streaming on the lowest supported device.

## Cloud filters

Get the complete catalog with the default compatibility version:

```dart
final filters =
    await NosmaiFlutter.instance.getCloudFilters();
```

Get one page and category:

```dart
final nosmai = NosmaiFlutter.instance;

final filters = await nosmai.getCloudFilters(
  filterType: NosmaiCloudFilterType.beautyEffect,
  page: 1,
  limit: 20,
);

final pagination = nosmai.lastPaginationInfo;
```

The `version` argument is optional. `NosmaiCloudFilterVersion.v2` is used by default.

Request a specific version explicitly only when the application must control catalog compatibility:

```dart
final filters = await NosmaiFlutter.instance.getCloudFilters(
  version: NosmaiCloudFilterVersion.v2,
);
```

The supported category values are:

- `NosmaiCloudFilterType.effects`
- `NosmaiCloudFilterType.filter`
- `NosmaiCloudFilterType.background`
- `NosmaiCloudFilterType.beautyEffect`

### Download and apply

```dart
final nosmai = NosmaiFlutter.instance;
final filter = filters.first;

final result = await nosmai.downloadCloudFilter(filter.cloudIdentifier);
final path = result['path'] as String?;

if (path != null) {
  await nosmai.applyEffect(path);
}
```

Remove a downloaded item:

```dart
await nosmai.removeCloudFilter(filter.cloudIdentifier);
```

## Capture a photo

Capture a photo with the active effects:

```dart
final nosmai = NosmaiFlutter.instance;
final photo = await nosmai.capturePhoto();

if (!photo.success || photo.imageData == null) {
  throw StateError(photo.error ?? 'Photo capture failed');
}
```

Save it to the device gallery:

```dart
await nosmai.saveImageToGallery(
  photo.imageData!,
  name: 'nosmai_photo',
);
```

Request the platform photo library or media permission before saving.

## Record video

Start recording:

```dart
final started =
    await NosmaiFlutter.instance.startRecording();
```

Stop and receive the result:

```dart
final recording =
    await NosmaiFlutter.instance.stopRecording();

if (!recording.success || recording.videoPath == null) {
  throw StateError(recording.error ?? 'Recording failed');
}
```

Save it:

```dart
await NosmaiFlutter.instance.saveVideoToGallery(
  recording.videoPath!,
  name: 'nosmai_video',
);
```

The application decides whether to save, upload, move, or delete the returned file.

## Handle errors and license changes

Listen to SDK errors:

```dart
StreamSubscription<NosmaiError>? errorSubscription;

void observeNosmaiErrors() {
  errorSubscription =
      NosmaiFlutter.instance.onError.listen((error) {
    // Show an appropriate message or record the failure.
  });
}
```

Listen to license status:

```dart
StreamSubscription<NosmaiLicenseStatus>? licenseSubscription;

void observeLicense() {
  licenseSubscription =
      NosmaiFlutter.instance.onLicenseStatusChanged.listen((status) {
    if (status != NosmaiLicenseStatus.valid) {
      // Disable features that require a valid license.
    }
  });
}
```

Cancel both subscriptions when their owner is disposed.

## Lifecycle and navigation

`NosmaiCameraPreview` handles normal application background and foreground changes.

For ordinary route navigation:

1. remove the camera screen from the widget tree
2. allow the native preview view to be disposed
3. create a new `NosmaiCameraPreview` when returning

Do not call `cleanup` from every widget `dispose` method. A global cleanup that overlaps creation of the next camera screen can stop the new preview.

For a short tab switch where the camera widget remains mounted:

```dart
await NosmaiFlutter.instance.pauseCamera();
await NosmaiFlutter.instance.resumeCamera();
```

For a custom screen that must detach before navigation:

```dart
await NosmaiFlutter.instance.stopProcessing();
await NosmaiFlutter.instance.detachCameraView();
```

Use full cleanup only when the application is finished with Nosmai:

```dart
await NosmaiFlutter.instance.cleanup();
```

After full cleanup, initialize again before creating another camera preview.

## Release checklist

Before distributing the application:

1. Use license keys issued for the final Android package name and iOS bundle identifier.
2. Test first launch with network access.
3. Test both platform permission flows.
4. Test front and back camera switching.
5. Test leaving and returning to the camera screen.
6. Test application background and foreground transitions.
7. Test local and cloud filter failures.
8. Test recording with beauty and background effects active.
9. Test Android and iOS on the oldest supported physical devices.
10. Confirm no test license key is committed in the repository.

## Common issues

### Initialization returns false

Confirm that:

- the device has internet access
- the license key is complete
- the Android package name or iOS bundle identifier matches the Nosmai Console project
- initialization is called once before creating the preview

### The preview is blank

Confirm that:

- camera permission was granted
- initialization completed successfully
- only one `NosmaiCameraPreview` is active
- the preview has a non-zero size
- no other camera package is holding the camera

### The second camera screen is blank

Do not run global cleanup while the next preview is being created. Remove the old preview from the widget tree and let its native view finish disposal before creating the next camera screen.

### A local filter is missing

Confirm that:

- its folder is declared in `pubspec.yaml`
- the `.nosmai`, manifest, and preview filenames use the expected filter name
- the manifest contains a supported package type
- `flutter clean` and `flutter pub get` were run after changing bundled assets

Use `getDebugFilters` only for loose development packages.

### Performance drops

Test one feature at a time. Face-tracked makeup, background replacement, recording, and streaming each add work. Keep one camera preview active and use 30 FPS as the normal target.

---

# Errors and troubleshooting
Source: https://docs.nosmai.com/docs/effects/errors/

## Overview

Nosmai operations can fail because of application configuration, device state, permissions, network access, an invalid filter package, or an incorrect camera lifecycle.

A production integration should:

1. check every asynchronous result
2. keep technical details out of user-facing messages
3. retry only temporary failures
4. avoid retry loops for invalid configuration
5. release the camera when a screen closes
6. record enough context to reproduce a failure
7. never record a complete license key

The exact error shape differs by platform:

| Platform | Main error form |
| --- | --- |
| Android | Exceptions and callback error strings |
| iOS | `NSError` with `NosmaiErrorDomain` and `NosmaiErrorCode` |
| Flutter | `NosmaiError` with `NosmaiErrorType`, `code`, `message`, and optional `details` |

## Recommended error flow

Handle an operation in this order:

```text
Start operation
    |
    v
Check immediate validation
    |
    v
Wait for asynchronous result
    |
    +---- success ----> update the interface
    |
    +---- failure ----> classify the error
                           |
                           +---- temporary ----> offer retry
                           |
                           +---- permission ----> explain required access
                           |
                           +---- configuration -> stop and fix configuration
```

Do not update a selected filter button before the apply completion reports success.

## Initialization failures

Common initialization problems:

| Problem | Meaning | Action |
| --- | --- | --- |
| Empty license key | The application supplied no usable key | Fix application configuration |
| Invalid key | The key does not exist, was revoked, or is incomplete | Check the full key in the Nosmai Console |
| Package mismatch | The key belongs to a different Android package name or iOS bundle identifier | Register and use the final application identifier |
| Platform mismatch | An Android key is used on iOS, or an iOS key is used on Android | Use the key issued for that platform |
| Expired license | The license is no longer active | Renew or replace the license |
| Unsupported SDK version | The installed SDK version is not accepted by the license service | Update to a supported release |
| First launch offline | No valid local license exists and the server cannot be reached | Connect the device and retry |
| Incorrect device time | The verification timestamp cannot be trusted | Enable automatic date and time |
| Monthly usage limit | The project reached its active-device allowance | Review the project plan |

Initialization should have one application-level owner. Starting it from several screens can produce confusing state and duplicate work.

## License error codes

The native license service can report these stable codes:

| Code | Meaning | Retry |
| --- | --- | --- |
| `LICENSE_EXPIRED` | The license or subscription expired | No, renew first |
| `API_KEY_INVALID` | The key is invalid, incomplete, or revoked | No, fix the key |
| `PACKAGE_ID_MISMATCH` | The key does not match the installed application identifier | No, fix project configuration |
| `PLATFORM_MISMATCH` | The key belongs to another platform | No, use the correct key |
| `SDK_VERSION_UNSUPPORTED` | The installed SDK version is not supported | No, update the SDK |
| `MAU_LIMIT_EXCEEDED` | The monthly active-device limit was reached | No, review the plan |
| `DEVICE_NOT_REGISTERED` | The device is not accepted by the license configuration | No, review the project |
| `TIMESTAMP_INVALID` | Device date or time is incorrect | After fixing device time |
| `TOO_MANY_REQUESTS` | Too many verification requests were sent | Yes, with delay |
| `MISSING_FIELDS` | Required verification information was not available | No, review integration |
| `NETWORK_ERROR` | The license server could not be reached | Yes, when connectivity returns |
| `INTERNAL_ERROR` | Verification could not complete because of a local or server error | Yes, once, then report |

Do not show these technical codes directly to an end user. Map them to a short explanation and a suitable action.

## Network failures

License verification and cloud filter operations need network access. Camera frames and active local effects do not need to be uploaded for normal processing.

Common network conditions:

| Condition | Typical symptom | Action |
| --- | --- | --- |
| DNS failure | Hostname cannot be resolved | Check internet, DNS, VPN, and private DNS settings |
| Connection failure | Server cannot be reached | Check firewall, proxy, VPN, and device network |
| Timeout | Request took too long | Retry with delay |
| TLS failure | Secure connection could not be verified | Check device time, certificates, proxy, and network inspection |
| Rate limit | Too many requests | Apply increasing retry delays |
| Interrupted transfer | Response ended before completion | Retry when the network is stable |

For example, cURL code `6` on Android means the device could not resolve the server hostname. It is a DNS or connectivity issue, not a camera or filter-rendering failure.

### Retry policy

Use a bounded retry:

```text
Attempt 1: immediately
Attempt 2: after 1 second
Attempt 3: after 3 seconds
Attempt 4: after 8 seconds
Then stop and wait for user action or a network-state change
```

Do not continuously retry invalid keys, package mismatches, expired licenses, or unsupported SDK versions.

## Android error handling

### Initialization

Android initialization validates required arguments immediately:

```java
try {
    NosmaiSDK.initialize(
        getApplicationContext(),
        BuildConfig.NOSMAI_LICENSE_KEY
    );
} catch (IllegalArgumentException error) {
    // Context or license key is missing.
} catch (RuntimeException error) {
    // Native SDK initialization could not start.
}
```

`NosmaiSDK.isInitialized()` confirms that the Android API was initialized:

```java
if (!NosmaiSDK.isInitialized()) {
    // Do not open the camera screen.
}
```

This state does not replace asynchronous license-status handling inside the SDK. A temporary network failure can still be reported while the SDK evaluates a valid local license.

### Filter apply

Always use the callback:

```java
NosmaiEffects.applyEffect(
    filterPath,
    new NosmaiEffects.EffectCallback() {
        @Override
        public void onSuccess() {
            // Update selected filter state.
        }

        @Override
        public void onError(String errorMessage) {
            // Keep the previous selected state and report the failure.
        }
    }
);
```

Android callback errors are descriptive strings. Do not parse them as a permanent numeric error contract. Use them for diagnostics and show a stable application message to the user.

### Direct camera input errors

Listen for direct-input failures:

```java
previewView.addOnOesInputErrorListener(reason -> {
    // Stop the current Camera2 session.
    // Reconnect using the normal Camera2 frame-input fallback.
});
```

A direct-input error should switch to the sample application's supported fallback. It should not leave the user on a black preview.

### Recording

```java
NosmaiSDK.startRecording(
    previewView,
    outputPath,
    new NosmaiSDK.RecordingCallback() {
        @Override
        public void onStarted(
            boolean success,
            String error
        ) {
            if (!success) {
                // Re-enable the record button.
            }
        }
    }
);
```

When stopping, wait for finalization:

```java
NosmaiSDK.stopRecording(
    new NosmaiSDK.RecordingCallback() {
        @Override
        public void onCompleted(
            String outputPath,
            boolean success,
            String error
        ) {
            if (!success) {
                // Do not publish an incomplete file.
            }
        }
    }
);
```

## iOS error handling

### Public error codes

iOS uses `NosmaiErrorDomain` with these `NosmaiErrorCode` values:

| Value | Constant | Meaning |
| --- | --- | --- |
| `1000` | `NosmaiErrorCodeUnknown` | An unclassified failure |
| `1001` | `NosmaiErrorCodeLicenseInvalid` | The license was rejected |
| `1002` | `NosmaiErrorCodeLicenseExpired` | The license expired |
| `1003` | `NosmaiErrorCodeNetworkError` | A required network request failed |
| `1004` | `NosmaiErrorCodeCameraPermissionDenied` | Camera permission was denied |
| `1005` | `NosmaiErrorCodeCameraNotAvailable` | The camera is unavailable |
| `1006` | `NosmaiErrorCodeEffectLoadFailed` | A filter or effect could not load |
| `1007` | `NosmaiErrorCodeInitializationFailed` | SDK initialization failed |
| `1008` | `NosmaiErrorCodeResourceNotFound` | A required file was not found |
| `1009` | `NosmaiErrorCodeInvalidParameter` | A method received an invalid value |
| `1010` | `NosmaiErrorCodeMemoryError` | Required memory could not be allocated |
| `403` | `NosmaiErrorCodeFeatureNotEnabled` | The license does not include the requested feature |

### Initialization

```objc
[[NosmaiCore shared]
    initializeWithAPIKey:licenseKey
              completion:^(BOOL success, NSError *error) {
        if (success) {
            return;
        }

        if ([error.domain isEqualToString:NosmaiErrorDomain]) {
            switch ((NosmaiErrorCode)error.code) {
                case NosmaiErrorCodeNetworkError:
                    // Offer retry when connectivity returns.
                    break;

                case NosmaiErrorCodeLicenseInvalid:
                case NosmaiErrorCodeLicenseExpired:
                    // Stop and review the license.
                    break;

                default:
                    // Record error.localizedDescription.
                    break;
            }
        }
    }];
```

### Delegates

Set delegates after initialization:

```objc
NosmaiCore *core = [NosmaiCore shared];
core.delegate = self;
core.camera.delegate = self;
core.effects.delegate = self;
```

Receive general failures:

```objc
- (void)nosmaiDidFailWithError:(NSError *)error {
    // Record the error domain, code, and safe context.
}
```

Receive camera failures:

```objc
- (void)nosmaiCameraDidFailWithError:(NSError *)error {
    // Stop capture and show a retry or permission action.
}
```

Receive effect failures:

```objc
- (void)nosmaiEffectDidFailWithError:(NSError *)error
                           forEffect:(NSString *)effectID {
    // Keep the previous selected effect state.
}
```

### License status

The iOS delegate can report:

- `VALID`
- `INVALID`
- `EXPIRED`
- `UNVERIFIED`

```objc
- (void)nosmaiDidChangeLicenseStatus:(BOOL)isValid
                              status:(NSString *)status {
    if ([status isEqualToString:@"UNVERIFIED"]) {
        // Wait for verification or a valid local-license decision.
    }
}
```

`UNVERIFIED` is not the same as a definitive invalid license. It can occur while verification is pending or temporarily unavailable.

## Flutter error handling

Flutter exposes `NosmaiError`:

```dart
class NosmaiError {
  final NosmaiErrorType type;
  final String code;
  final String message;
  final String? details;
}
```

### Catch operation errors

```dart
try {
  final applied =
      await NosmaiFlutter.instance.applyEffect(filter.path);

  if (!applied) {
    // The package was not applied.
  }
} on NosmaiError catch (error) {
  final message = error.userMessage;
  final canRetry = error.isRecoverable;

  // Present message and offer retry only when appropriate.
}
```

### Listen for native errors

```dart
StreamSubscription<NosmaiError>? errorSubscription;

void startErrorListener() {
  errorSubscription =
      NosmaiFlutter.instance.onError.listen((error) {
    // Record error.type, error.code, and error.details.
  });
}
```

Cancel the listener:

```dart
await errorSubscription?.cancel();
```

### Flutter error types

| Type | Meaning |
| --- | --- |
| `unknown` | An unclassified failure |
| `stateError` | The requested action is not valid in the current state |
| `operationTimeout` | The operation exceeded its allowed time |
| `platformError` | The Android or iOS call failed |
| `networkError` | A network request failed |
| `invalidParameter` | A method received an invalid value |
| `sdkNotInitialized` | A method was called before initialization |
| `invalidLicense` | The license was rejected |
| `licenseExpired` | The license expired |
| `cameraPermissionDenied` | Camera access was denied |
| `cameraUnavailable` | The camera cannot be opened |
| `cameraConfigurationFailed` | The camera configuration was rejected |
| `cameraSwitchFailed` | Front or back camera switch failed |
| `filterNotFound` | The filter file does not exist |
| `filterInvalidFormat` | The filter package is not valid |
| `filterLoadFailed` | The filter package could not be loaded |
| `filterDownloadFailed` | A cloud filter download failed |
| `recordingPermissionDenied` | Required recording permission was denied |
| `recordingStorageFull` | The device has insufficient free storage |
| `recordingWriteFailed` | The output file could not be written |
| `recordingInProgress` | Another recording is already active |

### Preview errors

```dart
NosmaiCameraPreview(
  onInitialized: () {
    // Enable camera controls.
  },
  onError: (error) {
    // Show a retry or permission message.
  },
)
```

The preview error callback reports camera-view startup and resume failures. It does not replace `onError`, which can also report filter, license, network, and recording failures.

## Camera permission problems

### Permission denied

When permission is denied:

1. stop the loading indicator
2. explain why camera access is required
3. show a retry action if the system can ask again
4. show an application-settings action after permanent denial
5. keep non-camera screens usable

Do not repeatedly trigger the system permission dialog without a user action.

### Camera unavailable

The camera can be unavailable when:

- another application is using it
- another camera library in the same application owns it
- the previous camera screen did not release it
- the device is transitioning between foreground and background
- the camera service has temporarily failed

Stop every previous camera owner before retrying.

## Black preview

Use this order:

1. confirm initialization succeeded
2. confirm camera permission is granted
3. confirm the preview has a non-zero size
4. confirm only one camera source is active
5. confirm the old camera closed before a switch
6. confirm the previous screen released its camera
7. confirm the app returned to the foreground
8. remove all active effects and test the plain preview

### Android

Check:

- `NosmaiSDK.startProcessing(previewView)` was called
- Camera2 starts after the preview is ready
- the camera sends the correct size and orientation
- a direct-input error activates the normal Camera2 fallback
- `cameraHelper.stopCamera()` runs when leaving the screen

### iOS

Check:

- initialization completion reported success
- `attachToView:` was called
- the view has a valid frame
- `startCapture` returned `YES`
- no other `AVCaptureSession` owns the camera
- the old view was detached before attaching a new one

### Flutter

Check:

- `NosmaiFlutter.initialize` returned `true`
- only one `NosmaiCameraPreview` is mounted
- global cleanup is not running while a new preview is created
- the previous platform view finished disposal
- the preview widget has a valid size

## Camera switching problems

Symptoms include:

- preview becomes black
- front camera works but back camera does not
- both cameras stop after switching back
- effect orientation becomes incorrect

Correct behavior:

1. ignore repeated switch taps while a switch is active
2. stop or reconfigure the old camera
3. wait for the new camera source
4. update front or back camera state
5. update mirroring once
6. resume effects after frames arrive

Do not open the new camera while the old Camera2 or AVFoundation session is still closing.

## Orientation and mirroring problems

| Symptom | Likely cause |
| --- | --- |
| Preview is sideways | Camera sensor rotation was not converted to display orientation |
| Preview is upside down | Rotation direction is reversed |
| Front preview is not mirrored | Front preview mirror was not enabled |
| Face effect moves in the opposite horizontal direction | The frame and preview use different mirror rules |
| Effect is correct but recorded video is reversed | Preview and output mirroring were treated as the same setting |

Apply horizontal mirroring in one place. A double mirror can make the preview appear correct while face landmarks and effects use the wrong direction.

## Filter does not load

Check:

1. the file path exists
2. the filename ends in `.nosmai`
3. the package is complete and was not modified after creation
4. the package supports the installed SDK version
5. the license includes the required feature
6. the apply completion or callback reports success
7. the package type is one of the supported types

Supported package types:

- `filter`
- `effect`
- `beauty_effect`
- `background`

Use `applyEffect` for all four types. Do not choose a lower-level apply method based on a guessed package type.

## Local filter is missing from the list

Production discovery requires:

```text
filter_name.nosmai
filter_name_manifest.json
filter_name_preview.png
```

The base name must match.

Check that:

- all three files are present
- the manifest JSON is valid
- the manifest type is supported
- the folder is included in the application bundle
- Flutter declares the folder in `pubspec.yaml`
- the local filter cache was refreshed after adding files

Use debug discovery for loose development packages. Do not use it as the production catalog.

## Filter appears in the wrong category

The package manifest controls its category.

| Intended behavior | Manifest type |
| --- | --- |
| Full-frame color or LUT filter | `filter` |
| AR mask, sticker, particle, or 3D effect | `effect` |
| Packaged makeup or face beauty effect | `beauty_effect` |
| Packaged background effect | `background` |

Fix the package manifest. Do not relabel the returned item only in application UI.

## Effect disappears or replaces another effect

Some external packages share the same active category.

Expected examples:

- applying a new regular `filter` replaces the previous regular filter
- applying a new `effect` replaces the previous AR or beauty package
- applying a `beauty_effect` replaces the previous AR or beauty package
- a regular filter can remain active with an AR effect

Read active state after applying a package. Do not assume every package can remain active with every other package.

## Face effect is delayed or unstable

Check:

- the preview remains close to the target frame rate
- the face is large enough and reasonably lit
- the camera frame orientation is correct
- frame timestamps increase normally
- the app is not doing expensive work on the main thread
- recording or streaming is not exceeding the device's capacity
- only visible features remain active

Do not solve tracking delay by skipping an uncontrolled number of face updates. A stale face result can make makeup detach during movement.

## Background replacement problems

### Nothing changes

Confirm that:

- advanced effects are included in the license
- the background method returned success where available
- image data is valid
- a video path is an accessible local file
- the device supports the required processing

### Edge quality is poor

Test:

- even lighting
- visible separation between subject and background
- a supported camera resolution
- lower motion
- a simpler set of active effects

Hair and motion boundaries are more difficult than static clothing and wall boundaries.

## Cloud filter problems

### Catalog is empty

Check:

- cloud filters are enabled for the license
- the device has internet access
- the requested category is valid
- the requested compatibility version is supported
- pagination is not requesting a page beyond the final page

Cloud category values are:

- `effects`
- `filter`
- `bg`
- `beauty_effect`

### Download fails

Check:

- filter ID is not empty
- network access remains available
- the device has free storage
- the application can write to its cache or files directory
- the item is still available in the catalog

Do not apply a cloud item until the download result provides a valid local path.

## Recording problems

### Recording does not start

Check:

- camera preview is already active
- no previous recording is starting, active, or stopping
- microphone permission is granted when audio is required
- the output location is writable
- the device has free storage

### Recording stops but no file is available

Wait for the stop completion. Video finalization can continue briefly after the user taps stop.

Do not:

- close the camera screen before finalization
- upload the file before success
- start another recording while stop is pending
- delete or move the output path early

## Application freezes

Common causes:

- blocking the main thread for every frame
- waiting synchronously for camera or graphics work
- applying many filters without waiting for the previous operation
- starting several downloads or decodes at once
- repeatedly creating and destroying the camera preview
- recording and streaming above the device's sustainable resolution

Actions:

1. reproduce without active effects
2. add one feature at a time
3. measure preview frame rate
4. inspect main-thread work
5. test without recording or streaming
6. test on another supported device
7. capture a system trace if the freeze remains

## Native crashes

Examples include:

- `UnsatisfiedLinkError` on Android
- `SIGSEGV` in a native library
- graphics errors followed by process termination
- release-only crashes after code shrinking

### Android `UnsatisfiedLinkError`

This normally means Java and native code do not match.

Check:

- the AAR comes from one SDK release
- no older SDK classes are copied into the app
- the final package includes `lib/arm64-v8a/libnosmai.so`
- the device is arm64-v8a
- code shrinking kept required JNI methods
- the application does not package two different Nosmai native libraries

### Native graphics crash

Collect:

- complete crash stack
- device model and Android or iOS version
- SDK version
- camera position
- active filter names and types
- actions immediately before the crash
- whether recording or streaming was active

Do not hide a repeatable native crash behind an automatic restart. Report the reproducible sequence.

## Performance troubleshooting

Use this isolation order:

1. plain preview
2. one regular color filter
3. one face-tracked effect
4. built-in beauty
5. makeup
6. background replacement
7. recording
8. live streaming

Record frame rate and device temperature at each step.

Common improvements:

- target a stable 30 FPS
- keep one camera preview active
- avoid invisible active effects
- use the recommended direct Android camera input with its device fallback
- reduce recording or streaming resolution on slower devices
- avoid large uncompressed filter textures
- prevent repeated filter changes while one apply is still running
- pause the camera when the app is not visible

## Lifecycle troubleshooting

### Camera indicator remains on

The camera source was not stopped.

- Android: stop Camera2 when the activity or fragment leaves the screen
- iOS: call `stopCapture` and `detachFromView`
- Flutter: remove `NosmaiCameraPreview` and allow its native view to dispose

### Camera does not return

The old view may still own the camera, or global cleanup may overlap the new screen.

Use:

- pause and resume for temporary backgrounding
- stop and detach when leaving a camera screen
- full cleanup only when the application is finished with the SDK

Do not call full cleanup from every Flutter widget `dispose` method.

## Logging

### iOS

Enable SDK debug logging during investigation:

```objc
[[NosmaiCore shared] setDebugLoggingEnabled:YES];
```

Disable it for release:

```objc
[[NosmaiCore shared] setDebugLoggingEnabled:NO];
```

### Android

Use Android Studio Logcat and filter by application package and Nosmai-related tags. The current public Android API does not expose the same logging toggle as iOS.

### Flutter

Capture:

- Dart exceptions
- `NosmaiFlutter.instance.onError`
- Android Logcat when testing Android
- Xcode console when testing iOS

Flutter logs alone may not include the native cause of a camera or graphics failure.

## Protect sensitive information

Before sharing logs:

- remove the complete Nosmai license key
- remove access tokens
- remove private live-stream channel tokens
- remove user identifiers
- review local file paths for personal information

Keep:

- SDK version
- application version
- platform and OS version
- device model
- error domain, code, and message
- filter type and non-sensitive filter identifier
- reproducible steps

## Support report checklist

Include:

1. platform
2. device model
3. OS version
4. application version
5. Nosmai SDK or Flutter package version
6. debug or release build
7. front or back camera
8. active filter types
9. whether recording or streaming was active
10. exact reproduction steps
11. complete relevant error or native crash stack
12. result without active filters

Do not send only a screenshot of the final frame when reporting a crash or freeze. Include the logs and the action sequence that produced it.

---

# Releases and compatibility
Source: https://docs.nosmai.com/docs/effects/releases-and-compatibility/

## Current production versions

| Component | Version | Delivery |
| --- | --- | --- |
| Android native SDK | `3.0.0` | `nosmai-sdk-3.0.0.aar` from GitHub Releases |
| iOS native SDK | `3.0.0` | `NosmaiCameraSDK` CocoaPod or `nosmai.framework.zip` from GitHub Releases |
| Flutter package | `3.0.6` | `nosmai_camera_sdk` from pub.dev |
| Cloud catalog schema | `2.0.0` | Selected by SDK request options |

Flutter package `3.0.6` is compatible with Android native SDK `3.0.0` and iOS native SDK `3.0.0`.

The Flutter, native SDK, and cloud schema versions are independent. A Flutter patch can improve the Dart or bridge layer without requiring a new native binary. Do not infer compatibility only because the version numbers look similar.

## Official distribution locations

| Component | Official location |
| --- | --- |
| Android | [github.com/nosmai/camera-sdk-android/releases](https://github.com/nosmai/camera-sdk-android/releases) |
| iOS | [github.com/nosmai/camera-sdk-ios/releases](https://github.com/nosmai/camera-sdk-ios/releases) |
| iOS CocoaPod | [`NosmaiCameraSDK`](https://cocoapods.org/pods/NosmaiCameraSDK) |
| Flutter | [`nosmai_camera_sdk`](https://pub.dev/packages/nosmai_camera_sdk) |
| Flutter source and releases | [github.com/nosmai/nosmai_camera_sdk_flutter](https://github.com/nosmai/nosmai_camera_sdk_flutter) |

Use official release assets. Do not copy a framework or AAR from an unrelated application, old build folder, chat attachment, or unverified mirror.

## Platform contract

| Platform | Minimum OS | Architecture | Current limitation |
| --- | --- | --- | --- |
| Android | API 21 | `arm64-v8a` | Other ABIs are not included |
| iOS | iOS 15.0 | Physical arm64 device | No iOS Simulator slice |
| Flutter Android | API 21 | `arm64-v8a` | Host app supplies the external AAR |
| Flutter iOS | iOS 15.0 | Physical arm64 device | Plugin resolves the CocoaPod |

The current Android native libraries are compatible with 16 KB native page-size requirements.

Use physical devices for final camera, face tracking, recording, streaming, lifecycle, and performance validation.

## Verify native downloads

Every native GitHub release includes `SHA256SUMS`.

Download the binary and checksum file into the same directory, then run:

```sh
shasum -a 256 -c SHA256SUMS
```

Expected Android result:

```text
nosmai-sdk-3.0.0.aar: OK
```

Expected iOS result:

```text
nosmai.framework.zip: OK
```

Do not integrate an artifact when checksum verification fails. Download both files again from the same release and contact Nosmai support if the mismatch remains.

## Native Android installation

Place the verified AAR in the native application's module:

```text
app/libs/nosmai-sdk-3.0.0.aar
```

Add it once:

```kotlin
dependencies {
    implementation(files("libs/nosmai-sdk-3.0.0.aar"))
}
```

Do not keep an old AAR under another filename. Gradle can package duplicate Java classes or native libraries even when only one version is called directly.

## Native iOS installation

CocoaPods is recommended:

```ruby
platform :ios, '15.0'

target 'CameraApp' do
  use_frameworks!
  pod 'NosmaiCameraSDK', '3.0.0'
end
```

Run:

```sh
pod install --repo-update
```

Open the generated `.xcworkspace`.

For manual integration, verify and unzip `nosmai.framework.zip`, then embed `nosmai.framework` with **Embed & Sign**. Do not keep a manual framework and CocoaPods copy in the same application target.

## Flutter installation contract

Add the Dart package:

```yaml
dependencies:
  nosmai_camera_sdk: ^3.0.6
```

### Flutter iOS

The plugin depends on `NosmaiCameraSDK ~> 3.0.0` and resolves it through CocoaPods. Do not manually copy an iOS framework into the Flutter project.

### Flutter Android

The proprietary AAR is intentionally excluded from pub.dev so the package remains small and the commercial binary is distributed through its authorized release.

Download and verify Android native SDK `3.0.0`, rename it to `nosmai-release.aar`, and place it at:

```text
android/app/libs/nosmai-release.aar
```

Add it to the host app module:

```gradle
dependencies {
    implementation files('libs/nosmai-release.aar')
}
```

The Flutter plugin uses the native AAR as a compile-only dependency. The consuming Android application must supply it at build and runtime.

## Upgrade procedure

Use this sequence for any upgrade:

1. Read the native and Flutter release notes.
2. Confirm the documented compatibility combination.
3. Download and verify new native artifacts.
4. Replace the previous Android AAR instead of keeping both.
5. Update the iOS pod version or replace the manual framework, not both.
6. Update the Flutter package constraint when applicable.
7. Run `flutter clean` or the native platform's clean build when binary dependencies change.
8. Resolve CocoaPods again and inspect `Podfile.lock`.
9. Build from a clean state.
10. Test licenses, camera lifecycle, package rules, cloud filters, capture, recording, and streaming.

For Flutter:

```sh
flutter clean
flutter pub get

cd ios
pod install --repo-update
```

Return to the project root before running Flutter build commands.

## Detect duplicate or stale native SDKs

Common signs include:

- Android duplicate-class or duplicate-native-library errors.
- An API compiles but fails with `UnsatisfiedLinkError` at runtime.
- iOS reports duplicate framework output or loads an older version.
- Flutter behavior differs between Android and iOS after only one native side was updated.
- A method exists in Dart but is missing from the installed native binary.

Check:

- every Android `libs` directory
- Gradle file dependencies and transitive dependency reports
- Xcode **Frameworks, Libraries, and Embedded Content**
- CocoaPods `Podfile.lock`
- manually copied iOS framework folders
- Flutter plugin caches only when diagnosing a stale dependency

Keep exactly one intended native SDK version per application binary.

## Compatibility policy

- Patch releases should remain source compatible unless release notes state otherwise.
- Minor releases can add APIs and capabilities while preserving the documented major-version contract.
- Major releases can require application changes.
- A newer Flutter package may require a specific minimum native version.
- Cloud catalog schema changes are documented separately from package versions.
- Published versions and release assets are immutable. A correction is delivered under a new version.

Pin exact native versions in production applications. Flutter applications may use a compatible Dart constraint, but release builds should preserve their resolved dependency lockfiles according to the application's dependency policy.

## Production validation after an upgrade

At minimum, verify:

1. Valid, invalid, expired, and temporary network license states.
2. First camera open, close, reopen, and app relaunch.
3. Front and back camera switching.
4. Local `filter`, `effect`, `beauty_effect`, and `background` packages.
5. Built-in beauty, makeup, face reshape, and color controls.
6. Mutual exclusion between built-in controls and AR-slot packages.
7. Cloud type tabs, pagination, download, cache, retry, apply, and removal.
8. Background None and global clear actions.
9. Photo capture and processed video recording.
10. Live streaming integration if the application uses it.
11. Repeated apply, remove, and reapply cycles.
12. Release-mode build with no internal diagnostics or sensitive log output.

Continue with the platform guide for [Android](/docs/effects/android), [iOS](/docs/effects/ios), or [Flutter](/docs/effects/flutter).

---

# Nosmai Moderation - Full Documentation

> On-device content and text moderation SDK for mobile apps. It flags NSFW imagery and moderates toxic text. Everything runs fully offline on the device, so no frame or message ever leaves the phone. Thresholds are tunable at runtime. Available for iOS, Android and Flutter.

This file is the complete Nosmai Moderation documentation concatenated into one
document for AI assistants. Each section lists its canonical page URL. For the
curated link index, see llms.txt.

Supported platforms: iOS (Swift) via CocoaPods `NosmaiModerationSDK`, Android
(Kotlin) via `nosmai-detection.aar`, Flutter via `nosmai_moderation_sdk`. The
license key looks like `NOSMAI-XXXX`, is bound to one app and platform, and is
verified online when required. The SDK may use securely stored license state for
limited offline operation according to the active SDK version and commercial plan.

---

# Introduction

Source: https://docs.nosmai.com/docs/moderations/introduction/

## What is Nosmai Moderation?

Nosmai Moderation is a content safety SDK that runs entirely on the user's device. It checks images, recorded video, chat text and the live camera for unsafe content, and returns a clear verdict your app can act on. There is no cloud round-trip, so frames and messages never leave the phone.

You install one SDK, initialize it with a license key, and call a small set of methods. All models ship inside the SDK, and limited offline operation may be available according to the active SDK version and commercial plan.

## What it moderates

| Surface | What it does |
| --- | --- |
| Image | Check a single photo |
| Video | Sample and aggregate a recorded clip |
| Text | Check a chat message or comment |
| Live camera | Real-time, per-frame moderation |

It detects adult / NSFW imagery and unsafe text (profanity, toxicity, hate, harassment, threats).

## Why on-device

- Privacy: user content never leaves the device, which helps with compliance.
- Latency: no network call per check, so results are immediate.
- Cost: no per-request cloud bill for moderation.
- Offline: works without connectivity after the one-time license check.

## Supported platforms

| Platform | Package | Minimum version |
| --- | --- | --- |
| iOS (Swift) | `NosmaiModerationSDK` (CocoaPods) | iOS 15.1+ |
| Android (Kotlin) | `nosmai-detection.aar` | API 24+ (arm64-v8a), Kotlin 2.2+ |
| Flutter | `nosmai_moderation_sdk` | Flutter 3+ |
| React Native | `@nosmai/moderation-react-native` | New Architecture (TurboModules) |

New here? The Quickstart takes you from install to your first verdict in a few minutes.

---

# Quickstart

Source: https://docs.nosmai.com/docs/moderations/quickstart/

## Before you start

You need a Nosmai license key for your app. It looks like `NOSMAI-XXXX` and is tied to your app's package name or bundle identifier and to the platform. Get one from the Nosmai dashboard.

The first launch verifies the key online, so make sure the device has connectivity the first time you run your app.

## 1. Install

Pick your platform.

Flutter:

```yaml
dependencies:
  nosmai_moderation_sdk: ^2.0.0
```

iOS:

```ruby
pod 'NosmaiModerationSDK', '~> 2.0'
```

Android (download `nosmai-detection.aar` from the releases page, put it in `app/libs/`, then reference it):

```kotlin
dependencies {
    implementation(files("libs/nosmai-detection.aar"))
}
```

## 2. Initialize

Initialize once at startup with your license key. Initialization does a network check and loads the models, so run it off the main thread.

Flutter:

```dart
final res = await NosmaiModeration.initialize('NOSMAI-XXXX');
if (!res.success) debugPrint('init failed: ${res.error}');
```

iOS:

```swift
DispatchQueue.global(qos: .userInitiated).async {
    let ok = NosmaiSDK.initialize(licenseKey: "NOSMAI-XXXX")
}
```

Android:

```kotlin
Executors.newSingleThreadExecutor().execute {
    val ok = NosmaiSDK.init(context, "NOSMAI-XXXX")
}
```

## 3. Run your first check

Moderate an image and read the verdict.

Flutter:

```dart
final r = await NosmaiModeration.analyzeImage(file.path);
print(r.isUnsafe ? 'UNSAFE' : 'SAFE');
```

iOS:

```swift
if let r = NosmaiSDK.analyzeImage(uiImage) {
    print(r.isUnsafe ? "UNSAFE" : "SAFE")
}
```

Android:

```kotlin
val r = NosmaiSDK.analyzeImage(bitmap)
Log.d("Mod", if (r.isUnsafe) "UNSAFE" else "SAFE")
```

## Next steps

- How moderation works: surfaces, detection types, results and thresholds.
- Platform guides: iOS, Android, Flutter.
- Authentication: how license keys and offline validation work.

---

# Authentication

Source: https://docs.nosmai.com/docs/moderations/authentication/

## License keys

The SDK authenticates with a license key that looks like `NOSMAI-XXXX`. Each key is issued for one app and is bound to:

- your app's package name (Android) or bundle identifier (iOS), and
- the platform (iOS or Android).

A key used from a different app or platform is rejected. Get your key from the Nosmai dashboard and pass it to `init` / `initialize` at startup.

The license key is an app-level identifier, not a server secret. It does not unlock anything on its own: the SDK verifies it with the Nosmai licensing service before any models load.

## How verification works

1. The SDK verifies the key online when required.
2. After a successful check, the SDK may use securely stored license state for limited offline operation according to the active SDK version and commercial plan.
3. When offline access is unavailable, the device must reconnect and verify again.

If the key is missing, invalid, or expired, initialization fails and the SDK does not run. Handle the failure return from `init` / `initialize` and do not moderate until it succeeds.

Because the first launch needs connectivity and verification loads the models, always call init off the main thread.

## What your license includes

A license enables a specific set of capabilities. The SDK only runs what your plan includes.

Moderation types:

| Capability | Method |
| --- | --- |
| Image moderation | `analyzeImage` |
| Video moderation | `analyzeVideo` |
| Text moderation | `moderateText` |
| Live moderation | `startStream` and live frames |

Detection categories: `adult` (NSFW) for visual classification.

If a moderation type is not in your plan, that call does nothing and returns a safe or empty result. If a detection category is not in your plan, it is never reported. Contact Nosmai to change what your license includes.

## Going to production

Use your production license key in release builds, and keep test and production keys separate so test traffic does not count against production usage.

---

# How moderation works (Concepts)

Source: https://docs.nosmai.com/docs/moderations/concepts/

## The four surfaces

One license key unlocks four ways to moderate:

| Surface | What it does | Method |
| --- | --- | --- |
| Image | Moderate a single photo | `analyzeImage` |
| Video | Sample and aggregate a recorded clip | `analyzeVideo` |
| Text | Moderate a chat message or comment | `moderateText` |
| Live camera | Real-time per-frame moderation | `startStream` and frames |

## What it detects

Adult / NSFW. A whole-image verdict:

- `safe`: clean.
- `warn`: suggestive or borderline. Advisory only, and it does not flag the content by itself.
- `block`: explicit. This makes the content unsafe.

Text. A message is classified as `safe`, `profanity`, `toxic`, `hate`, `harassment` or `threat`. Text runs in two layers: a fast keyword blocklist first, then an AI classifier for anything the blocklist misses. The result tells you which layer decided.

## Results

A visual check (image or live frame) returns a result with:

| Field | Type | Description |
| --- | --- | --- |
| `isUnsafe` | bool | True if NSFW is `block` |
| `nsfw` | enum | `safe`, `warn` or `block` |
| `nsfwScores` | object | `safe`, `sexy`, `explicit` (0 to 1) |

A text check returns `{ blocked, layer, category, score, matchedWord }`.

A video check returns `{ isUnsafe, categories, flags, framesAnalyzed, nsfw }`, where `flags` lists the timestamps that were flagged.

## Thresholds

Both NSFW bars are adjustable at runtime. Lower is stricter. The defaults are tuned to balance catching unsafe content against false positives.

```text
setNsfwThreshold(explicit, 0.45)   // NSFW block bar
setNsfwThreshold(sexy, 0.55)       // NSFW warn bar
```

Start with the defaults. Tighten a threshold only if you see misses, and loosen one only if you see false positives for your content.

## Live performance

On the live camera, the NSFW check runs on a cadence you choose (favor responsiveness or battery). This keeps the camera preview smooth across a wide range of devices. See the platform guides for the controls.

---

# iOS Platform Guide

Source: https://docs.nosmai.com/docs/moderations/ios/

## Install

Install via CocoaPods. Add the pod to your `Podfile` and run `pod install`. The SDK and its bundled models come with the pod, and everything runs on-device with no external dependencies.

```ruby
pod 'NosmaiModerationSDK', '~> 2.0'
```

Requirements: iOS 15.1+, arm64. In `Info.plist` add:

- `NSCameraUsageDescription`, required for the live-camera path.
- `ITSAppUsesNonExemptEncryption` set to `NO`. The SDK only encrypts its own bundled models locally, which is export-compliance exempt. Without this, every TestFlight or App Store upload stalls on the encryption question.

Consumers also need `ENABLE_USER_SCRIPT_SANDBOXING = NO` (Xcode 15+); a Podfile `post_install` can force it.

## Initialize

`initialize` is blocking (network plus model load), so call it off the main thread. It returns `false` if the license is invalid or expired.

```swift
import NosmaiDetection

DispatchQueue.global(qos: .userInitiated).async {
    let ok = NosmaiSDK.initialize(licenseKey: "NOSMAI-XXXX")
}
```

## Moderate an image

```swift
if let r = NosmaiSDK.analyzeImage(uiImage) {     // NosmaiResult?
    if r.isUnsafe {
        // r.nsfw -> .safe / .warn / .block
    }
}
```

## Moderate a video

```swift
NosmaiSDK.analyze(videoURL: url, frameIntervalMs: 500,
    progress: { p in /* 0...1 */ },
    completion: { result in /* result.isUnsafe, .categories, .flags */ })
```

## Moderate text

Call `initialize` first, since it validates the license. `initializeText` needs that before it can load the text model.

```swift
try? NosmaiSDK.initializeText()                  // after initialize(), off the main thread
if let t = NosmaiSDK.moderateText("some message") {
    if t.blocked { /* t.layer, t.category, t.matchedWord */ }
}
```

## Live camera (AVCaptureSession)

Feed pixel buffers from your `AVCaptureVideoDataOutput` to `pushFrame`. Results arrive via the `NosmaiListener`.

```swift
NosmaiSDK.startStream(listener: self)            // self conforms to NosmaiListener

// in captureOutput(_:didOutput:from:)
if let pb = CMSampleBufferGetImageBuffer(sampleBuffer) {
    NosmaiSDK.pushFrame(pb, rotationDegrees: 90)  // 90 makes a back-camera portrait buffer upright
}

// on leave
NosmaiSDK.stopStream()
```

Show the preview with an `AVCaptureVideoPreviewLayer` backed by the same `AVCaptureSession`.

## Thresholds

```swift
NosmaiSDK.setNsfwThreshold(.explicit, value: 0.45)  // BLOCK
NosmaiSDK.setNsfwThreshold(.sexy, value: 0.55)      // WARN
```

## Cleanup

```swift
NosmaiSDK.shutdown()
```

`analyzeImage` and `moderateText` are synchronous and fast, but they still run inference, so call them off the main thread. `analyzeVideo` is async with a completion handler.

---

# Android Platform Guide

Source: https://docs.nosmai.com/docs/moderations/android/

## Install

1. Download the latest `nosmai-detection.aar` from the releases page.
2. Put it in your app module's `libs/` folder (for example `app/libs/nosmai-detection.aar`).
3. Reference it in Gradle (below). The SDK is self-contained: everything runs on-device and all models are bundled, so there are no extra dependencies to add.

```kotlin
android {
    defaultConfig {
        minSdk = 24
        ndk { abiFilters += "arm64-v8a" }   // the SDK ships arm64-v8a
    }
}

dependencies {
    implementation(files("libs/nosmai-detection.aar"))
    // CameraX, only needed for the live-camera path
    val camerax = "1.4.2"
    implementation("androidx.camera:camera-core:$camerax")
    implementation("androidx.camera:camera-camera2:$camerax")
    implementation("androidx.camera:camera-lifecycle:$camerax")
    implementation("androidx.camera:camera-view:$camerax")
}
```

The consuming app must use Kotlin 2.2.0 or later (the AAR ships Kotlin 2.2.0 metadata).

Permissions (`AndroidManifest.xml`): `INTERNET` for the license check and `CAMERA` for the live path only.

## Initialize

`init` is blocking (network plus model load), so call it off the main thread. It returns `false` if the license is invalid or expired.

```kotlin
import com.nosmai.detection.NosmaiSDK

Executors.newSingleThreadExecutor().execute {
    val ok = NosmaiSDK.init(context, "NOSMAI-XXXX")
}
```

## Moderate an image

```kotlin
val bitmap = BitmapFactory.decodeFile(path)
val r = NosmaiSDK.analyzeImage(bitmap)          // NosmaiResult
if (r.isUnsafe) {
    // r.nsfw -> SAFE / WARN / BLOCK
    Log.d("Mod", "nsfw ${r.nsfw}")
}
```

## Moderate a video

```kotlin
NosmaiSDK.analyzeVideo(
    context, uri, frameIntervalMs = 500L,
    onProgress = { p -> /* 0..1 */ },
    onComplete = { v -> /* v.isUnsafe, v.categories, v.flags */ },
)
```

## Moderate text

Call `init` first, since it validates the license. `initText` needs that before it can load the text model.

```kotlin
NosmaiSDK.initText(context)                      // after init(), off the main thread
val t = NosmaiSDK.moderateText("some message")   // NosmaiTextResult
if (t.blocked) Log.d("Mod", "${t.category} via ${t.layer} (${t.matchedWord})")
```

## Live camera (CameraX)

Feed CameraX frames to `pushFrame`. Results arrive on the main thread via `NosmaiListener`.

```kotlin
NosmaiSDK.startStream(object : NosmaiListener {
    override fun onResult(r: NosmaiResult) { /* every frame */ }
    override fun onUnsafe(r: NosmaiResult) { /* turned unsafe */ }
    override fun onSafe() { /* recovered */ }
})

val analysis = ImageAnalysis.Builder()
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .setOutputImageRotationEnabled(true)
    .build()
    .also { it.setAnalyzer(executor) { proxy -> NosmaiSDK.pushFrame(proxy) } }

cameraProvider.bindToLifecycle(owner, CameraSelector.DEFAULT_BACK_CAMERA, preview, analysis)

// on leave
NosmaiSDK.stopStream()
```

## Thresholds

```kotlin
NosmaiSDK.setNsfwThreshold(NosmaiNsfwClass.EXPLICIT, 0.45f) // BLOCK
NosmaiSDK.setNsfwThreshold(NosmaiNsfwClass.SEXY, 0.55f)     // WARN
```

## Cleanup

```kotlin
NosmaiSDK.shutdown()
```

All `NosmaiListener` callbacks are delivered on the main thread, so you can update UI directly.

---

# Flutter Platform Guide

Source: https://docs.nosmai.com/docs/moderations/flutter/

## Install

Add the plugin to `pubspec.yaml`, then run `flutter pub get`.

```yaml
dependencies:
  nosmai_moderation_sdk: ^2.0.0
```

Android. Bundle the native AAR in your host app. Download `nosmai-detection.aar` from the Android releases, put it in `android/app/libs/`, and reference it:

```kotlin
android {
  defaultConfig {
    minSdk = 24
    ndk { abiFilters += "arm64-v8a" }
  }
}
dependencies {
  implementation(files("libs/nosmai-detection.aar"))
}
```

iOS needs no extra step. `pod install` pulls the native SDK with the plugin.

Requirements: Flutter 3+, Android `minSdk 24` (arm64-v8a) with Kotlin 2.2+, iOS 15.1+. For live camera add the camera permission to `AndroidManifest.xml` and `NSCameraUsageDescription` to `Info.plist` (the example uses `permission_handler`). For iOS App Store uploads also set `ITSAppUsesNonExemptEncryption` to `NO` in `Info.plist`, since the SDK only encrypts its own bundled models, which is export-compliance exempt.

Distribute Android as an App Bundle (AAB). The SDK ships `arm64-v8a` only, so Play hides it from 32-bit-only devices, and it does not run on x86_64 emulators.

## Initialize

Call once at startup with your license key. It runs natively on a background thread.

```dart
import 'package:nosmai_moderation_sdk/nosmai_moderation_sdk.dart';

final res = await NosmaiModeration.initialize('NOSMAI-XXXX');
if (!res.success) {
  debugPrint('Nosmai init failed: ${res.error}');
}
```

## Moderate an image

Pass a file path (for example from `image_picker`). Returns a `NosmaiResult`.

```dart
final r = await NosmaiModeration.analyzeImage(file.path);
if (r.isUnsafe) {
  // r.nsfw -> safe / warn / block
  print('nsfw: ${r.nsfw}');
}
```

## Moderate a video

Samples one frame every `frameIntervalMs` and aggregates.

```dart
final v = await NosmaiModeration.analyzeVideo(file.path, 500);
if (v.isUnsafe) {
  print('categories: ${v.categories}, frames: ${v.framesAnalyzed}');
}
```

## Moderate text

Call `initialize` first (it validates the license), then load the text model once (it is larger), then moderate messages.

```dart
await NosmaiModeration.initializeText();          // after initialize()

final t = await NosmaiModeration.moderateText('some message');
if (t.blocked) {
  // t.layer (blocklist or classifier), t.category, t.matchedWord
  print('blocked: ${t.category?.name}');
}
```

## Live camera

The camera, frame capture and detection all run natively. Only the per-frame `NosmaiResult` crosses to Dart.

```dart
import 'package:permission_handler/permission_handler.dart';

// 1. show the native preview
const NosmaiCameraPreview();

// 2. start and listen, after granting camera permission
await Permission.camera.request();
final sub = NosmaiLive.results().listen(
  (r) => setState(() => _verdict = r.isUnsafe ? 'UNSAFE' : 'SAFE'),
  onError: (e) => debugPrint('camera failed to start: $e'),  // permission or no camera
);
await NosmaiLive.start(facing: NosmaiCameraFacing.back);  // .front is also supported, and
                                                          // falls back to the other if unavailable

// 3. on leave, always stop. The camera does not stop itself on widget dispose.
await NosmaiLive.stop();
sub.cancel();
```

## Thresholds

Adjust the NSFW bars at runtime (lower is stricter).

```dart
NosmaiModeration.setNsfwThreshold(NosmaiNsfwClass.explicit, 0.45); // BLOCK
NosmaiModeration.setNsfwThreshold(NosmaiNsfwClass.sexy, 0.55);     // WARN
```

## Cleanup

```dart
NosmaiModeration.shutdown();
```

`analyzeImage`, `analyzeVideo`, `moderateText` and `initialize` are async. They run native inference off the platform thread, so the UI never blocks.

---

# React Native Platform Guide

Source: https://docs.nosmai.com/docs/moderations/react-native/

## Install

```sh
npm install @nosmai/moderation-react-native
```

iOS. No extra step. The pod depends on `NosmaiModerationSDK`, so `cd ios && pod install` pulls the native SDK and its protected models. Set the Podfile platform to `15.1`, keep `ENABLE_USER_SCRIPT_SANDBOXING = NO` (the SDK is a static xcframework), and add `NSCameraUsageDescription` and `ITSAppUsesNonExemptEncryption = NO` to `Info.plist`.

Android. The native SDK ships as a large AAR that the host app provides (the plugin references it at compile time only). Download `nosmai-detection.aar` from the Android releases into `android/app/libs/`, reference it, and use Kotlin 2.2.0+:

```gradle
android {
  defaultConfig {
    minSdkVersion 24
    ndk { abiFilters "arm64-v8a" }
  }
}
dependencies {
  implementation files("libs/nosmai-detection.aar")
}
```

Requirements: React Native with the New Architecture (TurboModules), Android `minSdk 24` (arm64-v8a) with Kotlin 2.2+, iOS 15.1+. Distribute Android as an App Bundle (AAB); the SDK ships `arm64-v8a` only.

## Initialize

Call once at startup, loading only the models you need. It runs natively on a background thread.

```tsx
import { NosmaiModeration } from '@nosmai/moderation-react-native';

const res = await NosmaiModeration.initialize('NOSMAI-XXXX', ['nsfw']);
if (!res.success) {
  console.warn('Nosmai init failed:', res.error);
}
```

## Moderate an image

Pass a local file path (for example from `react-native-image-picker`). Returns a `NosmaiResult`.

```tsx
const r = await NosmaiModeration.analyzeImage(filePath);
if (r.isUnsafe) {
  // r.nsfw -> 'safe' | 'warn' | 'block'
  console.log('nsfw:', r.nsfw);
}
```

## Moderate a video

Samples one frame every `frameIntervalMs` and aggregates.

```tsx
const v = await NosmaiModeration.analyzeVideo(filePath, 500);
if (v.isUnsafe) {
  console.log('categories:', v.categories, 'frames:', v.framesAnalyzed);
}
```

## Moderate text

Load the text model once after `initialize`, then moderate messages.

```tsx
await NosmaiModeration.initializeText();            // after initialize()

const t = await NosmaiModeration.moderateText('some message');
if (t.blocked) {
  // t.layer -> 'blocklist' | 'classifier', t.category, t.matchedWord
  console.log('blocked:', t.category, t.layer, t.matchedWord);
}
```

## Live camera

Render `<NosmaiCameraPreview />` to open the camera and stream per-frame results. Mounting starts it, unmounting stops it. Only the per-frame `NosmaiResult` crosses into JS. Request the camera permission before mounting.

```tsx
import { NosmaiCameraPreview } from '@nosmai/moderation-react-native';

<NosmaiCameraPreview
  style={{ flex: 1 }}
  facing="back"                 // or "front"
  onResult={(r) => setVerdict(r.isUnsafe ? 'UNSAFE' : 'SAFE')}
/>
```

## Thresholds

Adjust the NSFW bars at runtime (lower is stricter).

```tsx
NosmaiModeration.setNsfwThreshold('explicit', 0.45); // BLOCK
NosmaiModeration.setNsfwThreshold('sexy', 0.55);     // WARN
```

## Cleanup

```tsx
NosmaiModeration.shutdown();
```

`initialize`, `analyzeImage`, `analyzeVideo`, `moderateText` and `initializeText` are async and run native inference off the JS thread. Type enums (`NosmaiModel`, `NosmaiNsfwVerdict`, ...) are plain strings, matching the iOS, Android and web SDKs.

---

# Live Streaming (Agora)

Source: https://docs.nosmai.com/docs/moderations/live-streaming/

Nosmai moderates a live stream by tapping each captured video frame and running it through the SDK on-device, in real time. It works with any streaming SDK that exposes raw frames (Agora, LiveKit, WebRTC). Nothing about the moderation leaves the device; only the streaming SDK's own media uses the network.

The pattern is the same everywhere: start the Nosmai stream with a listener (mobile) or an analysis loop (web); tap each captured frame from the streaming SDK; push it to Nosmai (`pushFrame` on mobile, `analyzeImage` on web); act on the SAFE / UNSAFE verdict. The tap is read-only, so the outgoing stream is never modified. Push a sampled subset of frames (about 3/sec), and do the conversion off the capture thread with drop-if-busy so the preview stays smooth.

## Example apps

Full working Agora + Nosmai examples for every platform: https://github.com/nosmai/moderation-agora-examples (folders: `android/`, `ios/`, `web/`, `flutter_example/`).

## iOS (Swift + Agora)

Become Agora's `AgoraVideoFrameDelegate`, request BGRA frames, and push each captured buffer.

```swift
NosmaiSDK.startStream(listener: self)   // self conforms to NosmaiListener

func getVideoFormatPreference() -> AgoraVideoFormat { .cvPixelBGRA }
func getRotationApplied() -> Bool { true }

func onCapture(_ videoFrame: AgoraOutputVideoFrame, sourceType: AgoraVideoSourceType) -> Bool {
  if frameCount % 5 == 0, let buffer = videoFrame.pixelBuffer {
    NosmaiSDK.pushFrame(buffer, rotationDegrees: 0)
  }
  return true
}

func nosmaiOnResult(_ result: NosmaiResult) { /* result.isUnsafe, result.nsfw, ... */ }
```

## Android (Kotlin + Agora)

Register an `IVideoFrameObserver`, convert each I420 frame to a bitmap off the capture thread, and push it.

```kotlin
NosmaiSDK.startStream(object : NosmaiListener {
  override fun onResult(result: NosmaiResult) { /* result.isUnsafe, ... */ }
})

rtcEngine.registerVideoFrameObserver(object : IVideoFrameObserver {
  override fun onCaptureVideoFrame(sourceType: Int, frame: VideoFrame?): Boolean {
    if (frameCount % 5 == 0) worker.execute {          // off-thread, drop-if-busy
      NosmaiSDK.pushFrame(i420ToBitmap(frame!!.buffer.toI420()), 0)
    }
    return true
  }
  override fun getVideoFrameProcessMode() = IVideoFrameObserver.PROCESS_MODE_READ_ONLY
  override fun getVideoFormatPreference() = IVideoFrameObserver.VIDEO_PIXEL_I420
})
```

## Web (JavaScript + Agora)

Read the Agora local track into a `<video>` element and analyze it on a loop.

```js
import AgoraRTC from "agora-rtc-sdk-ng";
import { NosmaiModeration } from "@nosmai/moderation-web";

await NosmaiModeration.initialize("NOSMAI-XXXX");
await NosmaiModeration.initializeNsfw({ modelUrl: "/models/nsm_nsfw.onnx" });

const cameraTrack = await AgoraRTC.createCameraVideoTrack();
await client.publish([cameraTrack]);
video.srcObject = new MediaStream([cameraTrack.getMediaStreamTrack()]);
await video.play();

setInterval(async () => {
  const r = await NosmaiModeration.analyzeImage(video);   // any CanvasImageSource
  render(r.isUnsafe ? "UNSAFE" : "SAFE");
}, 1000);
```

## Flutter (Agora bridge)

The `moderation_agora_bridge_flutter` package does the native frame plumbing, so your existing Agora code changes by one line. It ships no SDK of its own; results come from `nosmai_moderation_sdk`. Install it from GitHub alongside `agora_rtc_engine` and `nosmai_moderation_sdk`:

```yaml
dependencies:
  moderation_agora_bridge_flutter:
    git:
      url: https://github.com/nosmai/moderation_agora_bridge_flutter.git
```

```dart
await NosmaiModeration.initialize('NOSMAI-XXXX', models: [NosmaiModel.nsfw]);
await NosmaiLive.startExternal();
final sub = NosmaiLive.results().listen((r) { /* r.isUnsafe, r.nsfw, ... */ });

// The only changed line vs a plain Agora setup:
final handle = await ModerationAgoraBridge.getNativeHandle(agoraAppId: appId);
final engine = createAgoraRtcEngine(sharedNativeHandle: handle);
// ... your ordinary Agora initialize / join / preview is unchanged ...
// Teardown: await NosmaiLive.stop(); await ModerationAgoraBridge.disposeNative();
```

The same tap works for any raw-frame source (a token server, screen share, or another WebRTC SDK), not just Agora.

---

# Errors and Troubleshooting

Source: https://docs.nosmai.com/docs/moderations/errors/

## Initialization

`init` / `initialize` reports a failure when the SDK cannot start. The common reasons:

| Reason | What happened | What to do |
| --- | --- | --- |
| Invalid key | The license key is wrong, or not registered for this app and platform | Check that the key, package name or bundle id, and platform match your dashboard |
| Expired license | The license is no longer active | Renew it in the dashboard |
| First launch offline | No cached license yet and the device has no connectivity | Retry once the device is online |
| Offline access unavailable | The SDK requires online revalidation | Reconnect to revalidate |

Always check the return value and do not moderate until initialization succeeds. A missing or invalid license means the SDK will not run.

## License error codes

When the licensing service rejects the verify request during `init` / `initialize`, it returns one of these codes. The SDK turns them into an initialization failure, and for a definitive rejection it treats the license as invalid. A `4xx` code means a real key or license problem to fix; `429` is transient, so retry with backoff.

| Code | HTTP | Meaning |
| --- | --- | --- |
| `MISSING_REQUIRED_FIELDS` | 400 | A required field is missing from the request. |
| `API_KEY_INVALID` | 401 | Key not found, wrong, or revoked. |
| `PRODUCT_MISMATCH` | 403 | The key is not a Moderation SDK key. |
| `PLATFORM_MISMATCH` | 403 | The key is for a different platform (iOS vs Android). |
| `PACKAGE_ID_MISMATCH` | 403 | The key is for a different app (package name / bundle id). |
| `LICENSE_EXPIRED` | 403 | The license or subscription has expired. |
| `MAU_LIMIT_EXCEEDED` | 403 | The monthly active user / device limit was reached. |
| `INVALID_SDK_VERSION` | 403 | This SDK version is not allowed. |
| `PROJECT_INACTIVE` | 403 | The project is disabled. |
| `RATE_LIMIT_EXCEEDED` | 429 | Too many requests. Retry with backoff. |

## Text moderation

`initText` / `initializeText` fails if you call it before a successful `init` / `initialize`, since the license must be validated first. Call init, confirm it succeeded, then initialize text.

If your license does not include text moderation, text initialization does not start and `moderateText` returns a safe result.

## Empty or always-safe results

- Everything is safe. Confirm initialization succeeded. If it failed, every check is treated as safe.
- A category never appears. Confirm that category is included in your license. Categories outside your plan are never reported.
- A moderation type does nothing. Confirm that type (image, video, text or live) is included in your license.

## Live camera

- No results. Make sure you started the stream and that camera permission is granted.
- Rotated or sideways frames. Pass the correct rotation when you push frames (see the platform guide).
- Preview stutters on weak devices. Lower the performance mode so the NSFW check runs less often.

## Platform notes

- Android. The SDK ships `arm64-v8a` only. Distribute as an App Bundle so 32-bit-only devices and x86_64 emulators are handled correctly, and test on a physical arm64 device.
- iOS. Set `ITSAppUsesNonExemptEncryption` to `NO` in `Info.plist` so App Store and TestFlight uploads are not blocked on the encryption question.