This reference covers @nosmai/[email protected]. In method tables, sdk means Nosmai.instance. Use the installation guide for runtime assets and the quickstart for a complete camera app.
Initialization and lifecycle
import { Nosmai, SDK_VERSION } from '@nosmai/web-sdk'
console.log('Nosmai Web SDK:', SDK_VERSION)
const result = await Nosmai.initialize('NOSMAI-YOUR-WEB-API-KEY', {
assetBase: '/nosmai',
})
if (!result.ok) {
throw result.error ?? new Error('Nosmai initialization failed')
}
const sdk = Nosmai.instance
const canvas = document.querySelector<HTMLCanvasElement>('#preview')
if (!canvas) throw new Error('Mount the preview canvas first')
await sdk.attach(canvas)
await sdk.camera.start({ position: 'front' })
| Signature or property | Result | Contract |
|---|---|---|
Nosmai.initialize(licenseKey: string, options?: InitializeOptions) | Promise<InitializeResult> | Configures the shared instance. options.assetBase is the public URL containing engine/, engine-baseline/, and models/. |
Nosmai.instance | Shared Nosmai instance | Use one active preview per instance. |
SDK_VERSION | string | The installed SDK version; 1.0.0 for this reference. |
sdk.attach(canvas: HTMLCanvasElement) | Promise<void> | Loads the runtime and models and starts licence verification. The canvas must already be in the document. |
sdk.dispose() | void | Stops capture and releases the preview runtime. Initialize and attach again for a new session. |
sdk.isInitialized | boolean | Whether SDK configuration has been accepted. |
sdk.isProcessing | boolean | Whether the camera processing loop is running. |
sdk.frameTimeMs | number | Latest CPU wall time in milliseconds for frame upload, processing, and render submission. GPU completion time is excluded. |
InitializeOptions has one optional field, assetBase?: string. Set it explicitly to your public asset directory. InitializeResult contains ok: boolean, licence: LicenceState, and optional error: NosmaiError. A successful configuration result precedes licence verification. Observe licenseStatusChanged or read sdk.licenceState after attaching the canvas.
Initialization is idempotent. To change the key or asset base, end the current session with dispose() before initializing again. Await asynchronous startup operations and release the SDK when its owning page or screen is finished.
Camera
| Signature or property | Result | Contract |
|---|---|---|
sdk.camera.start(options?: CameraOptions) | Promise<void> | Opens the camera or supplied stream and starts processing. Stop an existing capture before starting a different source. |
sdk.camera.stop() | void | Stops processing and releases camera tracks opened by the SDK. Retains the attached runtime for restart. |
sdk.camera.switchCamera() | Promise<boolean> | Switches between front and back. Resolves false when capture is stopped or switching fails; check isRunning before retrying. |
sdk.camera.setMirror(mode: MirrorMode) | void | auto mirrors the front camera; on and off override it. Default: auto. |
sdk.camera.list() | Promise<MediaDeviceInfo[]> | Lists video input devices. Device labels depend on browser permission. |
sdk.camera.isRunning | boolean | Whether capture is running. |
sdk.camera.position | CameraPosition | Current front or back position. |
CameraOptions fields:
| Field | Type | Default and meaning |
|---|---|---|
position | CameraPosition | front initially; later starts retain the last position when omitted. |
width | number | 1280; an ideal camera constraint, subject to browser and device support. |
height | number | 720; an ideal camera constraint. |
stream | MediaStream | Optional external source used instead of opening the device camera. The caller owns and stops its tracks. |
CameraPosition is front or back. MirrorMode is auto, on, or off. The SDK opens device cameras with video only. Supply your own stream when your application needs an existing capture or an audio track.
Effects and parameters
| Signature or property | Result | Contract |
|---|---|---|
sdk.effects.apply(source: string | Blob | ArrayBuffer | Uint8Array) | Promise<void> | Applies a Web-compatible .nosmai package. A string is fetched as a URL; a browser File is accepted as a Blob. |
sdk.effects.clear() | Promise<void> | Removes the loaded package. |
sdk.effects.isActive | boolean | Whether a package is active. |
sdk.effects.activeType | string | The active package type, or an empty string when none is loaded. |
sdk.effects.active | NosmaiEffect | null | Optional effect metadata. Can remain null while a package is active; use isActive for runtime state. |
sdk.effects.parameters() | EffectParameter[] | Declared parameters of the loaded effect, or an empty array. |
sdk.effects.setParameter(name: string, value: number) | boolean | Updates a numeric parameter; false if the parameter is unavailable. |
sdk.effects.setParameterString(name: string, value: string) | boolean | Updates a string parameter; false if unavailable. |
sdk.effects.getParameter(name: string) | number | Reads a numeric value; returns NaN when unavailable. |
sdk.effects.getParameterString(name: string) | string | Reads a string value; returns an empty string when unavailable. |
sdk.effects.pause() / sdk.effects.resume() | void | Pauses or resumes the package's own runtime and audio. Camera capture continues. |
Attach the canvas before applying an effect. Handle rejection from apply and check the rendered result in your target browsers. Await effect changes before updating controls that depend on the new package.
EffectParameter includes name, type, displayName, description, hasRange, minValue, maxValue, passId, currentValue, defaultValue, and options. Values can be numbers, strings, or number arrays as declared by the effect. Use this metadata to build controls with the effect's own ranges.
Colours and value ranges
Methods accepting Rgb | string support hex colours such as '#c45a72', or { r, g, b } with channels from 0 to 1. rgbFromHex(hex: string): Rgb converts a hex colour to normalized channels.
Beauty, makeup, hair, background blur, and LUT intensities use 0..1. Normalized reshape and signed colour adjustments use -1..1 with 0 neutral. Raw colour and reshape methods have their own units, described below.
Beauty
Call these methods after attaching the preview and confirm that your licence grants the required feature access.
| Signature or property | Result | Range or meaning |
|---|---|---|
sdk.beauty.setSkinSmoothing(level: number) | void | 0..1; 0 removes smoothing. |
sdk.beauty.setSkinWhitening(level: number) | void | 0..1. |
sdk.beauty.setTeethWhitening(level: number) | void | 0..1. |
sdk.beauty.setSharpening(level: number) | void | 0..1. |
sdk.beauty.setContourHighlighter(contour: number, highlight?: number) | void | 0..1 strengths. Supply both arguments for separate contour and highlight control. |
sdk.beauty.setFoundation(coverage: number, colour: Rgb | string) | void | Coverage 0..1; 0 removes foundation. |
sdk.beauty.setFreckles(intensity: number) | void | 0..1. |
sdk.beauty.setDarkCircleCorrector(level: number) | void | 0..1 under-eye correction. |
sdk.beauty.setEyeColor(colour: Rgb | string, intensity?: number) | void | Intensity 0..1, default 0.5; 0 removes the tint. |
sdk.beauty.setEyeColorIntensity(intensity: number) | void | Adjusts an existing eye tint. Set its colour first. |
sdk.beauty.setVisible(visible: boolean) | void | Shows or hides face beauty, reshape, and hair tint while retaining their settings. |
sdk.beauty.clear() | void | Clears manual beauty controls. |
sdk.beauty.isVisible | boolean | Current beauty visibility. |
sdk.beauty.isReady | boolean | Model readiness after loading and warm-up; use faceDetected for face presence. |
sdk.beauty.isEyeColorActive | boolean | Whether an eye tint is active. |
sdk.beauty.activeMask | number | Active-control bitmask: reshape 1, skin 2, teeth 4, makeup 8, eye tint 16. |
Reshape
| Signature or property | Result | Contract |
|---|---|---|
sdk.beauty.reshape.set(type: ReshapeType, value: number) | void | Normalized -1..1, with 0 neutral. smile uses 0..1; negative smile values are treated as 0. |
sdk.beauty.reshape.setAbsolute(type: ReshapeType, value: number) | void | Sets a raw per-axis value. RESHAPE_RANGE[type] provides that axis's slider magnitude. |
sdk.beauty.reshape.clear() | void | Resets all reshape axes. |
sdk.beauty.reshape.isActive | boolean | Whether any reshape axis is active. |
Use the exported ReshapeType members below. RESHAPE_LABELS[type] provides each control's display label.
| Group | Members of ReshapeType |
|---|---|
| Face and head | faceSlim, jaw, chin, chinWidth, cheekbones, forehead, foreheadWidth, faceLength, headSize |
| Eyes | eyeSize, eyeSpacing, eyeSlant, eyeHeight |
| Nose | noseSize, noseWidth, noseLength, noseBridge, noseTip |
| Brows | brow, browThickness, browArch, browSpacing |
| Mouth | lipSize, upperLip, lowerLip, mouthWidth, mouthPosition, smile |
| Depth | noseTipDepth, noseBridgeHeight, noseGlobalHeight, chinDepth, midfaceDepth, mouthDepth, eyeLeftDepth, eyeRightDepth, browsDepth |
| Independent eyes | eyeLeftSize, eyeRightSize, eyeLeftDistance, eyeRightDistance, eyeLeftRotation, eyeRightRotation |
Positive noseSize values shrink the nose. For normalized set, a value of 1 means the full supported slider strength for the selected axis. Raw ranges vary: for example, RESHAPE_RANGE[ReshapeType.eyeSize] is 1.3, while RESHAPE_RANGE[ReshapeType.noseSize] is 0.5.
import { ReshapeType } from '@nosmai/web-sdk'
sdk.beauty.setSkinSmoothing(0.4)
sdk.beauty.reshape.set(ReshapeType.faceSlim, 0.25)
sdk.beauty.setEyeColor('#6c8a63', 0.3)
Makeup
| Signature | Result | Contract |
|---|---|---|
sdk.beauty.makeup.apply(layer: MakeupLayer, style: number, colour: Rgb | string, intensity?: number) | void | Intensity 0..1, default 0.5. Use a style constant for the chosen layer. |
sdk.beauty.makeup.setIntensity(layer: MakeupLayer, intensity: number) | void | Adjusts an existing layer's strength. |
sdk.beauty.makeup.isActive(layer: MakeupLayer) | boolean | Whether the layer is present. |
sdk.beauty.makeup.remove(layer: MakeupLayer) | void | Unloads one layer. |
sdk.beauty.makeup.clear() | void | Unloads all makeup layers. |
Exported layer and style constants:
| Layer | Styles |
|---|---|
MakeupLayer.lipstick | LipstickStyle.classic, LipstickStyle.matte, LipstickStyle.natural |
MakeupLayer.eyeshadow | EyeshadowStyle.smokey, EyeshadowStyle.shimmer, EyeshadowStyle.natural |
MakeupLayer.blusher | BlusherStyle.round, BlusherStyle.contour, BlusherStyle.natural |
MakeupLayer.eyelash | EyelashStyle.natural, EyelashStyle.dramatic, EyelashStyle.wispy |
MakeupLayer.eyebrow | EyebrowStyle.natural, EyebrowStyle.bold, EyebrowStyle.arched |
Style values are integers 0..2; other values throw invalidParameter. The eyelash asset supplies its own colour. Use remove to unload an existing layer when it is no longer needed.
Hair colour
| Signature or property | Result | Contract |
|---|---|---|
sdk.hair.setColor(colour: Rgb | string, intensity?: number) | void | Tints existing hair. Intensity 0..1, default 0.5. |
sdk.hair.setIntensity(intensity: number) | void | Adjusts an existing tint; call setColor first. |
sdk.hair.clear() | void | Removes hair colouring. |
sdk.hair.isEnabled | boolean | Whether hair colouring is enabled. |
These controls change hair colour. Hairstyle length and shape require different capabilities and are outside this API.
Background
| Signature | Result | Contract |
|---|---|---|
sdk.background.blur(strength: number) | void | Blur strength 0..1; 0 turns blur off. |
sdk.background.color(colour: Rgb | string, alpha?: number) | void | Colour replacement with alpha 0..1, default 1. |
sdk.background.image(source: string | Blob | HTMLImageElement | HTMLCanvasElement | ImageBitmap) | Promise<void> | Replaces the background with the supplied image. |
sdk.background.video(source: string | Blob | HTMLVideoElement) | Promise<void> | Uses muted, looping video as the background. Resolves after playback starts. |
sdk.background.clear() | void | Removes the background replacement or blur and stops background video. |
Selecting blur, colour, or an image stops a running background video. Remote images and videos must allow the browser to read their pixels through CORS. Video playback is also subject to browser autoplay policy.
Colour adjustment
| Signature | Result | Range and neutral value |
|---|---|---|
sdk.color.setBrightness(amount: number) | void | -1..1, neutral 0. |
sdk.color.setContrast(amount: number) | void | -1..1, neutral 0. |
sdk.color.setSaturation(amount: number) | void | -1..1, neutral 0. |
sdk.color.setWarmth(amount: number, tint?: number) | void | Signed warmth -1..1, neutral 0; tint defaults to 0. |
sdk.color.setRgb(r?: number, g?: number, b?: number) | void | Signed channel adjustments -1..1, each defaulting to neutral 0. |
sdk.color.setGrayscale(on: boolean) | void | Enables or disables grayscale. |
sdk.color.setLut(source: string | Blob | Uint8Array | ArrayBuffer, intensity?: number) | Promise<void> | Applies a PNG LUT; intensity 0..1, default 1. |
sdk.color.setLutIntensity(intensity: number) | void | Adjusts an existing LUT's strength, 0..1. |
sdk.color.clear() | void | Clears manual colour adjustments. |
Raw adjustment methods:
| Signature | Result | Units and neutral values |
|---|---|---|
sdk.color.setContrastMultiplier(multiplier: number) | void | Raw multiplier; neutral 1. |
sdk.color.setHsb(hueDegrees: number, saturation?: number, brightness?: number) | void | Hue in degrees; saturation and brightness multipliers default to 1. |
sdk.color.setWhiteBalance(temperatureK: number, tint?: number) | void | Kelvin; neutral 5000, tint default 0. |
sdk.color.setRgbGains(r?: number, g?: number, b?: number) | void | Raw channel gains; each defaults to neutral 1. |
Cloud Filters
| Signature | Result | Contract |
|---|---|---|
sdk.cloud.list(options?: { category?: EffectCategory; page?: number; limit?: number }) | Promise<CloudPage> | Lists effects accessible to the key. Supply pagination explicitly when needed. |
sdk.cloud.download(effectId: string, onProgress?: (fraction: number) => void) | Promise<Uint8Array> | Downloads an effect or reads its cached bytes. |
sdk.cloud.apply(effectId: string, onProgress?: (fraction: number) => void) | Promise<void> | Downloads and applies the effect. |
sdk.cloud.isDownloaded(effectId: string) | Promise<boolean> | Checks whether the browser cache contains the effect. |
sdk.cloud.downloadedIds() | Promise<string[]> | Returns cached effect IDs. |
sdk.cloud.clearCache(effectId?: string) | Promise<void> | Removes one cached download, or all downloads when omitted. A running effect stays loaded. |
Wait for licence verification before calling list, download, or apply. These operations require the cloudFilters entitlement and reject with NosmaiError.code === 'FEATURE_NOT_LICENSED' when it is unavailable.
Use NosmaiEffect.id returned by list as effectId. EffectCategory is filter, effect, background, or beautyEffect. CloudPage contains items: NosmaiEffect[], page, totalPages, total, and hasNextPage.
NosmaiEffect includes id, name, displayName, category, source, isFree, isDownloaded, and optional url, previewUrl, and fileSize. Check isDownloaded(id) for current cache availability. The browser manages cached storage and may evict downloads.
Progress callbacks report fractions from 0 to 1 when download progress is available; a callback is not guaranteed when the response omits its size or the effect is already cached.
if (sdk.isCloudFilterEnabled) {
const page = await sdk.cloud.list({ category: 'effect', page: 1, limit: 20 })
const first = page.items[0]
if (first) await sdk.cloud.apply(first.id)
}
Processed stream output
| Signature or property | Result | Contract |
|---|---|---|
sdk.output.stream() | MediaStream | Acquires a video-only stream containing the processed preview. |
sdk.output.videoTrack() | MediaStreamTrack | Acquires the processed video track for a streaming SDK or peer connection. |
sdk.output.isActive | boolean | Whether a consumer holds processed output. |
sdk.output.release() | void | Releases one acquisition from stream() or videoTrack(). |
Start the preview before publishing. Pair each acquisition with one release after its consumer has finished. Microphone tracks are managed separately by your application or streaming SDK. Audio played by an effect is excluded from this video output.
const processedStream = sdk.output.stream() const processedTrack = processedStream.getVideoTracks()[0] // Publish processedTrack through your streaming SDK. console.log(processedTrack.kind) // After unpublishing and removing the consumer: sdk.output.release()
Recording and photo capture
| Signature or property | Result | Contract |
|---|---|---|
recordingSupported() | boolean | Exported browser capability check for recording. |
sdk.recording.isSupported | boolean | Whether the browser exposes the required recording APIs. |
sdk.recording.isRecording | boolean | Whether a recording is in progress. |
sdk.recording.duration | number | Elapsed recording time in seconds; 0 while idle. |
sdk.recording.start(options?: RecordingOptions) | void | Starts recording the processed output. Requires an attached, running preview. |
sdk.recording.stop() | Promise<RecordingResult> | Stops and returns the recording. Check success and blob before using it. |
sdk.recording.cancel() | void | Discards the current recording. |
sdk.recording.capturePhoto(type?: string, quality?: number) | Promise<Blob> | Captures the processed preview; type defaults to image/png. Quality uses the browser's 0..1 encoding scale where supported. |
RecordingOptions accepts:
audio?: boolean: defaults totrue, including audio only when the supplied camera stream contains audio tracks. This does not request microphone access.videoBitsPerSecond?: number: browser default when omitted.mimeTypes?: string[]: ordered container preferences. By default the SDK tries supported MP4 options, then WebM options.
Effect audio is excluded from recordings. RecordingResult contains success: boolean, optional blob: Blob, duration in seconds, fileSize in bytes, optional mimeType, and optional error. Read the result's MIME type when choosing a download extension. Stop and save a recording before disposing the SDK.
Interactive games
| Signature or property | Result | Contract |
|---|---|---|
sdk.game.isReady | boolean | Whether the loaded effect has a ready game runtime. |
sdk.game.tap(x: number, y: number) | boolean | Sends a tap in normalized 0..1 coordinates, with the origin at the top left. Returns false when unavailable. |
sdk.game.input(name: string, x?: number, y?: number, value?: number) | boolean | Sends an input declared by the effect. Numeric arguments default to 0. |
sdk.game.pause() / sdk.game.resume() | void | Pauses or resumes the running game. |
sdk.game.restart() | void | Restarts the loaded game. |
Licence state and feature access
| Signature or property | Result | Contract |
|---|---|---|
sdk.licenceState | LicenceState | Current verification state and presentation flags. |
sdk.isFeatureEnabled(feature: string) | boolean | Whether the current licence grants the named feature. Can return false before verification completes. |
sdk.isCloudFilterEnabled | boolean | Convenience check for cloudFilters. |
sdk.isBeautyEffectEnabled | boolean | Convenience check for beautyEffects. |
Feature names include cloudFilters, beautyEffects, localFilters, lutFilters, and advancedFilters. Check access after verification.
LicenceState contains:
status: LicenceStatus:unverified,valid,expired, orinvalid.secondsRemaining: number.watermarked: booleanandblurred: boolean: current presentation flags.- Optional
code,message, andfeatures: Record<string, boolean>.
Events
sdk.on(event, handler) returns an unsubscribe function. Subscribe once for the owning screen and call that function when the listener is no longer needed.
| Event | Payload | Meaning |
|---|---|---|
ready | void | Camera processing has started. |
error | NosmaiError | An SDK error was reported. Also handle rejected asynchronous calls. |
licenseStatusChanged | LicenceState | The licence state changed. |
faceDetected | boolean | Sampled face-presence state changed. |
fps | number | Rendered frames per second, sampled approximately once per second. |
recordingProgress | number | Elapsed recording time in seconds. |
gameEvent | NosmaiGameEvent | Event emitted by an interactive game effect. |
NosmaiGameEvent has event, game, sequence, and data: Record<string, unknown>.
const unsubscribe = sdk.on('licenseStatusChanged', (state) => {
console.log(state.status, state.watermarked, state.blurred)
})
// When the owning screen is finished:
unsubscribe()
Errors
Import NosmaiError from @nosmai/web-sdk. Its fields are type, code, message, optional details, and optional cause. It also exposes userMessage, recoveryActions: string[], and isRecoverable: boolean.
Common NosmaiErrorType values include sdkNotInitialized, invalidParameter, invalidLicense, licenseExpired, cameraPermissionDenied, cameraUnavailable, webglUnavailable, networkError, effectNotFound, effectInvalidFormat, effectLoadFailed, recordingInProgress, and recordingWriteFailed. Other values are unknown, stateError, and platformError. The more specific code can provide additional context.
import { NosmaiError } from '@nosmai/web-sdk'
try {
await sdk.effects.clear()
} catch (error) {
if (error instanceof NosmaiError) {
console.error(error.type, error.code, error.userMessage, error.recoveryActions)
} else {
console.error(error)
}
}
React preview
React 18 or later can use NosmaiCameraPreview from @nosmai/web-sdk/react. Initialize Nosmai in the browser before mounting it. The component owns its canvas and starts the camera.
| Prop | Type | Default or meaning |
|---|---|---|
width, height | number | string | CSS dimensions, defaulting to 100% width and auto height. |
facingMode | CameraPosition | front. |
mirror | MirrorMode | auto. |
objectFit | contain | cover | contain. |
effect | string | Uint8Array | Blob | null | Defaults to null. Changes apply a package; null clears it. |
onInitialized | () => void | Called after camera startup. |
onError | (error: NosmaiError) => void | Reports preview startup and effect-loading errors. |
onGameTap | Callback with tap coordinates | Receives normalizedX, normalizedY, locationX, locationY, previewWidth, and previewHeight. Game taps are also forwarded by the component. |
className, style | string, React.CSSProperties | Canvas presentation. |
The forwarded NosmaiCameraPreviewHandle exposes reinitialize(): Promise<void>, isInitialized: boolean, and currentError: string | null. NosmaiCameraPreviewProps and NosmaiCameraPreviewHandle are exported from the React entry point.
The component stops its camera on unmount. Dispose the shared SDK when the owning session ends, and remove any event listeners your application added.