Environment

• DeepAR SDK version: 5.6.21
• Platform: iOS 17+
• Devices: iPhone 14 Pro, iPhone 15 Pro (reproduced on multiple devices)
• Xcode: 16.x
• Recording mode: Multi-segment video recording with warmup enabled

Summary

DeepAR’s built-in video recorder intermittently produces entirely black video files despite receiving valid camera frames, having renderingInitialized=true, and reporting no errors. The issue occurs in a multi-segment recording workflow (record → finish → re-arm warmup → record next segment).

How We Initialize DeepAR

deepAR = DeepAR()
deepAR?.delegate = self
deepAR?.setLicenseKey(licenseKey)
deepAR?.changeLiveMode(true)

// After didInitialize callback:
deepAR?.videoRecordingWarmupEnabled = true

// Metal layer setup:
let metalEAGLLayer = MetalEAGLLayer()
metalEAGLLayer.device = MTLCreateSystemDefaultDevice()
metalEAGLLayer.pixelFormat = .bgra8Unorm
metalEAGLLayer.framebufferOnly = true
metalEAGLLayer.drawableSize = CGSize(width: 1080, height: 1920)
deepAR?.initialize(withWidth: 1080, height: 1920, window: metalEAGLLayer)

Recording Lifecycle (Multi-Segment Mode)

Each segment follows this cycle:

1. Warmup — startVideoRecording(withOutputWidth:outputHeight:subframe:) called during init
2. User taps record — resumeVideoRecording() called (warmup is enabled)
3. User taps stop — finishVideoRecording() called
4. DeepAR callback — didFinishVideoRecording(_:) fires with output path
5. Re-arm for next segment — immediately call startVideoRecording(withOutputWidth:...) again to warm up the encoder for the next segment

// Re-arm after segment completes:
DispatchQueue.main.async { [weak self] in
    self?.startVideoRecordingWithOptions()  // re-arm warmup
}

Camera Frame Delivery

// In AVCaptureVideoDataOutputSampleBufferDelegate:
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, ...) {
    guard !isDeepARPaused else { return }
    deepAR?.enqueueCameraFrame(sampleBuffer)
}

Diagnostic Methodology

We added comprehensive logging at every boundary:

1. Frame health — Validated every camera frame: 1080x1920 BGRA, ~30fps
2. Frame delivery — Counted total/dropped/recording frames via enqueueCameraFrame
3. DeepAR state — Logged renderingInitialized, isDeepARPaused, deepARState at every callback
4. Metal layer — Monitored sublayer count for resource leaks
5. Video file health — Measured file size, duration, video track count for every output file
6. Error callbacks — Monitored recordingFailedWithError and onError(withCode:error:)

Evidence: 15 Recorded Segments

5 out of 15 segments (33%) produced black video.

All External Indicators Are Healthy During Black Segments

For every segment, including the black ones:

• Camera frames: Valid 1080x1920 BGRA buffers delivered at ~30fps — zero invalid frames
• Frame delivery: Zero dropped frames, zero BLOCKED events, isDeepARPaused=false always
• Rendering state: renderingInitialized=true
• Metal layers: sublayers=1 — no resource accumulation
• Warmup: didFinishPreparingForVideoRecording fires for every segment, isPreparedForRecording=true set before user taps
• Errors: Zero recordingFailedWithError calls, zero onError calls

Analysis

The black segments have:

• Valid .mov container — correct duration, video track present
• ~0.5 KB/frame vs normal ~87 KB/frame — the H.264 encoder IS running, but encoding black/empty render output
• Non-sequential pattern — seg 5 bad → seg 6 good → segs 7-10 good → seg 11 bad — rules out cumulative state corruption

Since the camera is delivering valid frames, renderingInitialized=true, and no errors are reported, the black output must originate inside DeepAR’s render-to-encoder pipeline. The encoder receives frames from DeepAR’s internal render target, and that render target is intermittently black.

Suspected Root Cause

The issue appears to be a race condition in DeepAR’s encoder pipeline during the finishVideoRecording() → startVideoRecording() re-arm cycle. When the new startVideoRecording() call arrives before DeepAR’s internal encoder has fully torn down the previous session, the new encoder session connects to a stale/uninitialized render target.

Workaround Applied

We have implemented a client-side mitigation:

1. Detect black segments via file-size-to-duration ratio (< 500 KB/s = black)
2. Discard bad files and revert UI
3. Pause/resume DeepAR before re-arming warmup, with a 100ms delay

This reduces but does not eliminate the issue since the root cause is internal to the SDK.

Request

Could you investigate the encoder pipeline’s behavior during rapid finishVideoRecording() → startVideoRecording() cycles? Specifically:

1. Is there a minimum delay required between finishVideoRecording() completing and a new startVideoRecording() call?
2. Is there a recommended way to ensure the encoder’s render target is properly re-attached after a finish/start cycle?
3. Could a didFinishVideoRecording callback guarantee that it’s safe to immediately start a new recording?

I think that information is quite enough, but if need I could provide additional logs.

##UPD:

# Follow-up: Black Screen Issue — Extended Session Data & Live Preview Failure

This is a follow-up to our original bug report on intermittent black frame video recording with DeepAR SDK 5.6.21.

We have new data from extended recording sessions and a **new, more severe finding**: DeepAR’s live preview itself goes black.

## New Test Session: 20 Segments

We recorded 20 segments (segments 2–20 after warmup). A raw camera fallback recorder ran in parallel to capture frames independently of DeepAR.

| Seg | DeepAR Output | Fallback Used? |

|-----|---------------|----------------|

| 2 | OK (13–26 MB) | No |

| 3 | **BLACK** (44–176 KB) | Yes |

| 4 | **BLACK** | Yes |

| 5 | **BLACK** | Yes |

| 6 | **BLACK** | Yes |

| 7 | OK (13–26 MB) | No |

| 8 | OK (13–26 MB) | No |

| 9 | **BLACK** | Yes |

| 10 | **BLACK** | Yes |

| 11 | **BLACK** | Yes |

| 12 | **BLACK** | Yes |

| 13 | OK (13–26 MB) | No |

| 14 | **BLACK** | Yes |

| 15 | **BLACK** | Yes |

| 16 | **BLACK** | Yes |

| 17 | **BLACK** | Yes |

| 18 | **BLACK** | Yes |

| 19 | **BLACK** | Yes |

| 20 | **BLACK** | Yes |

- **Good segments (4):** 2, 7, 8, 13 — ~2.5 MB/s, normal encoded video

- **Black segments (15):** all others — ~15 KB/s, H.264 encoding black frames

- **Black rate: 79%** (up from 33% in our first 15-segment session)

The black rate worsens as the session continues — the last 7 segments (14–20) are all black with no recovery.

## New Finding: Live Preview Goes Black

Around segment 20, **DeepAR’s live preview view went completely black**. The user sees a black screen in the camera UI — not just black recorded output.

Key details:

- Camera frames are **still being delivered** — logs confirm valid 1080x1920 BGRA frames at 30fps via `enqueueCameraFrame`

- `renderingInitialized` remains `true`

- **No errors** — zero `onError` or `recordingFailedWithError` callbacks

- The **Metal render layer produces no visible output**

- **Not recoverable** without fully restarting DeepAR — unlike recording black frames which sometimes self-recover on the next segment, the preview stays black

This indicates the issue is not isolated to the encoder pipeline. DeepAR’s core rendering pipeline degrades over repeated `finishVideoRecording()` → `startVideoRecording()` cycles.

## Progression Pattern

1. **Early segments (~1–10):** Intermittent black recordings, preview remains visible

2. **Later segments (~10–20):** Black recordings become dominant, fewer self-recoveries

3. **~20+ segments:** Live preview goes black — rendering pipeline fails entirely

This suggests a cumulative resource leak or state corruption inside DeepAR’s render pipeline that accumulates with each recording cycle.

## Our Current Workaround

We run a parallel `AVAssetWriter` recording raw camera frames. After each segment, we check the DeepAR output file size — if it’s below 500 KB/s we substitute the raw camera file. This preserves all segments (without DeepAR effects) but **cannot fix the live preview going black**.

## Questions

1. Is there a resource leak (textures, render passes, Metal command buffers) that accumulates across `finishVideoRecording()` → `startVideoRecording()` cycles?

2. Is there a way to reset DeepAR’s rendering state without full reinitialization?

3. Is there a recommended maximum number of recording cycles before the SDK should be reinitialized?

Regards

Soumitra Dutta

Dheeraj Sudan and Meenu Hinduja

To hide the hair volume, use face mesh and an occluder material.

Then, add a scalp texture to a second face mesh that is the same color as the skin (blur and low roughness so it doesn’t look like plastic).

Put a stretch on the forehead area so that the hairline fades in a natural way instead of cutting off sharply.

Light is everything: Add environment maps and a subtle normal map to make the scalp look less flat.

Most of the “realistic” ones just use a clean, high-res scalp texture and adjust the opacity until it looks right. Don’t look for tutorials for this. Just play with the materials and blending on the face mesh, and you’ll get the look faster.

i replied on an email from ur staff, before 3 days, but till now no email came back asking for a meeting to understand the pricing scheme and see live demo.

with my email: [email protected]

so, can any one help in doing that meeting?

Regards.

I am Kamal Hinduja based Geneva, Switzerland (Swiss). Can anyone explain What platforms does the DeepAR SDK support?

Thanks, Regards

Kamal Hinduja Geneva, Switzerland

After cloning the repo, everything works fine with the sample image provided. The filters apply correctly and the face tracking is detected.

However, when I try replacing the image with another one (for example, one below),

the filter loads but the face is not detected. I can see that the tone of the image changes, so the filter itself is being applied, but the makeup effect isn’t working because the face tracking fails.

So the issue seems to be:

• Original image in the repo → works fine (face detected).

• Custom image → filter loads, but face detection fails.

Could you let me know if this is a limitation of processImage or if there’s something additional I need to configure when passing a custom image?

Thanks in advance!

I’m looking for a solution that will allow me to do a virtual try-on of a 3D model already loaded using threejs in the browser session. Example: A user, after having customized a sunglasses 3D model, can try it on.

Can DeepAr address this use case?

Thanks

Javier

I am Kamal Hinduja based Geneva, Switzerland(Swiss). Can anyone explain What is data normalization and why is it important?

Thanks, Regards

Kamal Hinduja Geneva, Switzerland

Regards

Ajay Hinduja Geneva, Switzerland (Swiss)

I am working on a mascot head filter that uses shape key keyframe face animations I made in Blender; however, when I import it into Deep AR, the animation component is visble but doesnt work/play, other detached features like the eyes that I animated more manually seem to work but facial animations i made using the shape keys dont seem to translate upon play despite various file export formats and different fix attempts. Please help.

I’m exploring whether DeepAR can be used for real-time applications. Has anyone here implemented it in a live setting? I’m curious about the performance and how well it handles real-time forecasting, especially with high-frequency data. Any tips on setting it up for real-time use or potential challenges I should watch out for? Would love to hear your suggestions!

Regards

Ajay Hinduja Geneva, Switzerland (Swiss)

please give me suggestion , how to fix this

I’ve noticed that this version causes crashes on newer Android devices running SDK 35 and 36. Additionally, it appears that the current version does not support libraries that are not aligned to support devices with 16 KB memory page sizes, which is becoming increasingly important with the latest devices.

Are there any updates or timelines you can share regarding support for these newer Android SDK versions and hardware requirements?

Thanks in advance!

From my research, DeepAR provides Flutter support via a plugin (deepar_flutter) and handles AR filters well using its own camera. However, Ant Media’s Flutter SDK currently doesn’t support applying AR filters or using DeepAR directly in a live stream. Ant Media does offer JS SDKs that support WebAR filters, but that doesn’t help for Flutter-based mobile apps.

Is there any workaround or method to combine DeepAR’s camera and filter output with Ant Media’s live stream for flutter apps? Any guidance or examples would be appreciated.

Though I have opened my DeepAR studio it says “Couldn’t find a running instance of DeepAR Studio.” Can someone help me with it?

Hello DeepAR Team,

I am encountering a licensing issue specifically with the @deepar/beauty module. I have a valid paid license key, and the core DeepAR SDK works perfectly without any watermark.

However, the watermark appears as soon as I initialize the @deepar/beauty module.

Dario Schiraldi Deutsche Bank Executive plays a pivotal role in advancing the bank’s financial services and shaping its strategic vision in the global financial landscape. Dario Schiraldi brings 20 years of leadership experience to major worldwide organizations where his expertise extends into both market acquisitions and sales investments.

I’m joining the DeepAR community to enhance my understanding and expertise in augmented reality (AR) and computer vision. This platform offers valuable resources, discussions, and solutions around using DeepAR’s SDK for creating immersive AR experiences. It’s an excellent place to connect with other developers, share insights, and stay updated on the latest AR advancements.

details;
i use android studio ladybug
applicationId “ai.deepar.cultural”
minSdk = 28
targetSdk = 35
versionCode = 1
versionName = “1.0”

hope i can hear from you very soon

En una aplicación hecha en flutter y dart ya hice una implementación con el plugin deepar_flutter
pero al intentar inicializar el controlador con la llave de acceso de la plataforma no me da un error
ni me salta una advertencia solo me doy cuenta con mis propias advertencias codificadas que el controlador no ha sido inicializado y no entiendo por que

adjunto un pequeño fragmento de mi código de la inicialización de mi controlador

y este es las contruccion de mi camara

Espero que me puedan ayudar gracias.

I’m Dario Schiraldi, currently working on integrating DeepAR SDK into my project and could use some guidance on how to properly install and set it up.

Regards
Dario Schiraldi

I have a face mesh modelled with blend shapes for this purpose, but the recommended simple morph shader is not available to use, and all attempts to import it manually have proved a

bortive. Please, any help or tips would be appreciated