FUSA VI syncpoint error (fusaViHandler.cpp:869 "syncpoint signaled but status value not updated" / Error 8) on stock 5.15.185-tegra, L4T r36.5.0

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

Summary

On L4T r36.5.0 (JetPack 6.2.2), nvargus-daemon throws a FUSA VI handler error — fusaViHandler.cpp:869 “Status syncpoint signaled but status value not updated” → Error 8 — simultaneously on all 6 GMSL2 sensors, after which every capture session is disposed and the libargus client connection is reset. In our case the daemon does not crash (it stays alive, NRestarts=0, no core) but all sessions become permanently unusable until the client process is fully restarted. This reproduces on the stock 5.15.185-tegra kernel (non-PREEMPT_RT) via the ZED SDK / libargus client path (no GStreamer).

Related / prior reports (same defect)

Same fusaViHandler.cpp:869 / Error 8 signature; that thread documents a deadlock/wedge variant and a SIGSEGV variant with a gdb backtrace pointing at a lost wakeup in libnvscf.so CaptureScheduler. This report is the same defect on the stock (non-RT) kernel, daemon-survives form.

Same libnvscf.so binary as the prior thread

  • Build ID a4b275ab3966399e8ea8c78a9ba46a9a3542641d

  • sha256 944bf7830e342181c0e92361375753762889b643dbcaba6672d63cd002bfcd20

These match the binaries posted in #366870 — i.e. the stock L4T r36.5.0 (GCID 43688277) libnvscf.so.

Setup

| | |

|—|—|

| Platform | Jetson AGX Orin Developer Kit |

| L4T / JetPack | R36.5.0, GCID 43688277 (# R36 (release), REVISION: 5.0, 2026-01-16) |

| Kernel | 5.15.185-tegra (stock, NON-PREEMPT_RT) |

| libargus | NVIDIA Libargus Camera Service 0.99.33 |

| Capture client | ZED SDK 5.3.0 via the libargus client (NOT nvarguscamerasrc/GStreamer) |

| GMSL driver | stereolabs-zedlink-quad 1.4.2-SL-MAX96712-L4T36.5.0 (MAX96712 deser + MAX9295 ser) |

| Cameras | 3× ZED X stereo = 6 sensors, GMSL2, 1920×1080 @ 30 fps, 6 concurrent sessions |

Trigger & timing

  • Continuous capture, all 6 sensors streaming concurrently (1080p30), auto-exposure locked at session start.

  • Stochastic timing. One session hit the error ~3 minutes in (start 06:07:34, first FUSA error 06:10:40); a separate session ran 13+ minutes with no failure. So it can strike well under the 8–13 min seen elsewhere on the ZED SDK path, but it is not deterministic — consistent with a load/timing race, not a fixed-time fault.

  • When it occurs, onset is immediate and simultaneous across all 6 sensors, not gradual per-camera degradation.

nvargus-daemon log at onset


(fusa) Error: InvalidState Status syncpoint signaled but status value not updated in:/capture/src/fusaViHandler.cpp 869

(fusa) Error: InvalidState propagating from:/capture/src/fusaViHandler.cpp 811

SCF: Error InvalidState: Error 8 Received for sensor 1 ..!

(in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 664)

SCF: Error InvalidState: Sending critical error event for Session 0

(in src/api/Session.cpp, function sendErrorEvent(), line 1101)

SCF: Error InvalidState: Error 8 Received for sensor 4 ..!

SCF: Error InvalidState: Error 8 Received for sensor 5 ..!

SCF: Error InvalidState: Error 8 Received for sensor 0 ..!

SCF: Error InvalidState: Error 8 Received for sensor 3 ..!

SCF: Error InvalidState: Error 8 Received for sensor 2 ..!

... (Error 8 + waitCsiFrameEnd():664 repeats for every sensor)

SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)

libargus client (ZED SDK) side — connection lost, cannot reconnect


(Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (SocketUtils.cpp::readSocket(), line 79)

(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: Connection refused (SocketClientDispatch.cpp::openSocketConnection(), line 205)

(Argus) Error FileOperationFailed: Cannot create camera provider

After this, every Camera::grab() fails permanently on all 6 sensors (within ~1 s of each other). Because the libargus client is a process-global singleton, the only recovery is a full restart of the client process.

Key distinction: the daemon does NOT crash

systemctl show nvargus-daemon reports NRestarts=0, Result=success, no SIGSEGV and no core dump for this event — yet all sessions are disposed and the client socket is dead. This is the “wedge / sessions-UNAVAILABLE” manifestation from #366870, here on the non-RT kernel and via the ZED SDK path. It fits the lost-wakeup theory in that thread: the FUSA error path at fusaViHandler.cpp:869 does not signal downstream waiters, so sessions never recover.

Questions

  1. Does a repro on the stock 5.15.185-tegra kernel (no PREEMPT_RT) help isolate the lost wakeup? It rules out an RT-scheduler-only timing window.

  2. The nvarguscamerasrc patch referenced in #366870 — does it touch the libargus client / libnvscf CaptureScheduler path that the ZED SDK uses? We use no GStreamer yet hit the identical fusaViHandler.cpp:869 / Error 8, so a GStreamer-plugin-only fix would not help.

  3. Is there a supported way to disable the FUSA VI capture path (libnvfusacap / fusaViHandler) or set a VI syncpoint-completion timeout? We do not require functional-safety guarantees for data collection.

  4. Stock argus_camera_jpeg reportedly runs for many hours without this fault. Is the difference a capture-request pattern (buffer / output-stream count, request cadence, ISP/AE usage) we could mirror through the ZED SDK to avoid the syncpoint race? Any guidance on a safe concurrency/throughput envelope for 6× 1080p30 GMSL2 would help.

I can attach the full nvargus-daemon journal, the libargus client log, and systemctl show nvargus-daemon for this event on request.

4

hello willzeng,

please give it a quick try to disable RCE suspend (rtcpu auto-suspend) for testing..
for instance,
# echo -1 > /sys/devices/platform/*rtcpu/power/autosuspend_delay_ms
# echo on > /sys/devices/platform/*rtcpu/power/control

5

Thanks @JerryChang — we tried disabling RCE (rtcpu) auto-suspend and it did not prevent the crash for us. Sharing detailed data, because what we capture looks like a second FUSA-VI fault path that ends in the same nvargus-daemon SIGSEGV.

Setup: Jetson Orin, L4T r36.5.0, kernel 5.15.185-tegra (stock, non-RT). 3× GMSL2 stereo modules = 6 sensors, continuous 1080p30 via libargus. Reproduces under sustained multi-camera capture with auto-exposure locked, typically 8–44 min in.

rtcpu auto-suspend workaround — did not help. Applied and verified it held:


echo -1 > /sys/devices/platform/*rtcpu/power/autosuspend_delay_ms

echo on > /sys/devices/platform/*rtcpu/power/control

cat .../rtcpu/power/runtime_status # -> active (stayed active all run)

runtime_status stayed active the whole session, yet nvargus-daemon still SIGSEGV’d ~22 min in.

What the log shows (may be a different variant than the OP’s syncpoint one): a CSI short-frame/channel fault, then a “Mutex not initialized” storm from inside the FuSa error-decode path, then SEGV:


PIX_SHORT: / corr_err: notifyBits: 0x200008

FuSaCaptureViErrorDecode VI Error in frame: 0 / FALCON_ERROR 0x00000e

SCF: Error ResourceError: Error 15 Received for sensor 3 ..!

(in FusaCaptureViCsiHw.cpp, waitCsiFrameEnd(), line 664)

InvalidState: Mutex not initialized (FusaCaptureViCsiHw.cpp:712 / :706) ← repeats

ChanselFault 0x020000

(Argus) FileOperationFailed: Socket write failed: Broken pipe

nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV

The originating HW fault (ChanselFault/PIX_SHORT on one channel) looks recoverable — but the FuSa error-handler itself logs Mutex not initialized, i.e. the fault appears to be in the recovery path. (Also a burst of hundreds of Wrong frequency range! in the ~70 s before — unsure if related.)

Backtrace (core, stripped libs): crashing thread is the Argus socket-server thread → libnvargus_socketserver.so (recvThreadCore/handleRequest) → libnvargus.so → SIGSEGV.

Questions: (1) Is Mutex not initialized in FusaCaptureViCsiHw.cpp:706/712 a known bug in the FuSa VI error-decode path? A single bad frame killing the daemon (vs. dropping it) suggests the recovery path is the issue. (2) Same root cause as the fusaViHandler.cpp:869 syncpoint variant, or separate? (3) Beyond rtcpu auto-suspend, is there a patch/BSP fix for r36.5? Happy to share a full core / journal window

7

hello willzeng,

let’s give it a try with this attachment, Topic372530_Jun10.zip (88.8 KB)
please update libnvfusacap.so, and reboot the system for testings.

8

Hi @JerryChang — thanks for the patched `libnvfusacap.so` (`Topic372530_Jun10.zip`). We tested it; unfortunately it does **not** resolve the crash on our setup.

## Test setup

- Jetson Orin, **stock L4T R36.5 / kernel 5.15.185-tegra**

- Replaced `/usr/lib/aarch64-linux-gnu/nvidia/libnvfusacap.so` with the Jun10 build, rebooted, and confirmed nvargus-daemon is running the new (unstripped) library

- Workload: 3× GMSL2 ZED-X cameras, ~30 fps, continuous recording

## Result

- **At camera init**, the `FusaCaptureViCsiHw.cpp:706/712 “Mutex not initialized”` error now logs *without* immediately crashing — this looked like an improvement.

- **But under sustained recording, nvargus still SIGSEGVs** (this run at ~2.5 minutes in). The decode is the **ChanselFault / CSI short-frame** path, not the syncpoint variant in this thread’s title:


Module_id 30 Severity 5 : ChanselFault 0x020000

Module_id 30 Severity 2 : PIX_SHORT:

Module_id 30 Severity 2 : corr_err: notifyBits: 0x200008

Module_id 30 Severity 2 : FuSaCaptureViErrorDecode VI Error in frame: 0

Module_id 30 Severity 2 : FALCON_ERROR 0x00000e

SCF: Error ResourceError: Error 15 Received for sensor 2 ..!

 (in src/services/capture/FusaCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 664)

(NvCameraUtils) Error InvalidState: Mutex not initialized

 (FusaCaptureViCsiHw.cpp:712 / :706) (in Mutex.cpp, function lock(), line 79)   \[repeated storm\]

(Argus) Error FileOperationFailed: Socket write failed: Broken pipe (SocketUtils.cpp:49)

nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV

So our dominant failure is the **`FusaCaptureViCsiHw.cpp` ChanselFault path**: a `ResourceError: Error 15` at `waitCsiFrameEnd():664`, which then hits the uninitialized-mutex storm in the FUSA **error-decode** path and SEGVs in the Argus socket-server thread. The Jun10 patch appears to address the **`fusaViHandler.cpp:869` “syncpoint signaled but status value not updated”** variant (this thread’s title), but not this ChanselFault / `PIX_SHORT` path.

It crashed twice in quick succession during the run (two `status=11/SEGV` cores within ~20 s of each other).

9

hello willzeng,

okay,
let’s give it another try with the attachment, Topic372530_Jun11.zip (89.0 KB)
please collect nvargus-daemon logs if the issue persist, i.e. $ sudo journalctl -b -u nvargus-daemon

10

Thanks @JerryChang — the Jun 11 build (`libnvfusacap.so.r36.5.dis_notify`) **works for us**. This is the first build that survives sustained capture.

## Test setup

- Jetson Orin, **stock L4T R36.5 / kernel 5.15.185-tegra**

- 3× GMSL2 ZED-X cameras, 1920×1080 @ 30 fps, continuous recording, exposure **not** locked

- Replaced `/usr/lib/aarch64-linux-gnu/nvidia/libnvfusacap.so` with the Jun 11 build, rebooted, confirmed loaded

## Result — zero nvargus crashes across >1 hour of continuous capture

With the Jun 11 build, `nvargus-daemon` `NRestarts` stayed **0** through every test:

- **6 × 10-minute back-to-back recordings** (~65 min of continuous streaming) — all six completed and finalized cleanly, ~5.9 GB each, no drops.

- **A single 30-minute continuous recording** — completed cleanly, ~445k frames.

- Plus an ~18-minute recording earlier in the session — also clean.

For comparison, on the **stock** lib and on the **Jun 10** build, `nvargus-daemon` SIGSEGV’d within **~2–8 minutes** every single time, via both variants:

- `fusaViHandler.cpp:869` “syncpoint signaled but status value not updated”, and

- the `FusaCaptureViCsiHw.cpp` ChanselFault path (`ResourceError: Error 15` at `waitCsiFrameEnd()` line 664 → `Mutex not initialized` storm → `status=11/SEGV`).

So the Jun 11 build held through **>1 hour** of streaming where every prior build (and the stock lib) crashed by ~8 min — a clear fix for the ChanselFault variant that the Jun 10 build did not address.

12

Update on the Jun 11 build (`libnvfusacap.so.r36.5.dis_notify`): it **stops the nvargus SIGSEGV**, but we’ve since found it introduces a **severe frame-drop regression** — it appears to trade the crash for silently dropping ~1/3 of frames. We’ve had to roll back to the stock lib. Details below; this turns our earlier Question #2 (does it mask/drop genuine frame faults?) into a confirmed yes.

## Test setup

- Jetson Orin, **stock L4T R36.5 / kernel 5.15.185-tegra**

- 3× GMSL2 ZED-X cameras (6 image streams), 1920×1080 @ 30 fps, continuous recording, exposure **not** locked

- Replaced `/usr/lib/aarch64-linux-gnu/nvidia/libnvfusacap.so` with the Jun 11 build, rebooted, confirmed loaded

## What’s good — no crashes

With the Jun 11 build, `nvargus-daemon` `NRestarts` stayed **0** across >1 hour of continuous capture (6×10-min back-to-back, a 30-min single recording, an ~18-min run). On the **stock** lib and the **Jun 10** build, `nvargus-daemon` SIGSEGV’d within **~2–8 minutes** every time, via both variants:

- `fusaViHandler.cpp:869` “syncpoint signaled but status value not updated”, and

- the `FusaCaptureViCsiHw.cpp` ChanselFault path (`ResourceError: Error 15` at `waitCsiFrameEnd()` line 664 → `Mutex not initialized` storm → `status=11/SEGV`).

So the crash is gone — but at a cost we only caught when we checked the recorded data quality (we’d initially judged “clean” purely on NRestarts=0).

## The regression — ~33% frame drop, all cameras, with the Jun 11 build

Every recording made on the Jun 11 build comes out at a steady **~20 fps instead of 30 — a uniform ~33% frame drop on all 6 image streams at once**:

| Recording (Jun 11 build) | Measured fps | Drop |

|—|—|—|

| 30-min single | 19.9 | 33.8% |

| 6×10-min soak | 19.8 | 33.9% |

| short clip | 20.0 | 33.1% |

Characteristics:

- **Dead-steady ~20 fps**, not bursty — frame interval ~50 ms vs the expected 33.3 ms.

- **Uniform across all 6 streams** simultaneously → a system-wide VI/CSI effect, not a per-sensor fault.

- **IMU is unaffected** (~200 Hz per IMU, clean) → isolated to the VI/CSI camera path.

- The capture stack logs **no error and no drop** — frames simply never arrive from the driver (`nvenc dropped=0` because the encoder never receives them). So the loss is happening below userspace, silently.

## Rolling back to the stock lib restores 30 fps

We reverted `/usr/lib/aarch64-linux-gnu/nvidia/libnvfusacap.so` to the stock R36.5 lib and re-ran the same setup:

- **30.0 / 29.8 fps, 0 dropped frames** on all 6 streams (15 s and 3 min clips), data-quality check **PASS**.

- `nvargus-daemon` NRestarts=0 across a 3-min run (the crash is intermittent, so this isn’t a full soak, but it’s clean so far).

That isolates the ~33% drop to the Jun 11 `dis_notify` build.

## Read of the situation

The `dis_notify` name suggests it disables a VI/CSI error-**notification** path. It looks like the underlying ChanselFault / syncpoint fault is still firing per-frame, and rather than escalating to the SIGSEGV, the build now **discards the faulted frame** — so on hardware that faults ~1 in 3 frames we land at a steady 20 fps. In other words it suppresses the *symptom* (the crash) by dropping the *data*, without fixing the root fault. For a data-collection platform that’s not usable — we need the frames.

## Questions

1. Can you confirm whether the Jun 11 build **drops the faulted frames** (vs. recovering them)? Our data is consistent with ~1/3 of frames being silently discarded at the VI/CSI layer.

2. Is the underlying ChanselFault / `ResourceError 15` / “syncpoint signaled but status not updated” a **recoverable** condition that the driver could retry/complete instead of either crashing (stock/Jun 10) or dropping (Jun 11)?

3. Is there a build that **fixes the root fault** so cameras run at full 30 fps without the crash — ideally a **production (non-debug)** binary we can roll to a fleet?

4. Any guidance on what’s causing the per-frame VI/CSI fault on this hardware in the first place (it reproduces on the stock lib as the crash, and on Jun 11 as the 33% drop)?

Happy to share full verifier output, frame-interval histograms, or run any instrumented build. Thanks again.

13

hello willzeng,

let’s give it another try with.. Topic372530_Jun25.zip (88.7 KB)

14

Is this still an issue to support? Any result can be shared?

15

Thanks for getting back, we switched to use v4l2 as backend for camera capturing.

17

We had this (and lead to a BCCPLEXWDT reset) , and in our case the issue went away when we did a patch to the synchronising code for the DMA controller, will do a writeup and present findings