SCF Timeout & ResourceAlreadyInUse errors on Orin NX with nvphsd missing

Hi Team,

I am facing SCF timeout issues on my Orin NX while streaming from a camera using the nvargus-daemon. Here is the scenario:

  • Hardware: Orin NX SOM on custom carrier board

  • Camera: IMX477

  • Software: L4T R35.6.2

  • nvargus-daemon version: 0.99.33

  • Observation:

    • Stream starts successfully, but immediately logs red errors:
  • SCF: Error BadValue: NvPHSSendThroughputHints
    socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed: No such file or directory
    connecting to Power Hinting Service failed. Is PHS running?
    SCF: Error Timeout: waitForIdle() timed out
    ResourceAlreadyInUse: All descriptors are already pending

Checked:

sudo find / -type f -name nvphsd
/usr/sbin/nvphsd

sudo /usr/sbin/nvphsd
[1]+ Done

Temporary workaround tried:

export NVARGUS_DISABLE_PHS=1
sudo systemctl restart nvargus-daemon

  • Questions:
  1. Is there a way to make nvphsd run successfully ?

  2. Can the SCF timeout / ResourceAlreadyInUse errors be safely ignored if PHS is disabled?

  3. Are there any recommended configurations or kernel modules to stabilize streaming?

    I have attached the log as well for your understanding.

    journalctl_logs.txt (6.4 KB)

Thanks in advance for guidance!

hello Sunim_1,

it’s the Power Hint Service (PHS), it is a service that helps optimize power consumption on Jetson Orin NX.
it’s /etc/systemd/system/nvphs.service to load/enable the service during boot-up.

please see-also Supported Modes and Power Efficiency.
since you’re working with a customize board, please use the power estimator tool to estimate the power and generate the nvpmodel config for your actual use-case.

hello Sunim_1,

in addition, PHS is deprecated, please see-also Topic 257434.

Hello,

Thank you for the explanation about Power Hint Service.

I would like to clarify that the core issue is not only PHS availability, but camera streaming stops mid-way with SCF / FuSa ISP timeouts, even with a single IMX477 Arducam.

The stream starts successfully, but after some time it fails with the following fatal Argus errors:

FuSa Capture Status failed
Status syncpoint signaled but status value not updated
Camera HwEvents wait, this may indicate a hardware timeout occurred
All descriptors are already pending
waitForIdle() timed out

This suggests the root cause is ISP/CSI pipeline stalling, not only missing PHS.

Additional details:

  • Platform: Jetson Orin NX

  • L4T: R35.6.2

  • Camera: Single IMX477 (Arducam)

  • Streaming via: gst-launch / nvarguscamerasrc

  • Failure occurs mid-stream, not at startup

    If PHS is mandatory for camera stability on Orin NX, could you confirm which package provides nvphsd.service and whether it should be running by default on full JetPack images?

    Any guidance to isolate the root cause would be greatly appreciated.

hello Sunim_1,

PHS is deprecated as mentioned in previous comment #5.
besides.. I’ve also test IMX477 with Orin NX developer kit, and it worked normally.

Thanks for the clarification.

Understood that PHS is deprecated and not required for camera stability — I will ignore nvphs-related errors.

Since IMX477 works normally on the Orin NX developer kit, this points to a platform-specific or driver-level issue on my setup.

However, I am seeing repeatable mid-stream failures with the following fatal Argus errors:

FuSa Capture Status failed
Status syncpoint signaled but status value not updated
Camera HwEvents wait, hardware timeout occurred
All descriptors are already pending
waitForIdle() timed out

These errors indicate the ISP did not receive frame-end, leading to buffer starvation and Argus collapse.

To help isolate the root cause, could you please advise:

  1. Which subsystem timeout typically triggers FuSa Capture Status failed?

    • CSI link stall?

    • Sensor not streaming?

    • ISP clock / bandwidth drop?

  2. What debug steps or traces are recommended on Orin NX to determine whether the stall is:

    • Sensor-side (frame not sent)

    • CSI-side (CRC / deskew / settle issues)

    • ISP-side (clock or syncpoint issues)

  3. Are there any known IMX477 corner cases on Orin NX (exposure, frame length, resolution/fps transitions) that can cause mid-stream ISP timeouts?

  4. Are there device-tree properties on Orin NX (CSI settle time, discontinuous clock, lane polarity, deskew) that commonly cause this behavior when mismatched?

    Also, for alignment with your internal validation, could you please confirm:

    • The JetPack / L4T version used when testing IMX477 on the Orin NX developer kit
    • Whether the test was done using LibArgus (nvarguscamerasrc / gst-launch) or V4L2-only

    I am currently using Jetson Linux R35.6.2, and I want to ensure I am testing against the same software baseline.

Any guidance on narrowing this down would be very helpful.

hello Sunim_1,

please refer to Camera Architecture Stack.
let’s have v4l2 standard IOCTL to test camera stream without ISP.
for instance,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=500

Thanks for the guidance.

I tested the IMX477 using standard V4L2 IOCTLs as suggested:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=500

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=500
<<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<<<<<<< 21.92 fps
<<<<<<<<<<<<<<<<

The stream runs stably for 500 frames with a consistent ~21.9 fps and no timeouts or errors.

However, when using the ISP / LibArgus path (nvarguscamerasrc), I still encounter:

  • FuSa Capture Status failed

  • waitIspFrameEnd timeout

  • Argus session deadlock after extended runtime

    Could you please advise:

    • Whether these FuSa / ISP timeouts are known issues on Jetson Linux R35.6.2

    • Which JetPack version you tested IMX477 with on Orin NX DevKit

    • If any recommended Argus settings, patches, or workarounds exist for long-running streams on Orin NX

    I’m happy to test additional configurations if needed.

hello Sunim_1,

JP-5.1.5/r35.6.1 has already include all the bug fixes although there’re some camera stability issue within early JP-5 release, see-also Topic 268833, Topic 276217, Topic 305949.

we’re testing with Raspberry Pi, IMX477 on JP-5.1.5 to confirm the stream stability, is it possible to confirm it (i.e. Arducam’s IMX477) with Orin NX developer kit as well?
I would like to double check how long did it took to reproduce fatal Argus errors?

please give it another try to boost all the VI/CSI/ISP clocks.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

you may also try to configure enableCamInfiniteTimeout and test with Argus sample app, for example, userAutoExposure

$ sudo pkill nvargus-daemon
$ export enableCamInfiniteTimeout=1
$ sudo nvargus-daemon &
$ ./argus_userautoexposure

Based on your recommendation, I applied the following clock locking commands:

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked

cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

Observed Behavior

  • After applying the above:

    • Streaming works perfectly

    • No timeout issues

    • Works across multiple soft and hard reboots

  • I tested 6 times, each run ~20 minutes

  • After several successful runs, the issue reappears

So this workaround is **not stable long-term

Logs – When streaming starts
**
From journalctl -u nvargus-daemon -f:

OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found
---- imager: No override file found. ----

E/ libnvphs:socket: Error[2]: socket connection /var/lib/nvphs/nvphsd.ctl to PHS failed
D/ libnvphs:socket: Warning: connecting to Power Hinting Service failed. Is PHS running?
SCF: Error BadValue: NvPHSSendThroughputHints

**Logs – When streaming gets stuck
**
SCF: Error Timeout: waitForNewerSample()
SCF_AutocontrolACSync failed to wait for an earlier frame to complete
SCF: Error Timeout: Sending critical error event for Session 1

waitForIdleLocked remaining request 24921
SCF: Error Timeout: waitForIdle() timed out

SCF: Error InvalidState: 1 buffers still pending during EGLStreamProducer destruction
PowerServiceCore:handleRequests: timePassed = 25537

After this point:

  • Stream freezes

  • Only restarting nvargus-daemon or reboot recovers

    Could you please help me with the reason why timeout issue occured while streaming.
    Any guidance on root cause analysis or official long-term fix would be appreciated.

hello Sunim_1,

it looks boosting clock helps, please refer to Sensor Pixel Clock to review your clock rate settings, which must be set correctly to avoid potential issues.
besides.. you may also check Supported Modes and Power Efficiency, since you’re working with a customize board, please use the power estimator tool to estimate the power and generate the nvpmodel config for your actual use-case.

Thanks for the suggestions. Before applying further mitigations, I would like to understand the root cause of the timeout we are observing.

  • The camera in use is Arducam IMX477, which has an internal 24 MHz oscillator

  • In the device tree, the clock configuration is already set accordingly:

    mode0 { /* IMX477_MODE_3840x2160 */
    mclk_khz = “24000”;
    num_lanes = “2”;
    tegra_sinterface = “serial_a”;
    };

    Regarding power:

    • The SOM is powered with a sufficient external power supply
      The problem still occurs after long-duration streaming, even with clocks locked at max

      Could you please advise:

      1. Whether Argus has known long-duration timeout issues on Orin NX with continuous CSI streaming?

      2. If there are additional required settings beyond clock boosting (e.g., specific nvpmodel profiles recommended for camera-heavy workloads)?

      3. Whether the nvphs warnings could contribute to Argus timeouts over time.

      We’d appreciate any guidance on a recommended long-term stable configuration for continuous camera streaming on a custom carrier board.

      Thanks for your support.

hello Sunim_1,

how long did it took to reproduce the timeout failure?
is it thermal related to trigger throttling which also lower clock rates?

Thanks for the follow-up. Please find the details below.

1. Reproduction time

  • The timeout does not occur immediately.

  • It typically reproduces after ~15–25 minutes of continuous streaming.

  • The issue is time-dependent, not load-dependent at startup.

  • Restarting nvargus-daemon recovers streaming temporarily.

This behavior is consistent across multiple test cycles.

  1. Additional observation
  • At stream start, Argus logs consistently report:
    NvPHSSendThroughputHints failed
    Power Hinting Service not running

    1. Clarification request

    Given the above, could you please advise:

    • Whether this is a known long-duration Argus / SCF issue on Orin NX for continuous CSI streaming

    • If there are recommended Argus or system configurations (beyond clock locking) for sustained camera workloads

    • Whether nvphs is required for long-term Argus stability on Orin platforms

    Our goal is to understand why Argus stops receiving or processing frames over time, rather than applying repeated mitigations.

    Thanks for your guidance.

hello Sunim_1,

we’ve test on developer kit with single camera stream for days to confirm camera stability.
I would suspect it’s thermal related to trigger throttling, please execute with $ sudo tegrastats to monitor system loadings and also temperature.

Thanks for the clarification.
Please find below the tegrastats observations captured during continuous streaming.

tegrastats_stream.log (147.8 KB)

hello Sunim_1,

it seems it’s running at relatively low power mode, which has 4 CPUs offline.
for instance, CPU [41%@1420,8%@1420,9%@1420,8%@1420,off,off,off,off]
please have system config for switching to MaxN for confirmation.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.