DeepStream 6.2 Upgrade Issues on Jetson Xavier NX: Argus Camera Socket Errors

angel.corrales · December 22, 2023, 1:31pm

• Hardware Platform (Jetson / GPU) Jetson
• DeepStream Version 6.2
• JetPack Version (valid for Jetson only) 5.1.1

Hello NVIDIA Community,

I am encountering a problem with my DeepStream pipeline since upgrading from version 6.0 to 6.2 on a Jetson Xavier NX device with the Jetpack 5.1.1 update. I would greatly appreciate any guidance or suggestions you can provide.

Problem Description:

The issue started occurring after updating my DeepStream environment from version 6.0 to 6.2 and Jetpack to 5.1.1. My pipeline uses multiple cameras through nvarguscamerasrc and other elements like nvvideoconvert, nvinfer, nvdewarper, and nvdsosd.

My pipeline works correctly for a while (somewhat randomly, sometimes 1 minute, other times longer) but then stops or is unexpectedly interrupted, and I receive a series of Argus-related errors, indicating problems in camera communication or socket handling. The errors are as follows:

[wb_video-1] (Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
[wb_video-1] (Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
[wb_video-1] (Argus) Error FileOperationFailed: Receive worker failure, notifying 4 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
[wb_video-1] (Argus) Error InvalidState: Argus client is exiting with 4 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
[wb_video-1] (Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
[wb_video-1] (Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
[wb_video-1] (Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
[wb_video-1] (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
[wb_video-1] (Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
[wb_video-1] (Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
[wb_video-1] (Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
[wb_video-1] (Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
[wb_video-1] (Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
[wb_video-1] (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
[wb_video-1] (Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91).

Steps I Have Tried:

Checking the physical camera connections and the configurations of nvarguscamerasrc.
Monitoring system resource usage to rule out memory or CPU issues.
I have made numerous changes to my pipeline without success.

After many tests, I have found that the same issue occurs even when launching /usr/src/jetson_multimedia_api/argus/build/apps/camera/ui/camera/.argus_camera. When running this app, I can view my 4 cameras for a while, and then after some time, they stop being visible, and everything crashes (similar to what happens from the Deepstream pipeline):

(Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
(Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error FileOperationFailed: Receive worker failure, notifying 4 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
(Argus) Error InvalidState: Argus client is exiting with 4 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
(Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/EventThread.cpp, threadExecute:104 Failed to get AF regions(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
Error generated. /usr/src/jetson_multimedia_api/argus/samples/utils/Thread.cpp, threadFunction:132 (propagating)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/Dispatcher.cpp, waitForEvents:1575 Failed to get events
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/EventThread.cpp, threadExecute:66 (propagating)
Error generated. /usr/src/jetson_multimedia_api/argus/samples/utils/Thread.cpp, threadFunction:132 (propagating)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/EventThread.cpp, threadExecute:104 Failed to get AF regions
Error generated. /usr/src/jetson_multimedia_api/argus/samples/utils/Thread.cpp, threadFunction:132 (propagating)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/Dispatcher.cpp, waitForEvents:1575 Failed to get events
Error generated. /usr/src/jetson_multimedia_api/argus/apps/camera/modules/EventThread.cpp, threadExecute:66 (propagating)
Error generated. /usr/src/jetson_multimedia_api/argus/samples/utils/Thread.cpp, threadFunction:132 (propagating)
^C(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)

I appreciate any help or comments in advance.

Best regards,
Ángel.

yingliu · December 22, 2023, 1:50pm

Moving to Xavier NX Forum for better support.

DaneLLL · December 27, 2023, 5:07am

Hi,
it looks like the camera source cannot be launched. Please try the command and see if camera preview is shown:

$ DISPLAY=:0 gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink sync=0

angel.corrales · January 3, 2024, 8:46am

Hello, good morning.
Thank you very much for your response and happy new year!

As I mentioned in the original post, the cameras are launched and run without any issues for a while.

Likewise, I have tested your command for my 4 cameras at the same time. After about 3/4 minutes of execution, the cameras have shut down one by one showing errors similar to those already mentioned:

$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink sync=0
Setting pipeline to PAUSED ...
0:00:00.715930669  5661 0xaaaaefa91800 FIXME                default gstutils.c:3980:gst_pad_create_stream_id_internal:<nvarguscamerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2712 x 1538 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: 1920 x 1080 FR = 41.000000 fps Duration = 24390244 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: 2712 x 1538 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: 1920 x 1080 FR = 41.000000 fps Duration = 24390244 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 3 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 41.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
(Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
(Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error FileOperationFailed: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
(Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
CONSUMER: Done Success
Got EOS from element "pipeline0".
Execution ended after 0:05:10.282800442
Setting pipeline to NULL ...
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
GST_ARGUS: Cleaning up
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Caught SIGSEGV
Caught SIGSEGV
Spinning.  Please run 'gdb gst-launch-1.0 5661' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
0:05:38.555841517  5661 0xaaaaef595b00 WARN             xvimagesink xvimagesink.c:554:gst_xv_image_sink_handle_xevents:<xvimagesink0> error: Output window was closed

$ gst-launch-1.0 nvarguscamerasrc sensor-id=2 ! nvvidconv ! video/x-raw,format=I420 ! xvimagesink sync=0
Setting pipeline to PAUSED ...
0:00:00.641504937  5307 0xaaab120c1000 FIXME                default gstutils.c:3980:gst_pad_create_stream_id_internal:<nvarguscamerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2712 x 1538 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: 1920 x 1080 FR = 41.000000 fps Duration = 24390244 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: 2712 x 1538 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: 1920 x 1080 FR = 41.000000 fps Duration = 24390244 ; Analog Gain range min 1.000000, max 3981.070801; Exposure Range min 20000, max 660000000;

GST_ARGUS: Running with following settings:
   Camera index = 2 
   Camera mode  = 3 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 41.000000 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
(Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
(Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error FileOperationFailed: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
(Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
CONSUMER: Done Success
Got EOS from element "pipeline0".
Execution ended after 0:06:09.475767997
Setting pipeline to NULL ...
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
GST_ARGUS: Cleaning up
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Caught SIGSEGV
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Segmentation fault (core dumped)

DaneLLL · January 3, 2024, 9:20am

Hi,
Looks like the sensor driver is not ready. Do you use Bayer camera? If you use Bayer camera, would suggest consult with the vendor to enable the driver

angel.corrales · January 3, 2024, 9:40am

I’m currently using an FSM-IMX464C-01S-V1A sensor. Just to provide some background, everything was operating flawlessly with no issues when I was on JetPack 4.6.1 (Deepstream 6.0). However, I started encountering problems after upgrading to JetPack 5.1.1 (Deepstream 6.2).

My problem seems identical to the one described in: Errors in Gstreamer Pipeline (JetPack 5.1 - L4T 35.2.1)

Additionally, I’d like to mention that a constant warning message appears when running my pipeline: " System throttled due to overcurrent". However, I have noticed that VDD_IN (Curr) does not exceed 2.5A at any time.

angel.corrales · January 9, 2024, 9:29am

I have tried replacing all 4 cameras and their wiring with completely new components, with no satisfactory results. I have tried replacing the Jetson NX with a completely new one. Definitely not a hardware problem. I have also tried hybrid things like flashing a NX with JP 5.1.1 and use the e l4t-multimedia package from L4T r35.4.1, without success.

Also tried to boost the clocks:

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

It also did not solve the problem.

But I finally found something that at least looks like the solution!
I found in a post a libnvargus.so library modified by a moderator for my version of Jetpack 5.1.1. After replacing it with this one, I’m running tests and my pipeline has been running for about 24h without interruptions!
Yo can find the libnvargus.so here.

It seems that this was the problem!

JerryChang · January 10, 2024, 1:40am

FYI,
this pre-built include the memory corruption fix for libnvargus.
this change has done code-review progress and merged into rel-35 code-line, hence, you may expect next Jetpack-5 public release version will also include the fix.

system · January 30, 2024, 5:36am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Argus Error OverFlow Jetson Xavier NX camera	19	1632	February 5, 2024
Argus errors when restarting pipeline - Jetson AGX Jetson AGX Xavier camera	10	250	February 7, 2025
Nvarguscamera crashes after frequent restarts Jetson Nano deepstream	10	1075	August 19, 2022
Nvargus errors, timeout, and inevitable failure (happens on 4x different nanos) DeepStream SDK camera , gstreamer , nvbugs	6	1811	October 12, 2021
Issues with Camera Streaming on Jetson Xavier NX Jetson Xavier NX camera	2	93	September 26, 2024
Argus Error while switch MIPI Camera between Gsteamer Pipeline , Deepstream App , Gstreamer Pipeline Jetson Nano camera	8	867	October 15, 2021
Nvargusdaemon crashing after camera has been active for a short time Jetson Xavier NX camera , gstreamer	4	1293	January 11, 2023
Nvargus errors when starting a gstreamer pipeline Jetson Xavier NX camera	30	529	May 28, 2025
Caught SIGSEGV Segmentation fault (core dumped) DeepStream SDK jetson , deepstream	5	169	November 27, 2024
Started getting an "Argus" error when running camera script Jetson Nano camera	12	5569	October 15, 2021

DeepStream 6.2 Upgrade Issues on Jetson Xavier NX: Argus Camera Socket Errors

Problem Description:

Steps I Have Tried:

Related topics