Nvargus-daemon crashing during recording

MatthewIrvin1 · October 29, 2024, 6:59pm

I’m working on an application that utilizes GStreamer pipelines to handle multiple camera sources. The current setup involves four camera inputs, which are muxed with audio into 4 separate file sinks, while also being streamed via UDP for live viewing. While everything works smoothly for short recordings, I run into issues when attempting longer recordings (typically 4 hours or more, but this has also occurred with shorter durations). The nvargus-daemon seems to crash during these sessions.

Here’s a brief overview of the pipelines I’m using:

Primary pipeline for capturing

Gst.parse_launch(f"""
        nvarguscamerasrc sensor_id=1 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t1
        t1. ! queue ! nvv4l2h264enc ! h264parse ! queue ! mux1.
        t1. ! queue max-size-buffers=2 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5000 
        nvarguscamerasrc sensor_id=0 ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,interlace-mode=interlaced ! tee name=t2
        t2. ! queue ! nvv4l2h264enc ! h264parse ! queue ! mux2.
        t2. ! queue max-size-buffers=2 ! nvvidconv ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5001 
        nvarguscamerasrc sensor_id=2 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1  ! tee name=t3
        t3. ! queue ! nvv4l2h264enc ! h264parse ! queue ! mux3.
        t3. ! queue max-size-buffers=2 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5002 
        nvarguscamerasrc sensor_id=3 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t4
        t4. ! queue ! nvv4l2h264enc ! h264parse ! queue ! mux4.
        t4. ! queue max-size-buffers=2 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5003
        alsasrc device=hw:2,0 ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,channels=1,rate=48000 ! volume name=volume1 ! mix1.
        alsasrc device=hw:3,0 ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,channels=1,rate=48000 ! volume name=volume2 ! mix1.
        audiomixer name=mix1 ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,channels=1,rate=48000 ! tee name=t
        t. ! queue ! mux1.
        t. ! queue ! mux2.
        t. ! queue ! mux3.
        t. ! queue ! mux4.
        qtmux name=mux1 ! filesink location=/media/{CAM1_output_file}
        qtmux name=mux2 ! filesink location=/media/{CAM2_output_file}
        qtmux name=mux3 ! filesink location=/media/{CAM3_output_file}
        qtmux name=mux4 ! filesink location=/media/{CAM4_output_file}
    """)

Pipeline for live view on Display 0

Gst.parse_launch(f"""
        udpsrc port={current_port_cam1} caps="application/x-rtp, media=video" name=udpsrc1 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! nvoverlaysink display-id=0 name=overlay1 overlay=1 overlay-depth=1 sync=false
    """)

Pipeline for live view on Display 1

Gst.parse_launch(f"""
        udpsrc port={current_port_cam2} caps="application/x-rtp, media=video" name=udpsrc2 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! nvoverlaysink display-id=1 name=overlay2 overlay=1 overlay-depth=1 sync=false
    """)

Gstreamer does run and capture frames into an mp4 but eventually leads to the following response. The program will hang until ctrl+C.


Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
===== NVMEDIA: NVENC =====
===== NVMEDIA: NVENC =====
===== NVMEDIA: NVENC =====
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
NvMMLiteBlockCreate : Block : BlockType = 4
NvMMLiteBlockCreate : Block : BlockType = 4
NvMMLiteBlockCreate : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
Opening in BLOCKING MODE
Recording started. Press 'Ctrl+C' to stop.
Enter command: NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 1
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

CONSUMER: Producer has connected; continuing.
GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 3
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Available Sensor modes :
CONSUMER: Producer has connected; continuing.
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 2
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Available Sensor modes :
CONSUMER: Producer has connected; continuing.
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 2
   Output Stream W = 1920 H = 1080
   seconds to Run    = 0
   Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
H264: Profile = 66, Level = 0
NvMMLiteOpen : Block : BlockType = 261
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
NvMMLiteBlockCreate : Block : BlockType = 261
a
Switched Camera 1 to port 5002.
Live Display Swapped
Enter command: areference in DPB was never decoded
a
Invalid command or not in the correct mode.
Enter command: a
Switched Camera 1 to port 5003.
Live Display Swapped
Enter command: reference in DPB was never decoded
CONSUMER: ERROR OCCURRED
CONSUMER: ERROR OCCURRED
CONSUMER: ERROR OCCURRED
(Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
(Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
(Argus) Error FileOperationFailed: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 340)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
(Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
CONSUMER: Done Success
^CStopping the pipeline...
Error received: NvArgusCameraSrc: DISCONNECTED (8), Argus Error Status
GST_ARGUS: Cleaning up
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Segmentation fault (core dumped)

I believe this error is only caused from nvargus-daemon stopping which has been my main way of reproducing the error above. What is stopping nvargus-daemon I am unsure of. To investigate further, I checked the logs using journalctl, syslog, and dmesg for any relevant messages and at best :

nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV
nvargus-daemon.service: Failed with result 'signal'.
nvargus-daemon.service: Service hold-off time over, scheduling restart.
nvargus-daemon.service: Scheduled restart job, restart counter is at 6.
Stopped Argus daemon.
Started Argus daemon.

My goal is to achieve stable recordings lasting at least 8+ hours without issues. I am unsure why or where else to look for errors. I would hope to be able to stop nvargus-daemon from crashing or if it does crash, it can restart and continue the previously opened pipeline. I have two Jetson Tx2 which this occurs on both.

ShaneCCC · October 30, 2024, 5:14am

Please get the daemon log by below command.

sudo journalctl -u nvargus-daemon -f

MatthewIrvin1 · October 30, 2024, 11:04am

Oct 29 13:22:15 tx2-desktop nvargus-daemon[8416]: === python3[8503]: CameraProvider initialized (0x7f94892510)LSC: LSC surface is not based on full res!
Oct 29 13:22:15 tx2-desktop nvargus-daemon[8416]: LSC: LSC surface is not based on full res!
Oct 29 13:22:15 tx2-desktop nvargus-daemon[8416]: LSC: LSC surface is not based on full res!
Oct 29 13:22:15 tx2-desktop nvargus-daemon[8416]: LSC: LSC surface is not based on full res!
Oct 29 18:32:36 tx2-desktop systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV
Oct 29 18:32:36 tx2-desktop systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
Oct 29 18:32:36 tx2-desktop systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
Oct 29 18:32:36 tx2-desktop systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 1.
Oct 29 18:32:36 tx2-desktop systemd[1]: Stopped Argus daemon.
Oct 29 18:32:36 tx2-desktop systemd[1]: Started Argus daemon.

I haven’t been able to figure out what specifically is happening when these logs occur.

I have been testing further and it seems when only recording with 1 camera I have been able to get a 12+ hr session to work.

[EDIT]: Ran again this morning and grabbed the log from the run.

Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...=== python3[12063]: Connection established (7FAF5631D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module4
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclHwGetModuleList: No module data found
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclHwGetModuleList: No module data found
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: ---- imager: No override file found. ----
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: (NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: (NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 107)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclStartPlatformDrivers: Failed to start module drivers
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclStateControllerOpen: Failed ImagerGUID 0. (error 0xA000E)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: NvPclOpen: PCL Open Failed. Error: 0xf
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 593)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 437)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 305)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 471)
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: Acquiring SCF Camera device source via index 0 has failed. ---- imager: No override file found. ----
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: ---- imager: No override file found. ----
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: ---- imager: No override file found. ----
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: ---- imager: No override file found. ----
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 07:47:29 tx2-desktop nvargus-daemon[9502]: === python3[12063]: CameraProvider initialized (0x7fa8434e60)LSC: LSC surface is not based on full res!
Oct 30 07:47:30 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 07:47:30 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 07:47:30 tx2-desktop nvargus-daemon[9502]: LSC: LSC surface is not based on full res!
Oct 30 09:15:59 tx2-desktop systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV
Oct 30 09:15:59 tx2-desktop systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
Oct 30 09:15:59 tx2-desktop systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
Oct 30 09:15:59 tx2-desktop systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 2.
Oct 30 09:15:59 tx2-desktop systemd[1]: Stopped Argus daemon.
Oct 30 09:15:59 tx2-desktop systemd[1]: Started Argus daemon.

The errors concerning NvCamV4l2 and SCF seem to occur on every program start up but do not inhibit the ability to create the mp4 if closed before crash.

MatthewIrvin1 · October 31, 2024, 1:35pm

Continued on with more tests,
I cut back on the main pipeline to record and live view with two cameras which has worked for 12+hrs before I cut it, will attempt to record 3 view 2 for further testing.

I also tested to see if I could just view for long periods of time, removing the file sink and just having the pipeline stream the source with the two other pipelines viewing them. This seemed to also break with the same error. Below are the pipelines used as well as the daemon log from sudo journalctl -u nvargus-daemon -f.

Streaming Pipeline

Gst.parse_launch(f"""
        nvarguscamerasrc sensor_id=1 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t1
        t1. ! queue max-size-buffers=2 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5000 
        nvarguscamerasrc sensor_id=0 ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,interlace-mode=interlaced ! tee name=t2
        t2. ! queue max-size-buffers=2 ! nvvidconv ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5001 
        nvarguscamerasrc sensor_id=2 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1  ! tee name=t3
        t3. ! queue max-size-buffers=2 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5002 
        nvarguscamerasrc sensor_id=3 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t4
        t4. ! queue max-size-buffers=2 ! nvv4l2h264enc preset-level=1 bitrate=4000000 insert-sps-pps=true ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=127.0.0.1 port=5003
    """)

Pipeline for live view on Display 0

Gst.parse_launch(f"""
        udpsrc port={currentPortCam1} caps="application/x-rtp, media=video" name=udpsrc1 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! nvoverlaysink display-id=0 name=overlay1 overlay=1 overlay-depth=1 sync=false
    """)

Pipeline for live view on Display 1

Gst.parse_launch(f"""
        udpsrc port={currentPortCam2} caps="application/x-rtp, media=video" name=udpsrc2 ! rtph264depay ! queue ! h264parse ! nvv4l2decoder ! nvvidconv ! nvoverlaysink display-id=1 name=overlay2 overlay=1 overlay-depth=1 sync=false
    """)

sudo journalctl -u nvargus-daemon -f

Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...=== python3[16454]: Connection established (7FA67F31D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module4
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclHwGetModuleList: No module data found
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclHwGetModuleList: No module data found
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: ---- imager: No override file found. ----
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: (NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: (NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: (NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 107)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclStartPlatformDrivers: Failed to start module drivers
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclStateControllerOpen: Failed ImagerGUID 0. (error 0xA000E)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: NvPclOpen: PCL Open Failed. Error: 0xf
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 593)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 437)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 305)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 471)
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: Acquiring SCF Camera device source via index 0 has failed. ---- imager: No override file found. ----
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: ---- imager: No override file found. ----
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: ---- imager: No override file found. ----
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: ---- imager: No override file found. ----
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: === python3[16454]: CameraProvider initialized (0x7fa099ec50)LSC: LSC surface is not based on full res!
Oct 30 15:46:44 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 15:46:45 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 15:46:45 tx2-desktop nvargus-daemon[15539]: LSC: LSC surface is not based on full res!
Oct 30 20:38:55 tx2-desktop systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV
Oct 30 20:38:55 tx2-desktop systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
Oct 30 20:38:55 tx2-desktop systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
Oct 30 20:38:55 tx2-desktop systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 2.
Oct 30 20:38:55 tx2-desktop systemd[1]: Stopped Argus daemon.
Oct 30 20:38:55 tx2-desktop systemd[1]: Started Argus daemon.

ShaneCCC · November 4, 2024, 3:53am

All of the failed tell the APP terminal unnormally.
Please remove the encoder/decoder by using fake sink to narrow down the problem.

MatthewIrvin1 · November 4, 2024, 12:28pm

I ran the following pipeline using fake sinks and I still was able to get the error occurring with no real change on both machines. I will provide the pipelines used, the gstreamer that ran and the logs from journalctl.

Pipeline used with fakesinks instead of udpsink and filesink. Viewing pipelines were disabled for the test. One Machine ran pipeline without the audio sources as well.

        nvarguscamerasrc sensor_id=1 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t1
        t1. ! queue ! fakesink
        t1. ! queue max-size-buffers=2 ! fakesink

        nvarguscamerasrc sensor_id=0 ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,interlace-mode=interlaced ! tee name=t2
        t2. ! queue ! fakesink
        t2. ! queue max-size-buffers=2 ! nvvidconv ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! queue ! fakesink

        nvarguscamerasrc sensor_id=2 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t3
        t3. ! queue ! fakesink
        t3. ! queue max-size-buffers=2 ! fakesink

        nvarguscamerasrc sensor_id=3 ! video/x-raw(memory:NVMM),width=1280,height=720,framerate=30/1 ! tee name=t4
        t4. ! queue ! fakesink
        t4. ! queue max-size-buffers=2 ! fakesink

        alsasrc device=hw:2,0 ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,channels=1,rate=48000 ! volume name=volume1 ! mix1.
        alsasrc device=hw:3,0 ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,channels=1,rate=48000 ! volume name=volume2 ! mix1.
        audiomixer name=mix1 ! audioconvert ! audioresample ! audio/x-raw,format=S16LE,channels=1,rate=48000 ! tee name=t
        t. ! queue ! fakesink
        t. ! queue ! fakesink
        t. ! queue ! fakesink
        t. ! queue ! fakesink

Gstreamer Logs

Running pipelines...
GST_ARGUS: Creating output stream
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

CONSUMER: Waiting until producer is connected...
GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 1
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
GST_ARGUS: Available Sensor modes :
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 2
   Output Stream W = 1920 H = 1080
   seconds to Run    = 0
   Frame Rate = 29.999999
GST_ARGUS: Available Sensor modes :
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 3
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

CONSUMER: Producer has connected; continuing.
GST_ARGUS: Running with following settings:
   Camera index = 2
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
CONSUMER: Producer has connected; continuing.
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
CONSUMER: ERROR OCCURRED
CONSUMER: ERROR OCCURRED
(Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
(Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
(Argus) Error FileOperationFailed: Receive worker failure, notifying 2 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 340)
(Argus) Error InvalidState: Argus client is exiting with 2 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
(Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error FileOperationFailed:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
CONSUMER: Done Success
CONSUMER: Done Success
^CError received: NvArgusCameraSrc: DISCONNECTED (8), Argus Error Status
GST_ARGUS: Cleaning up
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Segmentation fault (core dumped)

Logs from sudo journalctl -u nvargus-daemon -f

-- Logs begin at Mon 2024-11-04 07:05:46 EST. --
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: ---- imager: No override file found. ----
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: LSC: LSC surface is not based on full res!
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: ---- imager: No override file found. ----
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: LSC: LSC surface is not based on full res!
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: ---- imager: No override file found. ----
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: LSC: LSC surface is not based on full res!
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: === python3[7276]: CameraProvider initialized (0x7f8d9cf8a0)LSC: LSC surface is not based on full res!
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: LSC: LSC surface is not based on full res!
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: LSC: LSC surface is not based on full res!
Nov 04 07:09:17 tx2-desktop nvargus-daemon[4688]: LSC: LSC surface is not based on full res!
Nov 04 09:29:37 tx2-desktop systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV
Nov 04 09:29:37 tx2-desktop systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
Nov 04 09:29:37 tx2-desktop systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
Nov 04 09:29:37 tx2-desktop systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 1.
Nov 04 09:29:37 tx2-desktop systemd[1]: Stopped Argus daemon.
Nov 04 09:29:37 tx2-desktop systemd[1]: Started Argus daemon.

[EDIT]
I have additionally ran all 4 of them using separate instances of nvgstcapture-1.0. The same error occurred on 3 of them while 1 crashed and had a different error that is below. They each ran and viewed fine until this error. This has occurred once so I will attempt to recreate it.

GST_ARGUS: Running with following settings:
   Camera index = 2
   Camera mode  = 4
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 59.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: ERROR OCCURRED
ERROR on bus: by /GstPipeline:capture_native_pipeline/GstBin:cap_bin/GstNvArgusCameraSrc:nvarguscamerasrc0: DISCONNECTED
debug info:
Argus Error Status
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Segmentation fault (core dumped)

ShaneCCC · November 5, 2024, 4:39am

What’s the version?

cat /etc/nv_tegra_release

MatthewIrvin1 · November 7, 2024, 7:05pm

cat /etc/nv_tegra_release
R32 (release), REVISION: 7.5, GCID: 36557527, BOARD: t186ref, EABI: aarch64, DATE: Tue Jun 11 23:18:04 UTC 2024

I was able to recreate the error in my earlier reply when running nvgstcapture-1.0 individually on each of the four cameras. One or sometimes two cameras would get that error then freeze the other two when nvargus-daemon restarts. The journalctl logs still seem to just show the same logs

Nov 04 13:58:48 tx2-desktop nvargus-daemon[10303]: === nvgstcapture-1.0[10327]: CameraProvider initialized (0x7f70434e80)LSC: LSC surface is not based on full res!
Nov 04 13:58:54 tx2-desktop nvargus-daemon[10303]: === nvgstcapture-1.0[10401]: Connection established (7F2DE0C1D0)=== nvgstcapture-1.0[10401]: CameraProvider initialized (0x7ed4000c40)LSC: LSC surface is not based on full res!
Nov 04 13:58:59 tx2-desktop nvargus-daemon[10303]: === nvgstcapture-1.0[10434]: Connection established (7EC9AB41D0)=== nvgstcapture-1.0[10434]: CameraProvider initialized (0x7f0006f270)LSC: LSC surface is not based on full res!
Nov 04 13:59:04 tx2-desktop nvargus-daemon[10303]: === nvgstcapture-1.0[10468]: Connection established (7EC19731D0)=== nvgstcapture-1.0[10468]: CameraProvider initialized (0x7f401cf680)LSC: LSC surface is not based on full res!
Nov 04 19:41:36 tx2-desktop systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=11/SEGV
Nov 04 19:41:36 tx2-desktop systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
Nov 04 19:41:37 tx2-desktop systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
Nov 04 19:41:37 tx2-desktop systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 2.
Nov 04 19:41:37 tx2-desktop systemd[1]: Stopped Argus daemon.
Nov 04 19:41:37 tx2-desktop systemd[1]: Started Argus daemon.

MatthewIrvin1 · November 12, 2024, 2:29pm

I attached gdb to the nvargus-daemon process. At a certain point while viewing tegra stats I notices a drastic change in RAM and CPU Usage before stabilizing. I decided to close the application and the following errors were in gdb. This was done using the main pipeline in the first post to write to a mp4., and oddly it has made readable files unlike before.

gdb print statements:

Thread 6 "nvargus-daemon" received signal SIGABRT, Aborted.
[Switching to Thread 0x7f9f7041d0 (LWP 6997)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51      ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) backtrace
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x0000007fa582f974 in __GI_abort () at abort.c:79
#2  0x0000007fa586872c in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fa59297e0 "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x0000007fa586eaa4 in malloc_printerr (str=str@entry=0x7fa59254b8 "double free or corruption (out)") at malloc.c:5342
#4  0x0000007fa5870704 in _int_free (av=0x7fa594fa70 <main_arena>, p=0x7f08500f50, have_lock=<optimized out>) at malloc.c:4308
#5  0x0000007fa4dabeb0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#6  0x0000007fa4d6ef8c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#7  0x0000007fa4d6e650 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#8  0x0000007fa4bfec54 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#9  0x0000007fa4ae09b0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#10 0x0000007fa4af05e0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#11 0x0000007fa4aebe30 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#12 0x0000007fa4a55af0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#13 0x0000007fa4a49c20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#14 0x0000007f9f773610 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
#15 0x0000007fa5a50550 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#16 0x0000007fa5a529cc in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#17 0x0000007fa5a6a804 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#18 0x0000007fa5a6aca0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#19 0x0000007fa5a6ae78 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#20 0x0000007fa5a6a274 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#21 0x0000007fa5a6a374 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so
#22 0x0000007fa5461088 in start_thread (arg=0x7fe302488f) at pthread_create.c:463
#23 0x0000007fa58cc0cc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78
(gdb) info locals
set = {__val = {0, 0, 0, 0, 0, 0, 0, 0, 0, 548135776192, 548237670608, 548238852096, 548135780816, 548159713280, 548135776496, 548135778768}}
pid = <optimized out>
tid = <optimized out>
ret = <optimized out>
(gdb) info registers
x0             0x0      0
x1             0x7f9f702eb8     548135775928
x2             0x0      0
x3             0x8      8
x4             0x0      0
x5             0x7f9f702eb8     548135775928
x6             0xffffffffffffffff       -1
x7             0xffffffffffffffff       -1
x8             0x87     135
x9             0xffffffffffffffff       -1
x10            0xffffffffffffffff       -1
x11            0xffffffffffffffff       -1
x12            0xffffffffffffffff       -1
x13            0xffffffffffffffff       -1
x14            0x8      8
x15            0x0      0
x16            0x7fa570b158     548236472664
x17            0x7fa58738d8     548237949144
x18            0x7fa594fa70     548238850672
x19            0x7fa594e000     548238843904
x20            0x6      6
x21            0x7fa0dd7000     548159713280
x22            0x7f9f7030f0     548135776496
x23            0x2      2
x24            0x7fa594e000     548238843904
x25            0x1      1
x26            0x7fa59254b8     548238677176
x27            0x2      2
x28            0x7f9f703120     548135776544
x29            0x7f9f702e90     548135775888
x30            0x7fa582e524     548237665572
sp             0x7f9f702e90     0x7f9f702e90
pc             0x7fa582e598     0x7fa582e598 <__GI_raise+176>
cpsr           0x0      [ EL=0 ]
fpsr           0x13     19
fpcr           0x0      0
(gdb) continue
Continuing.
Unable to fetch general registers.: No such process.
Unable to fetch general registers.: No such process.
(gdb) [Thread 0x7f267fc1d0 (LWP 7236) exited]
[Thread 0x7f26ffd1d0 (LWP 7232) exited]
[Thread 0x7f27fff1d0 (LWP 7224) exited]
[Thread 0x7f38ff91d0 (LWP 7223) exited]
[Thread 0x7f397fa1d0 (LWP 7222) exited]
[Thread 0x7f39ffb1d0 (LWP 7221) exited]
[Thread 0x7f3a7fc1d0 (LWP 7220) exited]
[Thread 0x7f3affd1d0 (LWP 7219) exited]
[Thread 0x7f3b7fe1d0 (LWP 7218) exited]
[Thread 0x7f3bfff1d0 (LWP 7217) exited]
[Thread 0x7f48ff91d0 (LWP 7216) exited]
[Thread 0x7f497fa1d0 (LWP 7215) exited]
[Thread 0x7f49ffb1d0 (LWP 7214) exited]
[Thread 0x7f4a7fc1d0 (LWP 7213) exited]
[Thread 0x7f817fa1d0 (LWP 7199) exited]
[Thread 0x7f81ffb1d0 (LWP 7198) exited]
[Thread 0x7f827fc1d0 (LWP 7197) exited]
[Thread 0x7f82ffd1d0 (LWP 7196) exited]
[Thread 0x7f837fe1d0 (LWP 7195) exited]
[Thread 0x7f83fff1d0 (LWP 7194) exited]
[Thread 0x7f88c0e1d0 (LWP 7193) exited]
[Thread 0x7f8940f1d0 (LWP 7192) exited]
[Thread 0x7f12ffd1d0 (LWP 7191) exited]
[Thread 0x7f127fc1d0 (LWP 7190) exited]
[Thread 0x7f11ffb1d0 (LWP 7189) exited]
[Thread 0x7f117fa1d0 (LWP 7188) exited]
[Thread 0x7ee2ffd1d0 (LWP 7135) exited]
[Thread 0x7ee37fe1d0 (LWP 7134) exited]
[Thread 0x7ee3fff1d0 (LWP 7131) exited]
[Thread 0x7ef97fa1d0 (LWP 7125) exited]
[Thread 0x7efa7fc1d0 (LWP 7121) exited]
[Thread 0x7f89c101d0 (LWP 7037) exited]
[Thread 0x7f8a4111d0 (LWP 7024) exited]
[Thread 0x7f8ae811d0 (LWP 7023) exited]
[Thread 0x7f917021d0 (LWP 7016) exited]
[Thread 0x7f91f031d0 (LWP 7015) exited]
[Thread 0x7f927041d0 (LWP 7014) exited]
[Thread 0x7f92f051d0 (LWP 7013) exited]
[Thread 0x7f93f551d0 (LWP 7012) exited]
[Thread 0x7f947561d0 (LWP 7011) exited]
[Thread 0x7f94ff91d0 (LWP 7010) exited]
[Thread 0x7f957fa1d0 (LWP 7009) exited]
[Thread 0x7f95ffb1d0 (LWP 7008) exited]
[Thread 0x7f967fc1d0 (LWP 7007) exited]
[Thread 0x7f96ffd1d0 (LWP 7006) exited]
[Thread 0x7f977fe1d0 (LWP 7005) exited]
[Thread 0x7f97fff1d0 (LWP 7004) exited]
[Thread 0x7f9ceff1d0 (LWP 7003) exited]
[Thread 0x7f9d7001d0 (LWP 7002) exited]
[Thread 0x7f9df011d0 (LWP 7001) exited]
[Thread 0x7f9e7021d0 (LWP 7000) exited]
[Thread 0x7f9ef031d0 (LWP 6999) exited]
[Thread 0x7fa16081d0 (LWP 6977) exited]
[Thread 0x7fa1e091d0 (LWP 6976) exited]
[Thread 0x7fa260a1d0 (LWP 6975) exited]
[Thread 0x7fa2e0b1d0 (LWP 6974) exited]
[Thread 0x7fa31ef430 (LWP 6973) exited]

Program terminated with signal SIGABRT, Aborted.

Tegrastats, a drastic change happens in the middle, RAM then stabilizes at new rate and CPU’s go back to normal.

RAM 1721/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [42%@1267,off,off,41%@1267,37%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@38.5C BCPU@42C thermal@41.1C
RAM 1722/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [50%@1267,off,off,41%@1267,35%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [42%@1267,off,off,36%@1267,34%@1267,45%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@41.1C
RAM 1721/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [47%@1267,off,off,38%@1267,37%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.6C
RAM 1722/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [42%@1267,off,off,43%@1267,36%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [45%@1267,off,off,37%@1267,42%@1267,35%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.6C
RAM 1721/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [45%@1113,off,off,40%@1113,34%@1113,38%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@38.5C BCPU@42C thermal@40.8C
RAM 1721/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [44%@1267,off,off,38%@1267,40%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [48%@1267,off,off,43%@1267,34%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [47%@1267,off,off,37%@1267,38%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@38.5C BCPU@42C thermal@40.6C
RAM 1721/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [48%@1267,off,off,44%@1267,41%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.6C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [49%@1113,off,off,41%@1113,39%@1113,40%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [46%@1267,off,off,42%@1267,37%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [47%@1267,off,off,41%@1267,41%@1267,45%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@38.5C BCPU@42C thermal@41.1C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [45%@1267,off,off,38%@1267,37%@1267,43%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1721/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [43%@1267,off,off,42%@1267,43%@1267,43%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [42%@1267,off,off,43%@1267,40%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [47%@1267,off,off,40%@1267,41%@1267,43%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@41.1C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [43%@1267,off,off,37%@1267,36%@1267,44%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@38.5C BCPU@42C thermal@40.8C
RAM 1721/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [46%@1267,off,off,44%@1267,40%@1267,42%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1723/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [49%@1267,off,off,46%@1267,45%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1722/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [44%@1267,off,off,40%@1267,37%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1723/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [45%@1267,off,off,43%@1267,37%@1267,42%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1726/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [67%@1267,off,off,66%@1267,65%@1267,59%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1725/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [45%@1267,off,off,42%@1267,37%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1725/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [45%@1267,off,off,41%@1267,43%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1727/3825MB (lfb 175x4MB) SWAP 16/1913MB (cached 0MB) CPU [51%@1267,off,off,45%@1267,48%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@40.8C
RAM 1751/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [61%@1267,off,off,71%@1267,69%@1267,63%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1758/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [60%@1267,off,off,61%@1267,55%@1267,75%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1758/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [48%@1267,off,off,39%@1267,38%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@41.1C
RAM 1761/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [66%@1267,off,off,52%@1267,48%@1267,53%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@40.8C
RAM 1759/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [49%@1267,off,off,59%@1267,44%@1267,43%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1759/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [47%@1267,off,off,40%@1267,40%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1759/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [46%@1113,off,off,41%@1113,39%@1113,42%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1759/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [47%@1113,off,off,42%@1113,39%@1113,42%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42C MCPU@42C PMIC@50C GPU@39C BCPU@42C thermal@40.8C
RAM 1763/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [68%@1267,off,off,57%@1267,56%@1267,57%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1789/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [81%@1267,off,off,82%@1267,78%@1267,82%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.4C
RAM 1829/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [96%@1267,off,off,92%@1267,92%@1267,95%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.4C
RAM 1866/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [89%@1267,off,off,87%@1267,93%@1267,91%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.9C
RAM 1882/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [86%@1267,off,off,86%@1267,84%@1267,86%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.9C
RAM 1897/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [81%@1267,off,off,74%@1267,81%@1267,70%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.4C
RAM 1892/3825MB (lfb 175x4MB) SWAP 17/1913MB (cached 0MB) CPU [87%@1267,off,off,86%@1267,87%@1267,89%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.9C
RAM 1908/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [73%@1267,off,off,67%@1267,66%@1267,72%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.6C
RAM 1908/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [89%@1267,off,off,86%@1267,89%@1267,81%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.9C
RAM 1917/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [68%@1267,off,off,70%@1267,63%@1267,60%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 1909/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [52%@1267,off,off,46%@1267,49%@1267,44%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 1911/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [59%@1267,off,off,44%@1267,47%@1267,50%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.6C
RAM 1905/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [73%@1267,off,off,55%@1267,53%@1267,61%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.6C
RAM 1905/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [41%@1113,off,off,38%@1113,37%@1113,44%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.6C
RAM 1904/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [44%@1113,off,off,39%@1113,37%@1113,41%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1904/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [42%@1267,off,off,35%@1267,36%@1267,42%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1904/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [43%@1267,off,off,42%@1267,37%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.1C
RAM 1911/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [54%@1267,off,off,52%@1267,51%@1267,46%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.1C
RAM 1923/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [99%@1267,off,off,54%@1267,59%@1267,57%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.6C
RAM 1924/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [88%@1267,off,off,49%@1267,56%@1267,69%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 1940/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [86%@1267,off,off,82%@1267,91%@1267,92%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.9C
RAM 1978/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [78%@1267,off,off,83%@1267,56%@1267,60%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 1983/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [66%@1267,off,off,88%@1267,67%@1267,62%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.6C
RAM 2023/3825MB (lfb 175x4MB) SWAP 18/1913MB (cached 0MB) CPU [97%@1267,off,off,93%@1267,92%@1267,95%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.6C
RAM 2051/3825MB (lfb 175x4MB) SWAP 19/1913MB (cached 0MB) CPU [85%@1267,off,off,79%@1267,77%@1267,100%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.9C
RAM 2070/3825MB (lfb 175x4MB) SWAP 19/1913MB (cached 0MB) CPU [84%@1267,off,off,87%@1267,81%@1267,88%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.9C
RAM 2084/3825MB (lfb 175x4MB) SWAP 19/1913MB (cached 0MB) CPU [79%@1267,off,off,96%@1267,74%@1267,74%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@42.1C
RAM 2093/3825MB (lfb 175x4MB) SWAP 19/1913MB (cached 0MB) CPU [76%@1267,off,off,82%@1267,72%@1267,69%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@39.5C BCPU@43.5C thermal@41.8C
RAM 2129/3825MB (lfb 175x4MB) SWAP 20/1913MB (cached 0MB) CPU [98%@1267,off,off,86%@1267,80%@1267,82%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@41.9C
RAM 2160/3825MB (lfb 175x4MB) SWAP 20/1913MB (cached 0MB) CPU [99%@1267,off,off,98%@1267,99%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.4C
RAM 2175/3825MB (lfb 175x4MB) SWAP 20/1913MB (cached 0MB) CPU [97%@1267,off,off,98%@1267,100%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.4C
RAM 2203/3825MB (lfb 175x4MB) SWAP 21/1913MB (cached 0MB) CPU [100%@1267,off,off,100%@1267,98%@1267,97%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.4C
RAM 2227/3825MB (lfb 175x4MB) SWAP 21/1913MB (cached 0MB) CPU [98%@1267,off,off,98%@1267,98%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.7C
RAM 2241/3825MB (lfb 175x4MB) SWAP 21/1913MB (cached 0MB) CPU [100%@1267,off,off,98%@1267,100%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.7C
RAM 2255/3825MB (lfb 175x4MB) SWAP 21/1913MB (cached 0MB) CPU [97%@1267,off,off,97%@1267,97%@1267,100%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.7C
RAM 2262/3825MB (lfb 175x4MB) SWAP 21/1913MB (cached 0MB) CPU [96%@1267,off,off,97%@1267,94%@1267,98%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44.5C MCPU@44.5C PMIC@50C GPU@40C BCPU@44.5C thermal@42.4C
RAM 2272/3825MB (lfb 175x4MB) SWAP 22/1913MB (cached 0MB) CPU [100%@1267,off,off,100%@1267,100%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.4C
RAM 2285/3825MB (lfb 175x4MB) SWAP 22/1913MB (cached 0MB) CPU [100%@1267,off,off,99%@1267,99%@1267,100%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44.5C MCPU@44.5C PMIC@50C GPU@40.5C BCPU@44.5C thermal@42.7C
RAM 2298/3825MB (lfb 175x4MB) SWAP 22/1913MB (cached 0MB) CPU [100%@1267,off,off,99%@1267,100%@1267,100%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.7C
RAM 2291/3825MB (lfb 175x4MB) SWAP 22/1913MB (cached 0MB) CPU [97%@1267,off,off,94%@1267,94%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.4C
RAM 2304/3825MB (lfb 175x4MB) SWAP 22/1913MB (cached 0MB) CPU [93%@1267,off,off,93%@1267,92%@1267,90%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44.5C MCPU@44.5C PMIC@50C GPU@40C BCPU@44.5C thermal@42.4C
RAM 2313/3825MB (lfb 175x4MB) SWAP 22/1913MB (cached 0MB) CPU [97%@1267,off,off,95%@1267,95%@1267,94%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.4C
RAM 2321/3825MB (lfb 175x4MB) SWAP 23/1913MB (cached 0MB) CPU [93%@1267,off,off,91%@1267,91%@1267,95%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.6C
RAM 2329/3825MB (lfb 175x4MB) SWAP 23/1913MB (cached 0MB) CPU [94%@1267,off,off,92%@1267,96%@1267,93%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.6C
RAM 2335/3825MB (lfb 175x4MB) SWAP 23/1913MB (cached 0MB) CPU [98%@1267,off,off,100%@1267,94%@1267,94%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.7C
RAM 2344/3825MB (lfb 175x4MB) SWAP 23/1913MB (cached 0MB) CPU [99%@1267,off,off,97%@1267,96%@1267,99%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44.5C MCPU@44.5C PMIC@50C GPU@40.5C BCPU@44.5C thermal@42.4C
RAM 2360/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [97%@1267,off,off,94%@1267,91%@1267,95%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44.5C MCPU@44.5C PMIC@50C GPU@40C BCPU@44.5C thermal@42.6C
RAM 2378/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [98%@1267,off,off,99%@1267,96%@1267,94%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44.5C MCPU@44.5C PMIC@50C GPU@40.5C BCPU@44.5C thermal@42.6C
RAM 2382/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [97%@1267,off,off,95%@1267,97%@1267,97%@1267] EMC_FREQ 0% GR3D_FREQ 44% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.9C
RAM 2295/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [87%@1267,off,off,93%@1267,87%@1267,82%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.9C
RAM 2299/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [72%@1267,off,off,73%@1267,68%@1267,92%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.6C
RAM 2303/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [79%@1267,off,off,66%@1267,87%@1267,68%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40C BCPU@44C thermal@42.4C
RAM 2314/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [100%@1267,off,off,81%@1267,80%@1267,79%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.4C
RAM 2319/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [84%@1267,off,off,86%@1267,85%@1267,97%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.4C
RAM 2327/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [78%@1267,off,off,93%@1267,74%@1267,85%@1267] EMC_FREQ 0% GR3D_FREQ 6% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.6C
RAM 2334/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [78%@1267,off,off,81%@1267,71%@1267,89%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@44C MCPU@44C PMIC@50C GPU@40.5C BCPU@44C thermal@42.4C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [77%@1267,off,off,82%@1267,70%@1267,68%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@40C BCPU@43.5C thermal@42.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [42%@1267,off,off,46%@1267,39%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 6% PLL@43.5C MCPU@43.5C PMIC@50C GPU@40C BCPU@43.5C thermal@41.8C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [47%@1267,off,off,47%@1267,49%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43.5C MCPU@43.5C PMIC@50C GPU@40C BCPU@43.5C thermal@42.1C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [46%@1267,off,off,42%@1267,39%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.8C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [50%@1267,off,off,44%@1267,41%@1267,44%@1267] EMC_FREQ 0% GR3D_FREQ 6% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.8C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [43%@1267,off,off,40%@1267,39%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.8C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1267,off,off,37%@1267,39%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.8C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [41%@1267,off,off,34%@1267,38%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 6% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.8C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [43%@1113,off,off,35%@1113,38%@1113,43%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [43%@1267,off,off,40%@1267,40%@1267,41%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [41%@1267,off,off,37%@1267,39%@1267,44%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [45%@1267,off,off,38%@1267,40%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [42%@1267,off,off,37%@1267,34%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [42%@1267,off,off,38%@1267,40%@1267,42%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@40C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1113,off,off,34%@1113,35%@1113,40%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1113,off,off,43%@1113,43%@1113,37%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [43%@1267,off,off,43%@1267,36%@1267,42%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [46%@1267,off,off,43%@1267,43%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1267,off,off,43%@1267,39%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [38%@1267,off,off,42%@1267,38%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [42%@1267,off,off,43%@1267,35%@1267,33%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1113,off,off,42%@1267,39%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [45%@1267,off,off,41%@1267,38%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [43%@1267,off,off,36%@1267,41%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [40%@1267,off,off,38%@1267,39%@1267,42%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [41%@1267,off,off,40%@1267,35%@1267,35%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1267,off,off,36%@1267,33%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [47%@1267,off,off,40%@1267,38%@1267,35%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [48%@1267,off,off,36%@1267,39%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [47%@1267,off,off,41%@1267,37%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [42%@1113,off,off,43%@1113,42%@1113,39%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [42%@1113,off,off,35%@1113,38%@1113,37%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1267,off,off,37%@1267,43%@1267,33%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1267,off,off,36%@1267,40%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [47%@1267,off,off,40%@1267,42%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [44%@1267,off,off,41%@1267,48%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [45%@1267,off,off,42%@1267,36%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [48%@1267,off,off,41%@1267,42%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.4C
RAM 2337/3825MB (lfb 175x4MB) SWAP 24/1913MB (cached 0MB) CPU [46%@1267,off,off,39%@1267,38%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [44%@1267,off,off,45%@1267,40%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.3C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [46%@1267,off,off,41%@1267,45%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [49%@1267,off,off,41%@1267,41%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [47%@1267,off,off,43%@1113,40%@1113,39%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2338/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [43%@1267,off,off,42%@1267,43%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39.5C BCPU@42.5C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [48%@1267,off,off,45%@1267,39%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [45%@1267,off,off,38%@1267,44%@1267,35%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [46%@1267,off,off,39%@1267,40%@1267,38%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [47%@1267,off,off,41%@1267,36%@1267,39%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39.5C BCPU@42.5C thermal@41.6C
RAM 2337/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [41%@1267,off,off,40%@1267,47%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39C BCPU@42.5C thermal@41.6C
RAM 2295/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [48%@1267,off,off,45%@1267,46%@1267,45%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39.5C BCPU@42.5C thermal@41.6C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [46%@1267,off,off,40%@1267,39%@1267,44%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39C BCPU@43C thermal@41.6C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [41%@1267,off,off,41%@1267,35%@1267,41%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39.5C BCPU@42.5C thermal@41.6C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [45%@1113,off,off,40%@1113,36%@1113,37%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.4C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [42%@1267,off,off,37%@1267,39%@1267,36%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.4C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [46%@1267,off,off,36%@1267,39%@1267,37%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.4C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [45%@1267,off,off,40%@1267,38%@1267,40%@1267] EMC_FREQ 0% GR3D_FREQ 0% PLL@43C MCPU@43C PMIC@50C GPU@39.5C BCPU@43C thermal@41.4C
RAM 2293/3825MB (lfb 175x4MB) SWAP 25/1913MB (cached 0MB) CPU [43%@1113,off,off,43%@1113,38%@1113,36%@1113] EMC_FREQ 0% GR3D_FREQ 0% PLL@42.5C MCPU@42.5C PMIC@50C GPU@39.5C BCPU@42.5C thermal@41.6C

Journalctl additionally changed

nvargus-daemon[6973]: double free or corruption (out)
systemd[1]: nvargus-daemon.service: Main process exited, code=killed, status=6/ABRT
systemd[1]: nvargus-daemon.service: Failed with result 'signal'.
systemd[1]: nvargus-daemon.service: Service hold-off time over, scheduling restart.
systemd[1]: nvargus-daemon.service: Scheduled restart job, restart counter is at 1.
systemd[1]: Stopped Argus daemon.
systemd[1]: Started Argus daemon.

[EDIT]
Ran again till failure with the same pipeline as above and got this for the gdb print. It seg faulted instead of abort but still is related to what seems to be libnvscf.so

Thread 14 "SCF Execution" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f82ffd1d0 (LWP 6782)]
0x0000007f908ce424 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
(gdb) where
#0  0x0000007f908ce424 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#1  0x0000007f907f0fec in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#2  0x0000007f9084313c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#3  0x0000007f908549e0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#4  0x0000007f908378bc in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#5  0x0000007f90803074 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#6  0x0000007f908032c4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#7  0x0000007f90801dd8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#8  0x0000007f9148ac20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#9  0x0000007f911f5088 in start_thread (arg=0x7f8b49721f) at pthread_create.c:463
#10 0x0000007f916600cc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

ShaneCCC · November 13, 2024, 5:14am

Please try below command. Modify the sensor-id for launch others.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=30/1' ! nvvidconv flip-method=0 ! nvvidconv ! xvimagesink

MatthewIrvin1 · November 13, 2024, 6:10pm

I ran the command you provided three separate times running all 4 cameras, each had the following errors below.

First test run:

gst-launch-1.0 nvarguscamerasrc sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=30/1' ! nvvidconv flip-method=0 ! nvvidconv ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 1 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: ERROR OCCURRED
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: TIMEOUT
Additional debug info:
Argus Error Status
Execution ended after 0:27:34.392914618
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 137)
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Caught SIGSEGV
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 137)
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
#0  0x0000007f9ea505bc in futex_reltimed_wait_cancelable (private=<optimized out>, reltime=0x7f00000000, expected=0, futex_word=0x557a5c7038)
#1  0x0000007f9ea505bc in __pthread_cond_wait_common (abstime=0x7fdd3e7188, mutex=0x557a5aa520, cond=0x557a5c7010) at pthread_cond_wait.c:533
#2  0x0000007f9ea505bc in __pthread_cond_timedwait (cond=0x557a5c7010, mutex=0x557a5aa520, abstime=0x7fdd3e7188) at pthread_cond_wait.c:667
#3  0x0000007f9def5ef8 in NvOsConditionWaitTimeout ()
#4  0x0000007f9df8c14c in nvcamerautils::ConditionVariable::wait(nvcamerautils::Mutex const&, nvcamerautils::UnscaledTimeout const&) const ()
#5  0x0000007f9e3c4e44 in  ()
#6  0x0000007f9e3c7908 in  ()
#7  0x0000007f9e343e4c in  ()
#8  0x0000007f9ed366a4 in  ()
#9  0x0000007f9e290ab0 in  () at /usr/lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#10 0x0000007f9ed08750 in _gst_debug_min ()
Spinning.  Please run 'gdb gst-launch-1.0 19018' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

Running gdb gst-launch-1.0 19018 as stated gave the following

gdb gst-launch-1.0 19018
GNU gdb (Ubuntu 8.1.1-0ubuntu1) 8.1.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from gst-launch-1.0...(no debugging symbols found)...done.
Attaching to program: /usr/bin/gst-launch-1.0, process 19018
[New LWP 19020]
[New LWP 19021]
[New LWP 19023]
[New LWP 19025]
[New LWP 19038]
[New LWP 19040]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
0x0000007f9ea505bc in futex_reltimed_wait_cancelable (private=<optimized out>, 
    reltime=0x7f00000000, expected=0, futex_word=0x557a5c7038)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:142
142	../sysdeps/unix/sysv/linux/futex-internal.h: No such file or directory.
(gdb) bt
#0  0x0000007f9ea505bc in futex_reltimed_wait_cancelable (private=<optimized out>, reltime=0x7f00000000, expected=0, futex_word=0x557a5c7038)
    at ../sysdeps/unix/sysv/linux/futex-internal.h:142
#1  0x0000007f9ea505bc in __pthread_cond_wait_common (abstime=0x7fdd3e7188, mutex=0x557a5aa520, cond=0x557a5c7010) at pthread_cond_wait.c:533
#2  0x0000007f9ea505bc in __pthread_cond_timedwait (cond=0x557a5c7010, mutex=0x557a5aa520, abstime=0x7fdd3e7188) at pthread_cond_wait.c:667
#3  0x0000007f9def5ef8 in NvOsConditionWaitTimeout ()
    at /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#4  0x0000007f9df8c14c in nvcamerautils::ConditionVariable::wait(nvcamerautils::Mutex const&, nvcamerautils::UnscaledTimeout const&) const ()
    at /usr/lib/aarch64-linux-gnu/tegra/libnvcamerautils.so
#5  0x0000007f9e3c4e44 in  ()
    at /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
#6  0x0000007f9e3c7908 in  ()
    at /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
#7  0x0000007f9e343e4c in  ()
    at /usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so
#8  0x0000007f9ed366a4 in  ()
    at /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstnvarguscamerasrc.so
#9  0x0000007f9e290ab0 in  () at /usr/lib/aarch64-linux-gnu/libgstbase-1.0.so.0
#10 0x0000007f9ed08750 in _gst_debug_min ()
    at /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0

Then I have the gdb log from nvargus-daemon

Thread 6 "SCF Execution" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f517fa1d0 (LWP 18878)]
0x0000007f9d1bd6e4 in __GI___pthread_mutex_lock (mutex=0x250000007f) at pthread_mutex_lock.c:67
67	pthread_mutex_lock.c: No such file or directory.
(gdb) bt
#0  0x0000007f9d1bd6e4 in __GI___pthread_mutex_lock (mutex=0x250000007f) at pthread_mutex_lock.c:67
#1  0x0000007f9d44fb50 in NvOsMutexLock () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#2  0x0000007f9cac8b38 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#3  0x0000007f9c959294 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#4  0x0000007f9c7efcb8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#5  0x0000007f9c81a6d8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#6  0x0000007f9c7fd8ec in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#7  0x0000007f9c7c9074 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#8  0x0000007f9c7c92c4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#9  0x0000007f9c7c7dd8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#10 0x0000007f9d450c20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#11 0x0000007f9d1bb088 in start_thread (arg=0x7f9745d21f) at pthread_create.c:463
#12 0x0000007f9d6260cc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

2nd the gst-launch error were exactly the same, the only difference was in the gdb log from nvagus-daemon

Thread 13 "SCF Execution" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f6dffb1d0 (LWP 7314)]
__memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:238
238	../sysdeps/aarch64/multiarch/../memcpy.S: No such file or directory.
(gdb) bt
#0  __memcpy_generic () at ../sysdeps/aarch64/multiarch/../memcpy.S:238
#1  0x0000007f7ba5fa64 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_replace(unsigned long, unsigned long, char const*, unsigned long) () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#2  0x0000007f75c208b8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
#3  0x0000007f75c4816c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
#4  0x0000007f75c49d60 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
#5  0x0000007f75c31428 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
#6  0x0000007f75c29b38 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvargus.so
#7  0x0000007f7af0cde4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#8  0x0000007f7af13f84 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#9  0x0000007f7af407a4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#10 0x0000007f7af588ec in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#11 0x0000007f7af24074 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#12 0x0000007f7af242c4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#13 0x0000007f7af22dd8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#14 0x0000007f7bbabc20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#15 0x0000007f7b916088 in start_thread (arg=0x7f75bb821f) at pthread_create.c:463
#16 0x0000007f7bd810cc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

the 3rd instance as well had no difference in gst-launch errors except for the gdb log

Thread 14 "SCF Execution" received signal SIGABRT, Aborted.
[Switching to Thread 0x7f9e7fc1d0 (LWP 6566)]
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x0000007face2f974 in __GI_abort () at abort.c:79
#2  0x0000007face6872c in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7facf297e0 "%s\n") at ../sysdeps/posix/libc_fatal.c:181
#3  0x0000007face6eaa4 in malloc_printerr (str=str@entry=0x7facf24fa8 "munmap_chunk(): invalid pointer") at malloc.c:5342
#4  0x0000007face6f054 in munmap_chunk (p=<optimized out>) at malloc.c:2846
#5  0x0000007fac3abeb0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#6  0x0000007fac36ef8c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#7  0x0000007fac36e650 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#8  0x0000007fac1fedc8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#9  0x0000007fac0e088c in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#10 0x0000007fac0f6f40 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#11 0x0000007fac08ccdc in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#12 0x0000007fac0c09e0 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#13 0x0000007fac0a38bc in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#14 0x0000007fac06f074 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#15 0x0000007fac06f2c4 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#16 0x0000007fac06ddd8 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvscf.so
#17 0x0000007faccf6c20 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#18 0x0000007faca61088 in start_thread (arg=0x7fa6cf721f) at pthread_create.c:463
#19 0x0000007facecc0cc in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone.S:78

ShaneCCC · November 14, 2024, 5:01am

Looks like none of camera able to work by run the daemon with gdb.
Could you run the daemon without gdb to check.

MatthewIrvin1 · November 14, 2024, 5:33pm

When running the command provided on 4 cams without attaching gdb to nvargus-daemon it still has the same result. It will display for a time before getting the errors.

gst-launch-1.0 nvarguscamerasrc sensor-id=2 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=30/1' ! nvvidconv flip-method=0 ! nvvidconv ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 2 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: ERROR OCCURRED
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: DISCONNECTED
Additional debug info:
Argus Error Status
Execution ended after 3:20:32.967972769
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
GST_ARGUS: Cleaning up
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Caught SIGSEGV
Caught SIGSEGV
#0  0x0000007f7ecbdd5c in __waitpid (pid=<optimized out>, stat_loc=0x7fceb8b814, options=<optimized out>) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x0000007f7ecf92e0 in g_on_error_stack_trace ()
#2  0x00000055946dfc3c in  ()
Spinning.  Please run 'gdb gst-launch-1.0 9468' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
Spinning.  Please run 'gdb gst-launch-1.0 9468' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

ShaneCCC · November 15, 2024, 2:32am

Please confirm each single camera working well for a while.

MatthewIrvin1 · November 18, 2024, 8:56pm

Each camera has recorded for my required minimum 8+ hours when recording by themselves. The issue seems to only occur when multiple cameras are using nvargus-daemon at once.

ShaneCCC · November 19, 2024, 6:50am

Please apply this attached lib to try.
libnvargus.so.32.7.5.so (1.1 MB)

MatthewIrvin1 · November 20, 2024, 7:38pm

Using the file provided, I was able to record a 14hr+ session with all for cameras from the original pipeline provided in the starting post.

On a second attempt to record after a fresh reboot, the feed with the same pipeline and it did seem to freeze but for a different reason. After about 8 minutes of recording, it froze and the sudo journalctl -u nvargus-daemon -f logs were filled with the following error constantly being repeated

 === python3[6815]: CameraProvider initialized (0x7f74a42000)LSC: LSC surface is not based on full res!
Nov 20 11:01:25 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:25 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:25 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:30 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:30 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:30 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:30 jetson-desktop nvargus-daemon[5048]: LSC: LSC surface is not based on full res!
Nov 20 11:01:31 jetson-desktop nvargus-daemon[5048]: PowerServiceCore:handleRequests: timePassed = 1156
Nov 20 11:09:04 jetson-desktop nvargus-daemon[5048]: (Argus) Error OverFlow: Too many pending events, ignoring new events (in src/api/EventProviderImpl.cpp, function addEvent(), line 158)
Nov 20 11:09:04 jetson-desktop nvargus-daemon[5048]: (Argus) Error OverFlow: Too many pending events, ignoring new events (in src/api/EventProviderImpl.cpp, function addEvent(), line 158)
Nov 20 11:09:04 jetson-desktop nvargus-daemon[5048]: (Argus) Error OverFlow: Too many pending events, ignoring new events (in src/api/EventProviderImpl.cpp, function addEvent(), line 158)

When looking at /var/log/syslog we can see

Nov 20 11:09:04 jetson-desktop nvargus-daemon[5048]: (Argus) Error OverFlow: Too many pending events, ignoring new events (in src/api/EventProviderImpl.cpp, function addEvent(), line 158)
Nov 20 11:12:10 jetson-desktop nvargus-daemon[5048]: message repeated 44482 times: [ (Argus) Error OverFlow: Too many pending events, ignoring new events (in src/api/EventProviderImpl.cpp, function addEvent(), line 158)]

nvargus-daemon did not crash/restart. After killing the process running the cameras and restarting it the camera feeds were able to keep on recording. Unsure if this is the files issue or a gstreamer issue but I have increased the max-size-buffers on queue. Will continue testing to ensure everything is working smoothly

MatthewIrvin1 · November 26, 2024, 11:43am

After further testing I haven’t seen any problems since using this new lib file so I will mark it as the solution to my problems, if anything were to arise, I will make a new forum post. Could you perhaps provide any information on what changed between this file and the original to fix the issue I was having?

system · December 17, 2024, 11:54am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Some basic problems with nvarguscamerasrc plugin for gstreamer Jetson AGX Xavier camera	15	1468	January 26, 2024
Jetpack 5.0.2 nvarguscamerasrc streaming stopped, reason not-negotiated (-4) Jetson Xavier NX camera , wayland	9	1812	January 18, 2023
Argus errors on some boots Jetson TX2 camera , gstreamer	8	922	March 28, 2023
Inconsistent gStreamer with IMX219/477 Jetson Xavier NX gstreamer	5	1430	October 31, 2022
Can't take pictures on some cameras using nvarguscamerasrc Jetson Xavier NX camera	19	903	April 12, 2023
Gstreamer nvarguscamerasrc not working at TX2 Custom Board with Serdes Gmsl Camera Jetson TX2 camera , gstreamer , device-tree	2	1416	October 18, 2021
Jetson AGX Orin nvargus_nvraw doesn't show sensors Jetson AGX Orin camera	18	69	October 11, 2024
Camera crashes nvargus-daemon Jetson Nano camera	10	1279	August 10, 2022
Nvargus-daemon hanging - Jetson Nano 32.7.3 Jetson Nano camera , gstreamer	17	74	August 11, 2024
Camera: DPB and MjstreamingSocket read error Jetson TX1	33	4964	September 5, 2016

Nvargus-daemon crashing during recording

Related topics