SOM: Jetson TX2 PN 900-83310-0001-000 BV
Carrier: LI-TX1-CB-6CAM
Cameras on MIPI ports A-E: LI-IMX290-MIPI
Camera on MIPI port F: LI-LMH0341-MPI-DSER
We have recently updated our system from L4T 32.5.1 to 32.7.6 to support the last revision of jetson TX2’s that have PCN211181 memory.
We have been running fine for ~6 years on 32.5.1 but after the update from 32.5.1 to 32.7.6 we are seeing an nvidia argus daemon crash when initializing the cameras.
Our system uses the Jetson TX2 with a Leopard Imaging LI-TX1-CB-6CAM carrier board and L4T is pulled into our custom buildroot image. We also run a custom video pipeline that uses argus via it’s library interface as well as the provided v4l extension.
We use 6 total cameras, 5 LI-IMX290-MIPI cameras on ports A-E, and on port F there is a LI-LMH0341-MPI-DSER .
Were there any updates in Argus between those versions that might be affecting camera initialization/timing or possibly changes in what is expected in the device tree?
Feb 02 15:29:59 [5152]: NVARGUS STARTED, PID 5153, 0 CPUS
Feb 02 15:29:59 [5152]: STARTED, PID 5154
Feb 02 15:29:59 [5154]: executing on 4 cpus
Feb 02 15:29:59 [5153]: executing nvargus-daemon on 2 cpus
Feb 02 15:30:00 [5153]: === NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...=== [5154]: Connection established (7F76BB8130)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Feb 02 15:30:00 [5153]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
Feb 02 15:30:00 [5153]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
Feb 02 15:30:00 [5153]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
Feb 02 15:30:00 [5153]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module4
Feb 02 15:30:00 [5153]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module5
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: No module data found
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: No module data found
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: No module data found
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: No module data found
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: No module data found
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Feb 02 15:30:00 [5153]: NvPclHwGetModuleList: No module data found
Feb 02 15:30:01 [5153]: OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
Feb 02 15:30:01 [5153]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Feb 02 15:30:01 [5153]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Feb 02 15:30:01 [5153]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Feb 02 15:30:01 [5153]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Feb 02 15:30:01 [5153]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Feb 02 15:30:01 [5153]: ---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
Feb 02 15:30:01 [5153]: === [5154]: CameraProvider initialized (0x7f70d16170)
Feb 02 15:30:01 [5154]: 6 cameras available
Feb 02 15:30:01 [5154]: output stream 0xc0d8f80 0xc0a4f70 0xc12f401
Feb 02 15:30:01 [5154]: CONNECTED!
Feb 02 15:30:02 [5154]: output stream 0xc0d8c30 0xc0a2e10 0xc0a2a51
Feb 02 15:30:02 [5154]: CONNECTED!
Feb 02 15:30:02 [5154]: output stream 0xc0a3dc0 0xc0a4800 0xc0a3061
Feb 02 15:30:02 [5154]: CONNECTED!
Feb 02 15:30:02 [5154]: output stream 0xc0e0800 0xc0dadd0 0xc0e06a1
Feb 02 15:30:02 [5154]: CONNECTED!
Feb 02 15:30:02 [5154]: output stream 0xc0dbab0 0xc0dc4f0 0xc0db571
Feb 02 15:30:02 [5154]: CONNECTED!
Feb 02 15:30:03 [5153]: PowerServiceCore:handleRequests: timePassed = 636
Feb 02 15:30:03 [5153]: PowerServiceCore:handleRequests: timePassed = 618
Feb 02 15:30:04 [5153]: PowerServiceCore:handleRequests: timePassed = 613
Feb 02 15:30:05 kernel: sdi3841a 35-001a: sdi3841a_power_on: power on
Feb 02 15:30:05 systemd[1]: Started video streaming server.
Feb 02 15:30:05 kernel: sdi3841a 35-001a: [sdi3841a_set_mode]
Feb 02 15:30:05 kernel: sdi3841a 35-001a: [sdi3841a_start_streaming] dummy.
Feb 02 15:30:05 [5153]: PowerServiceCore:handleRequests: timePassed = 609
Feb 02 15:30:05 [5152]: SERVICE STARTED!
Feb 02 15:30:05 [5952]: NvMMLiteOpen : Block : BlockType = 4
Feb 02 15:30:05 [5952]: ===== NVMEDIA: NVENC =====
Feb 02 15:30:05 [5952]: NvMMLiteBlockCreate : Block : BlockType = 4
Feb 02 15:30:05 [5952]: Requested 8 buffers, got 8
Feb 02 15:30:05 [5952]: H264: Profile = 66, Level = 31
Feb 02 15:30:11 systemd[1]: Started NVIDIA Late Init Script.
Feb 02 15:30:11 systemd[1]: Startup finished in 9.857s (kernel) + 24.086s (userspace) = 33.944s.
Feb 02 15:30:11 systemd[1]: nv-late-init.service: Succeeded.
Feb 02 15:30:12 kernel: spmic-ldo0: disabling
Feb 02 15:30:12 kernel: spmic-ldo1: disabling
Feb 02 15:30:12 kernel: en-vdd-sd: disabling
Feb 02 15:30:12 kernel: vdd-usb0-5v: disabling
Feb 02 15:30:12 kernel: vdd-usb1-5v: disabling
Feb 02 15:30:12 kernel: vdd-fan: disabling
Feb 02 15:30:12 kernel: vdd-3v3: disabling
Feb 02 15:30:12 kernel: en-vdd-vcm-2v8: disabling
Feb 02 15:30:12 kernel: vdd-usb2-5v: disabling
Feb 02 15:30:12 kernel: vdd-sys-bl: disabling
Feb 02 15:31:52 [5153]: corrupted double-linked list
Feb 02 15:31:52 audit[5153]: ANOM_ABEND auid=4294967295 uid=1000 gid=1000 ses=4294967295 pid=5153 comm=53434620457865637574696F6E exe="/usr/sbin/nvargus-daemon" sig=6
Feb 02 15:31:52 kernel: audit_printk_skb: 15 callbacks suppressed
Feb 02 15:31:52 kernel: audit: type=1701 audit(1612279912.132:7): auid=4294967295 uid=1000 gid=1000 ses=4294967295 pid=5153 comm=53434620457865637574696F6E exe="/usr/sbin/nvargus-daemon" sig=6
Feb 02 15:31:52 [5154]: (Argus) Error FileOperationFailed: Failed socket read: Connection reset by peer (in src/rpc/socket/common/SocketUtils.cpp, function readSocket(), line 79)
Feb 02 15:31:52 [5154]: (Argus) Error FileOperationFailed: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
Feb 02 15:31:52 [5154]: (Argus) Error FileOperationFailed: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 340)
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
Feb 02 15:31:52 [5154]: (Argus) Error FileOperationFailed: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
Feb 02 15:31:52 [5154]: (Argus) Error FileOperationFailed: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
Feb 02 15:31:52 [5154]: (Argus) Error FileOperationFailed: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Feb 02 15:31:52 [5154]: Failed to acquire ISourceSettings interface!
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Feb 02 15:31:52 [5154]: Failed to acquire ISourceSettings interface!
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Feb 02 15:31:52 [5154]: Failed to acquire ISourceSettings interface!
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Feb 02 15:31:52 [5154]: Failed to acquire ISourceSettings interface!
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
Feb 02 15:31:52 [5154]: (Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Feb 02 15:31:52 [5154]: Failed to acquire ISourceSettings interface!
Feb 02 15:31:52 [5152]: NVARGUS DIED, STATUS 6
Feb 02 15:31:52 [5152]: KILLING 5154
Feb 02 15:31:52 kernel: sdi3841a 35-001a: [sdi3841a_stop_streaming] dummy
Feb 02 15:31:52 kernel: nvcsi 150c0000.nvcsi: csi4_cil_check_status (5) CILA_INTR_STATUS 0x00000044
Feb 02 15:31:52 kernel: nvcsi 150c0000.nvcsi: csi4_cil_check_status (5) CILA_ERR_INTR_STATUS 0x00000044
Feb 02 15:31:52 kernel: sdi3841a 35-001a: sdi3841a_power_off: power off