Hi Sandip,
This is Jason at D3. When we discussed this offline, we recommended trying to isolate the cameras or streaming Ethernet to determine where the crash is occurring that causes the reboot. The inspection of your logs looks like the crash originates with Ethernet. We’re you able to make any progress testing without the D3 modifications to the kernel?
Regards,
Jason
I am curious if you see “
quiet” in the output of “cat /proc/cmdline”? If so, then remove “quiet” from “/boot/extlinux/extlinux.conf” and you might see more useful logs.
No, as suggested earlier, I removed the quiet from the extlinux.conf file.
This is Jason at D3. When we discussed this offline, we recommended trying to isolate the cameras or streaming Ethernet to determine where the crash is occurring that causes the reboot. The inspection of your logs looks like the crash originates with Ethernet. We’re you able to make any progress testing without the D3 modifications to the kernel?
Hi Jason,
The crash is not observed on the unmodified kernel with the same L4T version running V4L2 cameras. With the bluedroid hack that was suggested earlier, the reboots still occur. From the syslogs shared yesterday, there is crash in the nvargus driver.
Let me know if there is anything you’d like me to test further.
Hi Sandip,
Do the cameras stop outputting for a time period before the Xavier reboot happens? Or is everything working as you would expect, then the system hits this error and reboots?
About how long into a run of the cameras streaming does this happen? Is there any consistency in the amount of time?
Can you try running just the D3 provided code and see if the crash happens with an example of displaying the cameras on the monitor?
Thanks,
Jason
Hi Jason,
Yes, one of the cameras stops outputting for a few minutes. It freezes and then the system reboots.
There is no consistency in the time it takes before the streaming stops.
Can you try running just the D3 provided code and see if the crash happens with an example of displaying the cameras on the monitor?
Yes we can try this and get back to you.
Hi again,
I see the crash once again with the network streaming, this time the syslog is more informative:
Nov 2 15:50:13 diesel dbus-daemon[27159]: [session uid=1000 pid=27159] Activating service name='org.freedesktop.Notifications' requested by ':1.5' (uid=1000 pid=12768 comm="/usr/bin/python3 /usr/bin/colcon build --merge-ins")
Nov 2 15:50:13 diesel org.freedesktop.Notifications[27159]: Unable to init server: Could not connect: Connection refused
Nov 2 15:50:13 diesel notify-osd[12993]: cannot open display:
Nov 2 15:50:13 diesel dbus-daemon[27159]: [session uid=1000 pid=27159] Activated service 'org.freedesktop.Notifications' failed: Process org.freedesktop.Notifications exited with status 1
Nov 2 15:50:27 diesel nvargus-daemon[6827]: CAM: serial no file already exists, skips storing again=== video_source[11444]: CameraProvider initialized (0x7f54be9be0)CAM: serial no file already exists, skips storing again=== video_source[11444]: CameraProvider destroyed (0x7f54be9be0)=== video_source[11444]: Connection closed (7FAE36B1D0)=== video_source[11444]: Connection cleaned up (7FAE36B1D0)=== imp_sndstream[13011]: Connection established (7FAE36B1D0)=== imp_sndstream[13012]: Connection established (7FB21B91D0)=== imp_sndstream[13010]: Connection established (7FB01B51D0)=== imp_sndstream[13013]: Connection established (7F98A001D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
Nov 2 15:50:27 diesel nvargus-daemon[6827]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
Nov 2 15:50:27 diesel nvargus-daemon[6827]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module2
Nov 2 15:50:27 diesel nvargus-daemon[6827]: OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module3
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: No module data found
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: No module data found
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: No module data found
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
Nov 2 15:50:27 diesel nvargus-daemon[6827]: NvPclHwGetModuleList: No module data found
Nov 2 15:50:27 diesel nvargus-daemon[6827]: OFParserGetVirtualDevice: virtual device driver node not found in proc device-tree: /proc/device-tree/tegra-virtual-camera-platform/modules/module0/drivernode1
Nov 2 15:50:27 diesel nvargus-daemon[6827]: OFParserGetVirtualDevice: virtual device driver node not found in proc device-tree: /proc/device-tree/tegra-virtual-camera-platform/modules/module1/drivernode0
Nov 2 15:50:27 diesel nvargus-daemon[6827]: ---- imager: Found override file [/var/nvidia/nvcam/settings/d3_0_ov10640.isp]. ----
Nov 2 15:50:27 diesel nvargus-daemon[6827]: CAM: serial no file already exists, skips storing again---- imager: Found override file [/var/nvidia/nvcam/settings/d3_1_ov10640.isp]. ----
Nov 2 15:50:27 diesel nvargus-daemon[6827]: CAM: serial no file already exists, skips storing again---- imager: Found override file [/var/nvidia/nvcam/settings/d3_2_ov10640.isp]. ----
Nov 2 15:50:27 diesel nvargus-daemon[6827]: CAM: serial no file already exists, skips storing again---- imager: Found override file [/var/nvidia/nvcam/settings/d3_3_ov10640.isp]. ----
Nov 2 15:50:27 diesel kernel: [14805.777441] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2
Nov 2 15:50:27 diesel ovpn-client-uk[6417]: AEAD Decrypt error: bad packet ID (may be a replay): [ #430747 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Nov 2 15:50:27 diesel ovpn-client-uk[6417]: AEAD Decrypt error: bad packet ID (may be a replay): [ #430770 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Nov 2 15:50:27 diesel ovpn-client-uk[6417]: AEAD Decrypt error: bad packet ID (may be a replay): [ #430771 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Nov 2 15:50:29 diesel nvargus-daemon[6827]: CAM: serial no file already exists, skips storing again=== imp_sndstream[13011]: CameraProvider initialized (0x7f54b43bd0)=== imp_sndstream[13012]: CameraProvider initialized (0x7f6801bc80)=== imp_sndstream[13010]: CameraProvider initialized (0x7f643d8e60)=== imp_sndstream[13013]: CameraProvider initialized (0x7f6001d1b0)CAM: serial no file already exists, skips storing againCAM: serial no file already exists, skips storing againCAM: serial no file already exists, skips storing againCAM: serial no file already exists, skips storing againPowerServiceCore:handleRequests: timePassed = 1026
Nov 2 15:50:30 diesel nvargus-daemon[6827]: PowerServiceCore:handleRequests: timePassed = 1016
Nov 2 15:50:31 diesel nvargus-daemon[6827]: PowerServiceCore:handleRequests: timePassed = 1026
@D3_jenslin will provide an update on the D3 app testing later today.