nvcamera-daemon crashing on custom carrier board

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I am using L4T 28.2.1. I have an IMX290 camera module from e-consystems. On the dev kit, works fine. On the custom carrier board, it seems to work in that the camera is found, /dev/video node is created, all seems well. Except when I go to use the camera I am getting this strange error:

[  358.080136] nvcamera-daemon[2315]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006
[  358.090105] pgd = ffffffc071fc9000
[  358.093566] [00000000] *pgd=00000000e3a7c003, *pud=00000000e3a7c003, *pmd=0000000000000000
[  358.101969] 
[  358.103468] CPU: 0 PID: 2315 Comm: nvcamera-daemon Not tainted 4.4.38 #3
[  358.110228] Hardware name: quill (DT)
[  358.113926] task: ffffffc1e0aa8c80 ti: ffffffc1b2480000 task.ti: ffffffc1b2480000
[  358.121454] PC is at 0x403038
[  358.124441] LR is at 0x403034
[  358.127406] pc : [<0000000000403038>] lr : [<0000000000403034>] pstate: 60000000
[  358.134845] sp : 0000007fafd50290
[  358.138193] x29: 0000007fafd549d0 x28: 0000000000000000 
[  358.143582] x27: 0000000000000001 x26: 0000007fafd54300 
[  358.148968] x25: 0000000000404000 x24: 0000000000000334 
[  358.154312] x23: 0000007fafd52300 x22: 0000007fafd51300 
[  358.159675] x21: 0000007fafd52244 x20: 0000007fafd50310 
[  358.165033] x19: 0000007fafd52930 x18: 0000000000000014 
[  358.170367] x17: 0000007fb158bfb0 x16: 0000007fb12de540 
[  358.175698] x15: 0000007fb1f98000 x14: 7265766972446172 
[  358.181042] x13: 656d61432f697061 x12: 2f637273206d6f72 
[  358.186381] x11: 6620676e69746167 x10: 61706f7270282020 
[  358.191713] x9 : 3a726574656d6172 x8 : 0000000000000040 
[  358.197052] x7 : 0000007fa86b3620 x6 : 0000000000000001 
[  358.202397] x5 : 0000000000000000 x4 : 0000007fa8000b10 
[  358.207741] x3 : 0000000000000000 x2 : 0000000000000001 
[  358.213092] x1 : 0000000000000000 x0 : 0000000000000000 
[  358.218681] 
[  358.220272] Library at 0x403038: 0x400000 /usr/sbin/nvcamera-daemon
[  358.227040] Library at 0x403034: 0x400000 /usr/sbin/nvcamera-daemon
[  358.233489] vdso base = 0x7fb1f97000
Socket read error. Camera Daemon stopped functioning.....
gst_nvcamera_open() failed ret=0

(optical_flow:2313): GStreamer-CRITICAL **: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)120/1 ! nvvidconv ! video/x-raw, width=(int)320, height=(int)180, format=(string)GRAY8 ! videoconvert ! appsink: Unable to query number of channels
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(3.4.5-dev) /home/nvidia/opencv/modules/cudawarping/src/cuda/pyr_down.cu:184: error: (-217:Gpu API call) invalid configuration argument in function 'pyrDown_caller'

Aborted (core dumped)

If I log the output from nvcamera-daemon before it crashes, I get this:

vPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: virtual device driver node not found in proc device-tree
OFParserGetVirtualDevice: virtual device driver node not found in proc device-tree
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
(NvOdmDevice) Error ModuleNotPresent: V4L2Device not available (in dvs/git/dirty/git-master_linux/camera-partner/imager/src/V4L2Device.cpp, function findDevice(), line 245)
(NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/V4L2Device.cpp, function initialize(), line 55)
(NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 103)
NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
NvPclStartPlatformDrivers: Failed to start module drivers
NvPclStateControllerOpen: Failed ImagerGUID 0. (error 0xA000E)
NvPclOpen: PCL Open Failed. Error: 0xf
SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 598)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 781)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 276)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 439)

Strange that it seems to not find the camera, since /dev/video does exist.

Has anyone seen this error before? Any idea what I can do next?

2

I have found multiple ways to make nvcamera-daemon segfault. Can you share the relevant device tree settings?

Did you update the “proc-device-tree” property in the tegra-camera-platform modules device tree node?

3

Hi,
From the log, it seems that there is a mismatch in the device tree parameters.
You need to modify the device tree as per your carrier board design. The nvcamera-daemon log mentions:

CF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 598)

This kind of problem will occur, whenever there is a mismatch in the device name or module path as mentioned in the “tegra-camera-platform” of the device tree.

  1. Can you kindly confirm whether the device name correlates to the i2c bus to which the module is connected(for e.g.: device name nomenclature for camera connected to i2c bus 1 = “eimx290 1-0042”).
    Device name = “ -00”
    Also the corresponding path should be mentioned in the “proc-device-tree”.
4

We actually did have a mistake in our device tree, you are correct. However, this did not resolve the “nvcamera-daemon[2315]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x9200000” issue. Now nvcamera just outputs nothing, but still has the same crash. We also see it in argus-daemon:

[  300.898580] argus_daemon[2189]: unhandled level 2 translation fault (11) at 0x00000008, esr 0x92000006
[  300.907909] pgd = ffffffc1abc64000
[  300.911356] [00000008] *pgd=000000022db1c003, *pud=000000022db1c003, *pmd=0000000000000000
[  300.919690] 
[  300.921196] CPU: 0 PID: 2189 Comm: argus_daemon Not tainted 4.4.38 #3
[  300.927640] Hardware name: quill (DT)
[  300.931308] task: ffffffc071652580 ti: ffffffc1e15f8000 task.ti: ffffffc1e15f8000
[  300.938791] PC is at 0x7f9e347c34
[  300.942112] LR is at 0x7f9e347c30
[  300.945434] pc : [<0000007f9e347c34>] lr : [<0000007f9e347c30>] pstate: 60000000
[  300.952828] sp : 0000007f9bf567b0
[  300.956137] x29: 0000007f9bf569d0 x28: 0000007f9bf571e0 
[  300.961477] x27: 0000007f9e35af48 x26: 000000000043f090 
[  300.966810] x25: 0000007f9bf56938 x24: 0000000000000068 
[  300.972143] x23: 0000000000000000 x22: 0000000000000068 
[  300.977482] x21: 0000007f946cc4c8 x20: 0000007f946cc4c0 
[  300.982815] x19: 0000007f946cc4c0 x18: 0000007f9d60e610 
[  300.988145] x17: 0000007f9e19f290 x16: 0000007f9e393fa8 
[  300.993485] x15: 0000007f9e3f6000 x14: ffffffffffffffff 
[  300.998815] x13: 0000000000000028 x12: 0101010101010101 
[  301.004145] x11: 7f7f7f7f7f7f7f7f x10: 65736d68716f6575 
[  301.009496] x9 : 7f7f7f7f7f7f7f7f x8 : 8080800000000000 
[  301.014832] x7 : 0000000000808080 x6 : 0000000000437c05 
[  301.020164] x5 : 0000000000000000 x4 : 0000000000000000 
[  301.025504] x3 : 0000007f9bf578d0 x2 : 0000007f9bf571e0 
[  301.030840] x1 : 0000000000000000 x0 : 0000000000000000 
[  301.036171] 
[  301.037679] Library at 0x7f9e347c34: 0x7f9e29b000 /usr/lib/aarch64-linux-gnu/tegra/libargus_socketserver.so
[  301.047417] Library at 0x7f9e347c30: 0x7f9e29b000 /usr/lib/aarch64-linux-gnu/tegra/libargus_socketserver.so
[  301.057151] vdso base = 0x7f9e3f5000

What are some potential avenues to explore? It is strange but we don’t get any actual backtrace, just a register dump.

5

Hi,
Can you share your device tree source and the complete dmesg log?
Kindly follow the below link Jetson TX2 Camera BringUp - eLinux.org
Also, Can you share the daemon log and VI trace log? To enable the daemon log, kindly follow the steps “To enable trace for more information” in the link and then launch the argus application. That information will help in finding the issue.