Run Sanity test (sensor_dt_compliance more specificly) system reboot

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Hi all, I have encountered a problem when I run the Sanity test. The test stuck at the sensor_dt_compliance test, and then the developer kit rebooted. I am very confused about why this is happing, and hope to get your help. PS. I am using a imx219sensor with only lens.
the last output on console listed below:

createNvCameraTools: NvCameraTools load success for Version7
getNvCameraTools: createCameraTools_70() success
Registering capture handler...
INFO: ############################################
INFO: Running the SensorDTCompliance test...
INFO: ############################################
INFO: RUN CMD: /bin/sh -c 'cat /proc/device-tree/tegra-camera-platform/modules/module0/drivernode0/proc-device-tree'
INFO: /proc/device-tree/cam_i2cmux/i2c@0/rbpcv2_imx219_a@10
INFO: RUN CMD: /bin/sh -c 'cat /proc/device-tree/cam_i2cmux/i2c@0/rbpcv2_imx219_a@10/compatible'
INFO: nvidia,imx219
INFO: RUN CMD: /bin/sh -c 'cat /proc/device-tree/cam_i2cmux/i2c@0/rbpcv2_imx219_a@10/name'
INFO: rbpcv2_imx219_a
INFO: RUN CMD: sudo /bin/sh -c '/usr/sbin/sensor-kernel-tests -f "Sensor DT Test" -c nvidia,imx219 -n rbpcv2_imx219_a'
packet_write_wait: Connection to 10.0.32.171 port 22: Broken pipe

summury_log has been enclosed.
summarylog.txt (36 KB)

3

What’s the BSP version?

4

The version of my bsp is Tegra186_Linux_R32.4.2_aarch64.

6

Does the sensor driver working normally?
Could you verify with r32.5.x

7

I have plugged in one sensor(dtsi has two configurations). and It works just fine, I can use it to capture both video and images. below are the log

ts@ts-desktop:~/Desktop$ dmesg | grep -i imx219
[ 3375.283998] imx219 9-0010: probing v4l2 sensor at addr 0x10
[ 3375.285333] imx219 9-0010: mclk name not present, assume sensor driven externally
[ 3375.285373] imx219 9-0010: avdd, iovdd and/or dvdd reglrs. not present, assume sensor powered independently
[ 3375.285628] imx219 9-0010: tegracam sensor driver:imx219_v2.0.6
[ 3375.285660] imx219 9-0010: imx219_power_on: power on
[ 3375.309689] imx219 9-0010: imx219_board_setup: error during i2c read probe (-121)
[ 3375.309821] imx219 9-0010: imx219_power_off: power off
[ 3375.309863] imx219 9-0010: board setup failed
[ 3375.310055] imx219: probe of 9-0010 failed with error -121
[ 3375.310216] imx219 10-0010: probing v4l2 sensor at addr 0x10
[ 3375.310353] imx219 10-0010: mclk name not present, assume sensor driven externally
[ 3375.310361] imx219 10-0010: avdd, iovdd and/or dvdd reglrs. not present, assume sensor powered independently
[ 3375.310579] imx219 10-0010: tegracam sensor driver:imx219_v2.0.6
[ 3375.310586] imx219 10-0010: imx219_power_on: power on
[ 3375.334484] imx219 10-0010: imx219_power_off: power off
[ 3375.334596] tegra194-vi5 15c10000.vi: subdev imx219 10-0010 bound
[ 3375.342271] imx219 10-0010: detected imx219 sensor
[ 3375.349657] imx219 10-0010: imx219_power_on: power on
[ 3375.360808] imx219 10-0010: imx219_open:
[ 3375.374527] imx219 10-0010: imx219_power_off: power off

I will try with the R32.5 version, i can only hope the new version will solve my problem.

Thank you for your kind help! Wish you have a nice day!

8

Suppose it’s fixed after 32.5.x
Or you can modify the imx219.c to call tegracam_device_unregister() if probe failed.

	err = imx219_board_setup(priv);
	if (err) {
		tegracam_device_unregister(tc_dev);
		dev_err(dev, "board setup failed\n");
		return err;
	}
1 Like
9

Ok I will try modify the imx219.c for now and move my work to 32.5x later.

10

Hi ShaneCCC:
I have implemented your second advice which is to modify the imx219.c. It did worked, it’s good to see the board not rebooting itself again, but the SensorDTCompliance test is still failed. I have already put the log info below, hope to see your opinions over this issues.

By the way I want to open the kernel’s log, I think it should be configed during the make ARCH=arm64 O=$TEGRA_KERNEL_OUT tegra_defconfig command to generate the Kconfig file for later use to compile the kernel. What should I set to make the compilied kernel output the kernel debug level logs so that i can catch and analyze it ?

Thank you for your kind help!
sensor_parse_dt.txt (10.4 KB)

11

You can add #define DEBUG in the top of imx219.c to enable the dev_dbg()

12

Yes, I had done that before. But now I want to track the log to see why did the test failed, however i don’t know which module the problem should be in. Thus I think maybe I can pass the KCFLAGS=-DDEBUG to the “make ARCH=arm64 O=$TEGRA_KERNEL_OUT -j8” to open all the debug files（not only drivers but also kernels.) The command should be like “make
KCFLAGS=-DDEBUG ARCH=arm64 O=$TEGRA_KERNEL_OUT -j8”.

By the way I have tried so, but the console get stucked and I have to reboot my host machine.

13

I don’t have experience for the debug kernel. However you can check those drivers in the …/kernel/nvidia/drivers/media/platform/tegra/camera/*

14

OK, Thanks a lot!

15

Hi ShaneCCC,
I have tried to falsh the R32.5.1 to the jetson nx developer kit, but when I type the “cat /proc/modules | grep -i imx219”, it shows nothing, however, the dmesg shows that the sensor driver has loaded successfuly. Why did that happened?

when I ran nvcstestmain.py it shows information down below:

createNvCameraTools: NvCameraTools load success for Version7
getNvCameraTools: createCameraTools_70() success
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
Config Error: sensor load cfg: 0, cfg check: 1
SCF: Error BadParameter: NvCameraIspInitialize failed (in src/services/autocontrol/NvCameraIspDriver.cpp, function initialize(), line 158)
SCF: Error BadParameter:  (propagating from src/services/autocontrol/NvCameraIspDriver.cpp, function fillStaticProps(), line 1968)
SCF: Error BadParameter:  (propagating from src/services/autocontrol/AutocontrolService.cpp, function fillStaticProps(), line 176)
SCF: Error BadParameter:  (propagating from src/api/Source.cpp, function initialize(), line 133)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function replaceSourceInList(), line 380)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 305)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSourceByGuid(), line 493)
(nvcameratools) Error BadParameter:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcameratools/scf/nvcameratools.cpp, function getNumSupportedSensorEntries(), line 4816)
---- imager: No override file found. ----
Config Error: sensor load cfg: 0, cfg check: 1
SCF: Error BadParameter: NvCameraIspInitialize failed (in src/services/autocontrol/NvCameraIspDriver.cpp, function initialize(), line 158)
SCF: Error BadParameter:  (propagating from src/services/autocontrol/NvCameraIspDriver.cpp, function fillStaticProps(), line 1968)
SCF: Error BadParameter:  (propagating from src/services/autocontrol/AutocontrolService.cpp, function fillStaticProps(), line 176)
SCF: Error BadParameter:  (propagating from src/api/Source.cpp, function initialize(), line 133)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function replaceSourceInList(), line 380)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 305)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSourceByGuid(), line 493)
(nvcameratools) Error BadParameter:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcameratools/scf/nvcameratools.cpp, function getNumSupportedSensorEntries(), line 4816)
(nvcameratools) Error BadParameter: sensor index 0 is larger than sensor property list (in /dvs/git/dirty/git-master_linux/camera/utils/nvcameratools/scf/nvcameratools.cpp, function getSensorProperty(), line 4883)
Traceback (most recent call last):
  File "nvcstestmain.py", line 1171, in <module>
    main()
  File "nvcstestmain.py", line 774, in main
    sensorProp.grepSensorInfo()
  File "nvcstestmain.py", line 84, in grepSensorInfo
    self.oGraph.getSensorProperty(i, camSsourceIndex)
  File "/usr/lib/python2.7/nvcamera/nvcameratools.py", line 3155, in getSensorProperty
    raise NvCameraException(err, "Error while calling into getSensorProperty")
nvcamera.nvcameratools.NvCameraException: ERROR: Error while calling into getSensorProperty
ErrorCode: 4: bad parameter to method or interface
16

IMX219 is build in kernel Image instead build as loadable module.
Do you run sudo python nvcstestsmain.py?

17

Yes I run with the sudo command, and I do understand it is a build in module. However when never I ran this nvcstestmain.py it shows the logs I mentioned above.

18

Could you change to another connector to try. Or try sudo python nvcstestmain.py -i 1

19

Hi ShaneCCC, Greetings from China!
After I implemented your advice on adding “tegracam_device_ungister(tc_dev)” in the driver, I analyze the log of Sanity test. Here is what I find:

I find that the reason why the board is rebooting itself is that the imx219’s dtsi file has two ports configured(9-0010 and 10-0010), however I only plugged in one sensor in the connector(10-0010). And when the sensor_dt_compliance test runs, the python script will generate a command which will call the excutable file in path “/usr/sbin/sensor-kernel-tests” with parameter “-f Sensor DT Test -c nvidia,imx219 -n rbpcv2_imx219_a”. This program will call the driver’s probe function when it reads the sensor’s register, however, it will probe the 9-0010(which has no sensor at all) firstly, thus the probe function failed at the board_set_up(), with before I add the unregister code, only a dev_err() message will sends.

Also the python scripts create a thread to run the “esnsor-kernel-tests” while in the mean time it uses a “While True” loop to constantly detect the status of the theread and sleeps “delay” seconds before it finished, however it didn’t expect the board_set_up() function will ever fail. thus the thread is running forever.

So there are two ways to solve my problem ,one is to pluggin two sensor, the other is to change the dtsi file to configure only one port(10-0010 or 9-0010).

It took me almost a week to locate and slove the problem ,hope you nvidia stuff can nicely provide these kind of reminders to the developers like me to save their time, thank you for answering my question all the time!

20

Will have jetson-io tools to configure the camera dtb configure in coming release.

1 Like