No image capture from camera via FPD Link for Orin nano R36.3 module

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
23

hello akvadak66,

(1) did you always start v4l2 IOCTL first then testing gst pipeline with nvarguscamerasrc plugin?
could you please give it a try to reboot the system, and running gst pipeline directly?

(2) could you please also gather another logs from workable setup?
let’s compare the logs reported after IspHw frameComplete start.

Worker thread IspHw frameComplete start
SCF: Error NotSupported:  (propagating from src/services/capture/FusaCaptureIspHw.cpp, function openIspHw(), line 252)
SCF: Error NotSupported:  (propagating from src/services/capture/CaptureServiceDeviceIsp.cpp, function open(), line 188)
Worker thread IspHw frameComplete stop
SCF: Error NotSupported:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function initialize(), line 352)
SCF: Error InvalidState: Isp is not opened (in src/services/capture/CaptureServiceDeviceIsp.cpp, function close(), line 201)
SCF: Error NotSupported:  (propagating from src/services/capture/CaptureService.cpp, function startService(), line 644)
SCF: Error InvalidState: Queue mutex not initialized (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamerautils/inc/QueueImpl.h, function dequeue(), line 228)
SCF: Error NotSupported:  (propagating from src/components/ServiceHost.cpp, function startServices(), line 123)
Stopping services...

(3) BTW, set_mode_delay_ms should be added outside mode info.
please also try increasing the timeout values for verification.
for instance,


			imx477_a@10 {
				status = "okay";
				compatible = "ridgerun,imx477";
				reg = <0x10>;
				physical_w = "3.680";
				physical_h = "2.760";
				set_mode_delay_ms = "3000";
				sensor_model = "imx477";
24

Hello JerryChang.

We do not run v4l2 IOCTL before running the gst pipeline
We run gst pipeline directly, after system booting

We will add more later

We increased the time according to your advice.
Alas no result

25

hello akvadak66,

since Orin NX/Nano they’re sharing the same software stack, looking forward the logs from workable setup (to confirm there’re NotSupported errors as well)

26

Hello JerryChang.
I got the log of a working installation.
But when I run your commands below

sudo su 
# pkill nvargus-daemon
# export enableCamScfLogs=5
# export enableCamPclLogs=5
# nvargus-daemon 2>&1 | tee argus-output.log

Then start capturing the image, then the window with the image closes.
Is this how debug output is supposed to work?

If you just run the gst pipeline, there is an image capture.
Please check the debug output.
Thanks.
argus-output-nx.log (20.8 KB)

27

hello akvadak66,

it looks camera stream did not enabled.

(Argus) Error NotSupported: Failed to initialize EGLDisplay (in src/eglutils/EGLUtils.cpp, function getDefaultDisplay(), line 77)
(Argus) Error NotSupported: Failed to get default display (in src/api/EGLOutputStreamImpl.cpp, function initialize(), line 99)
(Argus) Error NotSupported:  (propagating from src/api/CaptureSessionImpl.cpp, function createEGLOutputStream(), line 987)
(Argus) Error InvalidState: Unknown stream deleted. (in src/api/CaptureSessionImpl.cpp, function outputStreamDeleted(), line 1104)
(Argus) Error NotSupported:  (propagating from src/api/CaptureSessionImpl.cpp, function createOutputStreamInternal(), line 849)

note,
you must configure those settings on the target directly to gather debug logs.

28

Hello JerryChang.
Thank you for your reply.

Strange, but the image is there in this case.

Could you please tell me how to adjust these settings?

Thanks

29

Hi akvadak66

We stumbled upon a similar issue with the Orin NX and the Orin Nano. Do you have the following line there, when flashing the device:

OVERLAY_DTB_FILE+=“,tegra234-p3768-0000+p3767-0000-dynamic.dtbo”;

Without this overlay, we also had a camera that did not work with the Orin Nano but worked with the Orin NX. Maybe someone from NVidia can elaborate which setting inside this overlay could have such an effect?

Kind regards

30

Hello, sevm89.

Yes, that’s a problem we’ve encountered too.
All due to the fact that in tegra234-p3768-0000+p3767-0000-dynamic.dtbo imx219 camera is prescribed.
It interferes with everything else.
We solved it the following way.

1.	In the sources for Orin NX find the file
hardware\nvidia\t23x\nv-public\overlay\tegra234-p3768-0000+p3767-0000-dynamic.dts
You need to comment out the lines 
//#include “tegra234-p3768-camera-rbpcv2-imx219.dtsi” and
//#include “tegra234-p3767-sku-handling.dtsi”.
2.	Build the device tree make dtbs 
How to do it is in the link below
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Kernel/KernelCustomization.html
3. find the compiled tegra234-p3768-0000+p3767-0000-dynamic.dtbo.
I have it here
nvidia-oot/device-tree/platform/gtntric-dts/dtbs/ tegra234-p3768-0000+p3767-0000-dynamic.dtbo
4.	Replace this file in the Linux distribution 
Linux_for_Tegra/kernel/dtb
6.	Re-flash the module

But this does not help in case of Orin Nano and camera connected to FPD Link

31

Hi JerryChang,

“note, you must configure those settings on the target directly to gather debug logs.”

*** Could you please provide moer details how to configure those settings?

Thank you very much!

32

hello akvadak66,

do you have physical access to your platform? you should setup those commands for nvargus-daemon to executed locally for gathering the logs.

you should not disable this, there’re specific settings for Orin Nano SKUs.

anyways, per r36.4 release note of [4.2. Device Registration] section.
it’s suggest to create a device tree overlay (DTB overlay or .dtbo) file to register the camera module.

34

Hi JerryChang,

Following your advice, we have created an overlay file to register the camera module.
No result.
Orin nano behavior is the same as in case of static device tree.
It is not clear why Orin nano does not work.
Orin Nx works fine on the same carrier board.

In this case include “tegra234-p3767-sku-handling.dtsi”. is included in the overlay file

When running the script we get the same errors

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)1920,height=(int)1080,framerate=60/1' ! nvvidconv ! xvimagesink sync=false --verbose

libEGL warning: DRI2: failed to authenticate
Setting pipeline to PAUSED ...
ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Could not initialise Xv output
Additional debug info:
../sys/xvimage/xvimagesink.c(1806): gst_xv_image_sink_open (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
No Xv Port available
ERROR: pipeline doesn't want to preroll.
Failed to set pipeline to PAUSED.
Setting pipeline to NULL ...
Freeing pipeline ...
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error EndOfFile: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 350)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 366)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
(Argus) Error EndOfFile: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error EndOfFile:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 92)

Another question is that when using the overlay file we have to completely re-flash the module.

Is there any way to replace only the new .dtbo overlay file?

35

hello akvadak66,

I’m curious why this always reported in your failure use-case.

no need to re-flash the target.
you can simply copy (via ssh) the dtbo file to the target, putting the overlay file under /boot/, Jetson-IO should recognize your overlay file.

36

Hello, JerryChang

Jetson-IO unfortunately does not work for me.
When running sudo /opt/nvidia/jetson-io/jetson-io.py a gray screen appears for a second and then goes out.
I read this is due to the fact that we are using a proprietary carrier board.

Please tell me, when you release a linux release, do you do a health check on physically connected devices?
Could you please check if orin nano and imx477 works?

37

hello akvadak66,

that’s due to Jetson-IO only works on developer kit.

yes, we did verify that.
here’re logs for your reference.

$ sudo dmesg | grep imx477
[    9.681338] imx477 9-001a: tegracam sensor driver:imx477_v2.0.6
[    9.981959] imx477 9-001a: imx477_board_setup: error during i2c read probe (-121)
[    9.981988] imx477 9-001a: board setup failed
[    9.982081] imx477: probe of 9-001a failed with error -121
[    9.985741] imx477 10-001a: tegracam sensor driver:imx477_v2.0.6
[   10.286702] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx477 10-001a bound

BTW,
please note that, there’s compatible property of camera device tree overlay.
it has defined as JETSON_COMPATIBLE_P3768 by default, which include the support board configs.
for instance,

#define JETSON_COMPATIBLE_P3768		"nvidia,p3768-0000+p3767-0000", \
                                        "nvidia,p3768-0000+p3767-0001", \
                                        "nvidia,p3768-0000+p3767-0003", \
                                        "nvidia,p3768-0000+p3767-0004", \
                                        "nvidia,p3768-0000+p3767-0005"
38

Hello, JerryChang

We have checked everything, alas nothing works with the Orin Nano.
We have ordered two more Orin Nano modules, will be available within two weeks.
Please do not close this forum thread.
I will post the results as soon as they are available.

39

hello akvadak66,

at least Raspberry Pi Camera, IMX219, and Raspberry Pi Camera, IMX477 should works on Orin Nano DevKit by default. otherwise, I doubt it’s hardware damage.
let’s have cross-validation, if you confirm that your Jetson product is defective, please contact the NVIDIA Customer Care team for the RMA process.

40

Hey, JerryChang.
We’ve been having some logistical problems.
But in two weeks we will have Orin Nano and we can continue our research.

Thanks for your support.