How to use ISP in Xavier when connected with IMX390

I try to run guvview, and it print error as follows:

nvidia@nvidia-desktop:~$ guvcview -d /dev/video0
nvbuf_utils: Could not get EGL display connection
nvbuf_utils: Could not get EGL display connection
Opening in O_NONBLOCKING MODE
GUVCVIEW: version 2.0.5
nvbuf_utils: Could not get EGL display connection
nvbuf_utils: Could not get EGL display connection
Opening in O_NONBLOCKING MODE
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (UVCIOC_CTRL_MAP) Error: No such file or directory
V4L2_CORE: (VIDIOC_G_EXT_CTRLS) failed
V4L2_CORE: using VIDIOC_G_EXT_CTRLS on single controls for class: 0x009a0000
V4L2_CORE: control id: 0x009a0903 failed to get (error -1)
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'cards.tegra-hda-galen.pcm.front.0:CARD=0'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5007:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_dmix.c:990:(snd_pcm_dmix_open) The dmix plugin supports only playback stream
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_dmix.c:1052:(snd_pcm_dmix_open) unable to open slave
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
V4L2_CORE: Could not grab image (select timeout): Resource temporarily unavailable
V4L2_CORE: Could not grab image (select timeout): Resource temporarily unavailable
GUVCVIEW: (status) saving video to /home/nvidia/my_video-1.mkv
ENCODER: add stream 0 to stream list
ENCODER: add stream 1 to stream list
ENCODER: (matroska) add seekhead entry 0 (max 10)
ENCODER: (matroska) add seekhead entry 1 (max 10)
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1771846
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1755621
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1755303
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1758142
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1756776
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1759644
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1330708
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1161788
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1161223
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1152723
ENCODER: Error encoding video frame: -1
[mpeg4 @ 0x7f5c000e50] Provided packet is too small, needs to be 1153572
ENCODER: Error encoding video frame: -1
AUDIO: (portaudio) Aborting audio stream
AUDIO: Closing audio stream...
ENCODER: (matroska) closing context
ENCODER: (matroska) closing cluster
ENCODER: (matroska)write seekhead
ENCODER: (matroska) end duration = 22647 (22647.000000)

While i can see the green screen which caps by camera, However, I play the record in VLC, it shows black always.

The dmesg show as follows:

[59037.522750] usb 2-4: new SuperSpeed USB device number 5 using tegra-xusb
[59037.542881] usb 2-4: LPM exit latency is zeroed, disabling LPM.
[59037.543294] usb 2-4: Int endpoint with wBytesPerInterval of 1024 in config 1 interface 0 altsetting 0 ep 130: setting to 64
[59037.544338] usb 2-4: New USB device found, idVendor=2a0b, idProduct=00c7
[59037.544393] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[59037.544400] usb 2-4: Product: LI-IMX390
[59037.544403] usb 2-4: Manufacturer: Leopard Imaging
[59037.544406] usb 2-4: SerialNumber: 0000000001
[59037.547073] uvcvideo: Found UVC 1.10 device LI-IMX390 (2a0b:00c7)
[59037.549736] uvcvideo 2-4:1.0: Entity type for entity Extension 3 was not initialized!
[59037.549931] uvcvideo 2-4:1.0: Entity type for entity Processing 2 was not initialized!
[59037.550063] uvcvideo 2-4:1.0: Entity type for entity Camera 1 was not initialized!
[59037.550406] input: LI-IMX390 as /devices/3610000.xhci/usb2/2-4/2-4:1.0/input/input11
[59040.421797] usb 2-4: usb_suspend_both: status 0
[59040.422244] usb usb2: usb_suspend_both: status 0
[59040.422279] tegra-xusb 3610000.xhci: entering ELPG
[59040.442302] tegra-xusb 3610000.xhci: entering ELPG done

Hi Chad_Ding,

Please follow below sequence to run the camera:

  1. Plug in the 12V power supply to USB 3.0 Tester.
  2. Connect the camera to Xavier (USB3.0 port) through USB 3.0 cable.
  3. Open guvcview.

Have you tested this camera on regular Windows PCs to make sure the camera works properly?

How to test the camera in windows PC, what software do i need to install?

I connect IMX390 USB into windows10 PC and test it using windows camera, and it show like this.

Hi Chad_Ding,

I cannot see the image. If possible, you can send the image to my Email (simonz@leopardimaging.com).

The picture is attach in #9,thanks!

Hi Chad_Ding,

Thanks for image. The image looks normal since this camera outputs raw data.
We will try the camera on Xavier here.

I wonder how to convert raw data into NV12 or RGB, so I can use this camera.

Hi Chad_Ding,

We can get vide output from IMX390-GMSL2 via VLC on Xavier. Maybe some libs missing for guvcview.

We have a Linux camera tool which can convert the raw to RGB. Please give it a try.

Hi,

is there any support for AR0231 Sensors which we have connected via GMSL2 and USB3.0 Deserializer? I can get a picture with the linux_camera_tool but with a lot of noise and weird colors. Any plan to support the RCCB Sensor?

Hi cschro,

The current Linux camera tool doesn’t support RCCB. Our team may give it a try to add the RCCB mode. Once there is a new version, we will update it to github.

Hi Simon,

so am I right that LI sells a product/sensor, which there is absolutely no support for?

→ CameraTool on Windows does not work, gives a greenish picture (not RAW data) at approx. 0.5 fps
→ guvcview on Linux gives same picture at approx 1-2 fps
→ linux camera tool also not working

It might be that we are doing something wrong but since there is no manual or instructions at all and support tends to only answer the easiest questions (did you plug in USB correctly?), we cannot tell what is wrong.

Just wondering…

Hi cschro,

Sorry for the trouble.
Actually, it is normal that you see the greenish image. The AR0231-GMSL2 camera outputs RAW data encapsulated in YUY2 format. Since UVC standard has no RAW image format, we use YUY2 as the wrapper. In regular video software, it doesn’t know this is RAW, and it interprets it as YUY2. That is why you see greenish image. Our camera tool can convert the raw to bmp image. We didn’t see any issues with the camera tool and AR0231-GMSL2 camera on Windows OS, and we can use our camera tool to capture raw images. If your camera has problems, please contact our support (support@leopardimaging.com) for further assistance.

Hello @SimonZhu I’m Harshwardhan Bhangale from India. I’m currently working on project which invovles capturing of raw format images from IMX390-dual-GMSL camera by leopard imaging, using xavier. So could you help me with resources which woukd guide me to install all the necessary drivers and capture images through that camera using only V4L2 and also with Libargus.

Hi hbhangale3,

Please contact support@leopardimaging.com to ask for the driver guide.

Thanks @SimonZhu Ive got the driver guide from the firm. They have provided steps for acquire video using LibArgus library. Is there anyway if one wants to use only the inbuilt V4L2 driver to access the image? so like just after installing the driver we could write some code and acquire image using V4L2 driver.

Hi hbhangale3,

Please try below v4l2 command.
v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10 --stream-to=imx390.raw -d /dev/video0

Hello SimonZhu,
after executing the above line im getting the following error could you please guide me about it:
failed to open /sys/dev/char/0:0/uevent
and really thank you for your instant and helpful replies.

hello SimonZhu,
Also I have a doubt , the file /dev/video0 , i opened it in vim editor its empty so it is always empty or we have to write entire code like calling buffers and storing values etc. in it?