Problem capturing video with IMX377 and argus

Autonomous Machines Jetson & Embedded Systems Jetson TX1
#1

We are using L4T R28.1 and imx377 camera sensor from leopard imaging (https://leopardimaging.com/product/li-jetson-kit-imx377cs-x/). While everything seems to work ok as far as camera controls and frame capture are concerned, we are having serious issues when trying to capture video. We have installed the camera driver provided by leopard imaging and we are using argus_camera app to control camera and capture frames and video. When trying to capture video the program halts after stopping the recording. That results a video file with zero byte size. We have tested both applications with original argus library provided by nvidia and the argus library provided by leopard imaging.

When using the gstvideoencode sample we also have problems acquiring video, again the program halts and below you can see the console output

Executing Argus Sample: argus_gstvideoencode
Argus Version: 0.96.2 (multi-process)

NvEglStreamSrcInitializeEgl: Load library: libEGL.so
PREVIEW CONSUMER: Creating OpenGL context.
PREVIEW CONSUMER: Connecting to EGLStream(s).
PREVIEW CONSUMER: Connected to stream(s).
PREVIEW CONSUMER: Waiting until producer(s) connect...
PREVIEW CONSUMER: Producer(s) connected; continuing.
PREVIEW CONSUMER: Acquired frame 1. Rendering.
PREVIEW CONSUMER: Acquired frame 2. Rendering.
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 132)
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 101)

We appreciate any help you can provide as we are close to a deadline for our prototype product.

#2

Hi Katerina,

The current argus software should not support full size (4104 x 3046) video record. You can select the output size as 3840x2160 or lower resolution (on argus GUI) to record the video.
Or you can also use below command to record the 4104x3046 video.

gst-launch-1.0 -e nvcamerasrc sensor-id=0 fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)4104, height=(int)3046, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! omxh264enc bitrate=80000000 ! ‘video/x-h264, stream-format=(string)byte-stream’ ! filesink location=IMX377_0.avi -e

#3

Hi SimonZhu,

I tried to change the output size form argus_camera gui but the drop-down has only one option 4104x304. Also, i printed the device sensor modes and only one exists.

Available Sensor modes:
mode 0: Resolution = 4104 x 3046, FPS=30

So, how i will be able to change the resolution from camera?

#4

Hi Katerina,

Which driver you are using? Is it for TX1 or TX2? The driver should only support 4104x3046, but the argus or gstreamer should be able to scale the resolution to 3840x2160 or lower. Since you have the driver source code, you can also add the lower resolutions in drivers.

#5

Hi,

Any update to the above issue. I am facing the same issue, except that its in TX2 (28.1).

My resolution is 1936*1102@30fps.

gstreamer with nvcamerasrc is working fine for me, but arguslib streaming is failing with the exact error message (acquiring 2 frames is ok, but then hangs).

Below is the command used for nvcamersc (working well):

gst-launch-1.0 nvcamerasrc sensor-id=0 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1936, height=(int)1102, format=(string)I420, framerate=(fraction)30/1' ! autovideosink

Below is the output for yuvjpeg sample:

Executing Argus Sample: argus_yuvjpeg
Argus Version: 0.96.2 (multi-process)
PRODUCER: Creating output stream
PRODUCER: Launching consumer thread
CONSUMER: Waiting until producer is connected...
PRODUCER: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Acquired Frame: 1, time 2866160395
CONSUMER: 	Sensor Timestamp: 2866120690000, LUX: 2213.322754
CONSUMER: 	IImage(2D): buffer 0 (640x480, 640 stride), f0 dd ef f4 ec f0 ef e3 f4 e5 f0 f0
CONSUMER: 	IImage(2D): buffer 1 (320x240, 640 stride), 7b 84 79 89 76 87 74 86 7c 88 77 88
CONSUMER: 	Wrote JPEG: 1.JPG
CONSUMER: Acquired Frame: 2, time 2866212057
CONSUMER: 	Sensor Timestamp: 2866179953000, LUX: 2213.939697
CONSUMER: 	IImage(2D): buffer 0 (640x480, 640 stride), ef f1 e4 ee ef eb f0 f1 f2 f1 ed f0
CONSUMER: 	IImage(2D): buffer 1 (320x240, 640 stride), 7a 85 79 87 7a 86 75 8b 7b 87 7b 81
CONSUMER: 	Wrote JPEG: 2.JPG
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 132)
(Argus) Error Timeout:  (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 101)

Below is the log from argus daemon:

root@tegra-ubuntu:/home/nvidia# sudo /usr/sbin/argus_daemon
=== Listening for connections... ===
=== Connection 7F91D0F1E0 established ===
NvPclHwGetModuleList: No module data found
PCLHW_DTParser
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/imx_rear_P5V27C.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/imx_rear_P5V27C.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/imx_rear_P5V27C.isp] 8/16---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
CameraProvider result: provider=0x7f8cbc8840, shim=0x7f8cbcb0b0, status=0, rpc status=1, size=9
LSC: LSC surface is not based on full res!
Growing thread pool to 1 threads
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc
launchCC abort cc 104 session 0
SCF: Error Timeout:  (propagating from src/api/Session.cpp, function capture(), line 830)
(Argus) Error Timeout: Failed to submit first capture request (propagating from src/api/CaptureSessionImpl.cpp, function submitCaptureRequests(), line 311)
(Argus) Error Timeout:  (propagating from src/api/CaptureSessionImpl.cpp, function threadFunction(), line 777)

Below is the dmesg output:

[ 6225.075477] nvmap_alloc_handle: PID 6387: argus_yuvjpeg: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Please pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant. 
[ 6283.879386] argus_yuvjpeg[6387]: unhandled level 2 translation fault (11) at 0x00000004, esr 0x92000006
[ 6283.888967] pgd = ffffffc1aa2fb000
[ 6283.892876] [00000004] *pgd=000000022a2f9003, *pud=000000022a2f9003, *pmd=0000000000000000

[ 6283.903493] CPU: 3 PID: 6387 Comm: argus_yuvjpeg Not tainted 4.4.38-tegra #151
[ 6283.910785] Hardware name: quill (DT)
[ 6283.914470] task: ffffffc0593b3e80 ti: ffffffc1914a0000 task.ti: ffffffc1914a0000
[ 6283.922012] PC is at 0x7f7bc8c3f4
[ 6283.925339] LR is at 0x7f7bc8c3e4
[ 6283.928665] pc : [<0000007f7bc8c3f4>] lr : [<0000007f7bc8c3e4>] pstate: 60000000
[ 6283.936060] sp : 0000007fd930ce50
[ 6283.939416] x29: 0000007fd930ced0 x28: 0000000000000000 
[ 6283.944758] x27: 0000000000000000 x26: 0000000000000000 
[ 6283.950094] x25: 0000000000000000 x24: 0000000000000000 
[ 6283.955488] x23: 0000000000000000 x22: 0000000000000000 
[ 6283.960829] x21: 0000000000000000 x20: 0000000000452f70 
[ 6283.966166] x19: 0000007fd930cea8 x18: 000000000000000f 
[ 6283.971551] x17: 0000007f7b929d30 x16: 0000007f7b7ef4a0 
[ 6283.976893] x15: 0000007f7bd6ccc0 x14: 2f746e65696c632f 
[ 6283.982230] x13: 74656b636f732f63 x12: 70722f637273206d 
[ 6283.987566] x11: 6f726620676e6974 x10: 616761706f727028 
[ 6283.992953] x9 : 20203a74756f656d x8 : 0000000000000040 
[ 6283.998289] x7 : 0000000000000000 x6 : 0000007fd9309fcc 
[ 6284.003629] x5 : 0000000000000000 x4 : 0000000000000000 
[ 6284.009017] x3 : 0000000000000000 x2 : 0000000000000001 
[ 6284.014551] x1 : 0000000000000000 x0 : 0000007fd930cea8 

[ 6284.021464] Library at 0x7f7bc8c3f4: 0x7f7bc33000 /usr/lib/aarch64-linux-gnu/tegra/libargus_socketclient.so
[ 6284.031210] Library at 0x7f7bc8c3e4: 0x7f7bc33000 /usr/lib/aarch64-linux-gnu/tegra/libargus_socketclient.so
[ 6284.040995] vdso base = 0x7f7bd6b000

Regards,
Rejeesh

#6

Hi Rejeesh_QueST,

Which camera and TX2 driver are you using? Our current IMX377 driver should only support 4104x3046@30fps.

#7

Hi SimonZhu,

I am not using a camera sensor from leopard imaging. I post my issue as a comment to this post because I am facing exactly similar behavior (though my sensor driver is different).

BTW, were you able to get IMX377 work using arguslib? If so, possibly, that fix will help me as well.

Regards,
Rejeesh

#8

Hi Rejeesh_QueST,

Yes, we can get video output with either argus (argus_camera --device=0) or gstreamer command.

#9

Hi SimonZhu,

I tested using IMX390 from Leopard imaging (https://leopardimaging.com/product/li-imx390-gmsl2/), but I am facing the same issue. I tried argus_camera application as well (gstreamer is working fine).

Please note that I did not get any driver/dtb or source code from Leopard. I am using my own driver and dtb. Could you share your working dtb & driver?

Also, I am interested to know how you fixed the issue mentioned in the original post.

Regards,
Rejeesh

#10

Hi Rejeesh_QueST,

We have below IMX390-GMSL2 driver for Nvidia TX2 developer kit. It cannot get video output with argus software either. Our team are working on it.

https://www.dropbox.com/s/ce084kka127iqlq/IMX390-GMSL2_R28.1_TX2_NV_20180323.txt?dl=0

#11

Hi SimonZhu,

About post #4, we are using the driver provided by leopard imaging. Can you provide us with the register map (code) for lower resolutions?

#12

Hi Katerina,

For IMX377 camera, we have only the register setting for 4104x3046 right now. You can check the IMX377 sensor data sheet to use cropping or binning mode to get lower resolutions.