Problem creating an CSI-cam inference pipeline( Arducam IMX477 and Jetson NX)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi guys,

I have an issue, I was implementing Deepstream app using Arducam IMX477, however when i run any deepstream app, no camera launch appears. Could you pls suggest what can be reason for that?

If am not mistaken, I need to modify create_camera_source_bin file, but i dont know what to modify there.

See below logs from command prompt.

**PERF: FPS 0 (Avg)
**PERF: 0.00 (0.00)
** INFO: <bus_callback:181>: Pipeline ready

** INFO: <bus_callback:167>: Pipeline running

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4032 x 3040 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 4032 H = 3040
seconds to Run = 0
Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
**PERF: 30.72 (30.28)
**PERF: 29.99 (30.08)
**PERF: 30.01 (30.05)
**PERF: 29.99 (30.03)
**PERF: 30.01 (30.03)
q
Quitting
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Succe

3

The log seems fine. Why do you say “no camera launch appears.”?

Can you provide your platform information?

**• Hardware Platform (Jetson / GPU)**
**• DeepStream Version**
**• JetPack Version (valid for Jetson only)**
**• TensorRT Version**
**• NVIDIA GPU Driver Version (valid for GPU only)**
4

I agree, but camera launch does not appearing, however when I run below command, it comes up.

gst-launch-1.0 nvarguscamerasrc ! nvegltransform ! nveglglessink

I am using Jetpack 4.5.1 on the Xavier NX.

5

Do you mean there is nothing displayed on screen? Are you using deepstream-app sample app?

6

Yes, nothing pops up on the screen, Yes i am using deepstream-app sample app.

7

Can you upload the deepstream-app config file?

8 
deepstream-app -c /opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/source1_csi_dec_infer_resnet_int8.txt

The content:

################################################################################

Copyright (c) 2018-2020, NVIDIA CORPORATION. All rights reserved.

Permission is hereby granted, free of charge, to any person obtaining a

copy of this software and associated documentation files (the “Software”),

to deal in the Software without restriction, including without limitation

the rights to use, copy, modify, merge, publish, distribute, sublicense,

and/or sell copies of the Software, and to permit persons to whom the

Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in

all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR

IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL

THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER

LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING

FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER

DEALINGS IN THE SOFTWARE.

################################################################################

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP 5=CSI
type=5
camera-width=4032
camera-height=3040
camera-fps-n=30
camera-fps-d=1

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=5
sync=0
display-id=0
offset-x=0
offset-y=0
width=0
height=0
overlay-id=1
source-id=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265 3=mpeg4
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming 5=Overlay
type=4
#1=h264 2=h265
codec=1
encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0

set below properties in case of RTSPStreaming

rtsp-port=8554
udp-port=5400

[osd]
enable=1
border-width=2
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0

[streammux]
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000

Set muxer output width and height

width=1280
height=720

If set to TRUE, system timestamp will be attached as ntp timestamp

If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached

attach-sys-ts-as-ntp=1

config-file property is mandatory for any gie section.

Other properties are optional and if set will override the properties set in

the infer config file.

[primary-gie]
enable=1
model-engine-file=…/…/models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine
#Required to display the PGIE labels, should be added even when using config-file
#property
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
#Required by the app for SGIE, when used along with config-file property
gie-unique-id=1
config-file=config_infer_primary.txt

[tests]
file-loop=0

9

see below another logs

<pre><font color="#8AE234"><b>jetson@jetson-desktop</b></font>:<font color="#729FCF"><b>/opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app</b></font>$ deepstream-app -c /opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/source6_csi_dec_infer_resnet_int8.txt 

(deepstream-app:10370): GLib-GObject-<font color="#FCE94F"><b>WARNING</b></font> **: <font color="#3465A4">19:54:13.634</font>: g_object_set_is_valid_property: object class &apos;GstNvArgusCameraSrc&apos; has no property named &apos;maxperf&apos;

(deepstream-app:10370): GLib-GObject-<font color="#FCE94F"><b>WARNING</b></font> **: <font color="#3465A4">19:54:13.637</font>: g_object_set_is_valid_property: object class &apos;GstNvArgusCameraSrc&apos; has no property named &apos;maxperf&apos;

(deepstream-app:10370): GLib-GObject-<font color="#FCE94F"><b>WARNING</b></font> **: <font color="#3465A4">19:54:13.638</font>: g_object_set_is_valid_property: object class &apos;GstNvArgusCameraSrc&apos; has no property named &apos;maxperf&apos;

(deepstream-app:10370): GLib-GObject-<font color="#FCE94F"><b>WARNING</b></font> **: <font color="#3465A4">19:54:13.639</font>: g_object_set_is_valid_property: object class &apos;GstNvArgusCameraSrc&apos; has no property named &apos;maxperf&apos;

(deepstream-app:10370): GLib-GObject-<font color="#FCE94F"><b>WARNING</b></font> **: <font color="#3465A4">19:54:13.640</font>: g_object_set_is_valid_property: object class &apos;GstNvArgusCameraSrc&apos; has no property named &apos;maxperf&apos;

(deepstream-app:10370): GLib-GObject-<font color="#FCE94F"><b>WARNING</b></font> **: <font color="#3465A4">19:54:13.641</font>: g_object_set_is_valid_property: object class &apos;GstNvArgusCameraSrc&apos; has no property named &apos;maxperf&apos;
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine open error
0:00:01.998888228 10370      0xc8910f0 <font color="#FCE94F"><b>WARN   </b></font>              nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:&lt;primary_gie&gt; NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() &lt;nvdsinfer_context_impl.cpp:1691&gt; [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed
0:00:01.999001927 10370      0xc8910f0 <font color="#FCE94F"><b>WARN   </b></font>              nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:&lt;primary_gie&gt; NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() &lt;nvdsinfer_context_impl.cpp:1798&gt; [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed, try rebuild
0:00:01.999042920 10370      0xc8910f0 <font color="#06989A">INFO   </font>              nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:&lt;primary_gie&gt; NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() &lt;nvdsinfer_context_impl.cpp:1716&gt; [UID = 1]: Trying to create engine from model files
INFO: [TRT]: Reading Calibration Cache for calibrator: EntropyCalibration2
INFO: [TRT]: Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
INFO: [TRT]: To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
INFO: [TRT]: 
INFO: [TRT]: --------------- Layers running on DLA: 
INFO: [TRT]: 
INFO: [TRT]: --------------- Layers running on GPU: 
INFO: [TRT]: conv1 + activation_1/Relu, block_1a_conv_1 + activation_2/Relu, block_1a_conv_2, block_1a_conv_shortcut + add_1 + activation_3/Relu, block_2a_conv_1 + activation_4/Relu, block_2a_conv_2, block_2a_conv_shortcut + add_2 + activation_5/Relu, block_3a_conv_1 + activation_6/Relu, block_3a_conv_2, block_3a_conv_shortcut + add_3 + activation_7/Relu, block_4a_conv_1 + activation_8/Relu, block_4a_conv_2, block_4a_conv_shortcut + add_4 + activation_9/Relu, conv2d_cov, conv2d_cov/Sigmoid, conv2d_bbox, 
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
ERROR: Serialize engine failed because of file path: /opt/nvidia/deepstream/deepstream-5.1/samples/models/Primary_Detector/resnet10.caffemodel_b6_gpu0_int8.engine opened error
0:00:21.098748240 10370      0xc8910f0 <font color="#FCE94F"><b>WARN   </b></font>              nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:&lt;primary_gie&gt; NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::buildModel() &lt;nvdsinfer_context_impl.cpp:1744&gt; [UID = 1]: failed to serialize cude engine to file: /opt/nvidia/deepstream/deepstream-5.1/samples/models/Primary_Detector/resnet10.caffemodel_b6_gpu0_int8.engine
INFO: [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x368x640       
1   OUTPUT kFLOAT conv2d_bbox     16x23x40        
2   OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40         

0:00:21.118877973 10370      0xc8910f0 <font color="#06989A">INFO   </font>              nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:&lt;primary_gie&gt; [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.1/samples/configs/deepstream-app/config_infer_primary.txt sucessfully

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF:  FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	FPS 3 (Avg)	FPS 4 (Avg)	FPS 5 (Avg)	
**PERF:  0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
** INFO: &lt;bus_callback:181&gt;: Pipeline ready

** INFO: &lt;bus_callback:167&gt;: Pipeline running

Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:649 Invalid camera device specified 4 specified, 0 max index
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:649 Invalid camera device specified 2 specified, 0 max index
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:649 Invalid camera device specified 3 specified, 0 max index
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:649 Invalid camera device specified 5 specified, 0 max index

(deepstream-app:10370): GStreamer-<font color="#AD7FA8"><b>CRITICAL</b></font> **: <font color="#3465A4">19:54:36.554</font>: gst_mini_object_set_qdata: assertion &apos;object != NULL&apos; failed

(deepstream-app:10370): GStreamer-<font color="#AD7FA8"><b>CRITICAL</b></font> **: <font color="#3465A4">19:54:36.554</font>: gst_mini_object_set_qdata: assertion &apos;object != NULL&apos; failed
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:649 Invalid camera device specified 1 specified, 0 max index

(deepstream-app:10370): GStreamer-<font color="#AD7FA8"><b>CRITICAL</b></font> **: <font color="#3465A4">19:54:36.555</font>: gst_mini_object_set_qdata: assertion &apos;object != NULL&apos; failed

(deepstream-app:10370): GStreamer-<font color="#AD7FA8"><b>CRITICAL</b></font> **: <font color="#3465A4">19:54:36.556</font>: gst_mini_object_set_qdata: assertion &apos;object != NULL&apos; failed

(deepstream-app:10370): GStreamer-<font color="#AD7FA8"><b>CRITICAL</b></font> **: <font color="#3465A4">19:54:36.555</font>: gst_mini_object_set_qdata: assertion &apos;object != NULL&apos; failed
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4032 x 3040 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 1 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
**PERF:  30.04 (29.91)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
**PERF:  29.98 (29.97)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
**PERF:  30.01 (29.98)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
**PERF:  29.99 (29.99)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	0.00 (0.00)	
^C** ERROR: &lt;_intr_handler:140&gt;: User Interrupted.. 

Quitting
CONSUMER: Done Success
GST_ARGUS: Cleaning up
GST_ARGUS: Done Success
App run successful
</pre>
10

Seems camera does not work correctly.
You should set “camera-csi-sensor-id” for your CSI camera.

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.