Ahoi everyone,
I face the following issue with my Jetson Nano 4GB when trying to run a previously build container from a running graph composer application
I’m trying to put as much information about my approach as possible.
basically im following this guide ba Microsoft: Exercise - Publish an NVIDIA DeepStream Graph Composer container to Azure Container Registry - Training | Microsoft Learn
Important to note: There is no display attached to my Jetson Nano 4GB
Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson)
• DeepStream Version
6.0 - installed via Quickstart Guide — DeepStream 6.0.1 Release documentation as the Jetson Nano has Jetpack 4.6.3 / L4T 32.7.3 installed
• JetPack Version (valid for Jetson only)
Jetpack 4.6.3 / L4T 32.7.3
• TensorRT Version
8.2.1.9
• NVIDIA GPU Driver Version (valid for GPU only)
NA
• Issue Type( questions, new requirements, bugs)
This is the error I get when I run the container
2024-07-26 22:29:22.308 ERROR extensions/nvdsbase/nvds_scheduler.cpp@179: Failed to set GStreamer pipeline to PLAYING
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Executing my previously build Container with the following YAML files
parameter.yaml
components:
- name: nv_ds_single_src_input3..c518
parameters:
uri: file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
name: NvDsSingleSrcInput..5320
---
components:
- name: nv_ds_stream_mux6..db00
parameters:
batch-size: 1
height: 1080
width: 1920
name: NvDsStreamMux..5d68
deepstream-test1.yaml
application:
name: TEST2
---
dependencies:
- extension: NvDsSourceExt
uuid: a632d022-3425-4848-9074-e6483ef74366
version: 0.0.6
- extension: NvDsBaseExt
uuid: 56d7e3ec-62c6-4652-bcc8-4f1c3b00df03
version: 0.0.4
- extension: NvDsMuxDemuxExt
uuid: 89b8398c-5820-4051-835c-a91f2d49766b
version: 0.0.4
- extension: NvDsInferenceExt
uuid: 0b02963e-c24e-4c13-ace0-c4cdf36c7c71
version: 0.0.4
- extension: NvDsSampleModelsExt
uuid: 909dc1ea-c654-44c1-97a3-0b8fec12141a
version: 0.0.4
- extension: NvDsVisualizationExt
uuid: 25903cd4-fc5c-4139-987b-47bb27e8b424
version: 0.0.3
- extension: NvDsInferenceUtilsExt
uuid: 27856a43-5ad4-4d8e-be36-0ec2cf9bbb58
version: 0.0.5
- extension: NvDsOutputSinkExt
uuid: 3fc9ad87-03e7-47a8-bbfc-8501c3f7ff2f
version: 0.0.3
---
components:
- name: nv_ds_single_src_input3..c518
parameters:
audio-out-%u: nv_ds_dynamic_output4..c7b8
uri: file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4
video-out-%u: nv_ds_dynamic_output5..cb70
type: nvidia::deepstream::NvDsSingleSrcInput
- name: nv_ds_dynamic_output4..c7b8
type: nvidia::deepstream::NvDsDynamicOutput
- name: nv_ds_dynamic_output5..cb70
type: nvidia::deepstream::NvDsDynamicOutput
name: NvDsSingleSrcInput..5320
ui_property:
position:
x: -178.45005798339844
y: -174.2137451171875
state: 0
---
components:
- name: nv_ds_stream_mux6..db00
parameters:
batch-size: 1
height: 1080
video-in-%u: nv_ds_on_request_input7..d9e8
video-out: nv_ds_static_output8..da90
width: 1920
type: nvidia::deepstream::NvDsStreamMux
- name: nv_ds_on_request_input7..d9e8
type: nvidia::deepstream::NvDsOnRequestInput
- name: nv_ds_static_output8..da90
type: nvidia::deepstream::NvDsStaticOutput
name: NvDsStreamMux..5d68
ui_property:
position:
x: -154.7552490234375
y: 60.636112213134766
state: 0
---
components:
- name: nv_ds_connection9..7978
parameters:
source: NvDsSingleSrcInput..5320/nv_ds_dynamic_output5..cb70
target: NvDsStreamMux..5d68/nv_ds_on_request_input7..d9e8
type: nvidia::deepstream::NvDsConnection
name: node..3ba8
---
components:
- name: nv_ds_infer10..2748
parameters:
infer-model-config: resnet10_4_class_detector3..b278
video-in: nv_ds_static_input11..5a90
video-out: nv_ds_static_output12..5b38
type: nvidia::deepstream::NvDsInferVideo
- name: nv_ds_static_input11..5a90
type: nvidia::deepstream::NvDsStaticInput
- name: nv_ds_static_output12..5b38
type: nvidia::deepstream::NvDsStaticOutput
- name: resnet10_4_class_detector3..b278
type: nvidia::deepstream::NvDsResnet10_4ClassDetectorModel
name: NvDsInferVideo..5080
ui_property:
position:
x: -172.15878295898438
y: 214.3798828125
state: 0
---
components:
- name: nv_ds_connection13..5e10
parameters:
source: NvDsStreamMux..5d68/nv_ds_static_output8..da90
target: NvDsInferVideo..5080/nv_ds_static_input11..5a90
type: nvidia::deepstream::NvDsConnection
name: node..7c18
---
components:
- name: nv_ds_osd14..5978
parameters:
video-in: nv_ds_static_input15..7b00
video-out: nv_ds_static_output16..7b70
type: nvidia::deepstream::NvDsOSD
- name: nv_ds_static_input15..7b00
type: nvidia::deepstream::NvDsStaticInput
- name: nv_ds_static_output16..7b70
type: nvidia::deepstream::NvDsStaticOutput
name: NvDsOSD..c668
ui_property:
position:
x: 231.75030517578125
y: -93.65518951416016
state: 0
---
components:
- name: nv_ds_connection17..ac88
parameters:
source: NvDsInferVideo..5080/nv_ds_static_output12..5b38
target: NvDsOSD..c668/nv_ds_static_input15..7b00
type: nvidia::deepstream::NvDsConnection
name: node..aeb8
---
components:
- name: per_class_object_counting21..ff28
parameters:
probe-connector: nv_ds_probe_connector0..2b00
type: nvidia::deepstream::NvDsPerClassObjectCounting
- name: nv_ds_probe_connector0..2b00
type: nvidia::deepstream::NvDsProbeConnector
name: NvDsPerClassObjectCounting..b710
ui_property:
position:
x: 229.2032012939453
y: 416.7696838378906
state: 0
---
components:
- name: nv_ds_probe1..5be0
parameters:
io: NvDsInferVideo..5080/nv_ds_static_output12..5b38
probe-connector: NvDsPerClassObjectCounting..b710/nv_ds_probe_connector0..2b00
type: nvidia::deepstream::NvDsProbe
name: node..3400
---
components:
- name: nv_ds_scheduler2..4a58
type: nvidia::deepstream::NvDsScheduler
name: NvDsScheduler..48d0
ui_property:
position:
x: 670.6022338867188
y: 501.16558837890625
state: 0
---
components:
- name: 1-to-N pipe fitting8
parameters:
in: Static Data Input9
out-%u: On Request Data Output10
type: nvidia::deepstream::NvDsTee
- name: Static Data Input9
type: nvidia::deepstream::NvDsStaticInput
- name: On Request Data Output10
type: nvidia::deepstream::NvDsOnRequestOutput
name: 1-to-N pipe fitting
ui_property:
position:
x: 428.6846008300781
y: 117.21231079101562
---
components:
- name: Deepstream Data Connection11
parameters:
source: NvDsOSD..c668/nv_ds_static_output16..7b70
target: 1-to-N pipe fitting/Static Data Input9
type: nvidia::deepstream::NvDsConnection
name: node5
---
components:
- name: Video File Output12
parameters:
output-file: out.mp4
video-in: Static Data Input13
type: nvidia::deepstream::NvDsFileOut
- name: Static Data Input13
type: nvidia::deepstream::NvDsStaticInput
name: Video File Output
ui_property:
position:
x: 800.837890625
y: 14.631561279296875
---
components:
- name: nv_ds_rtsp_out14
parameters:
audio-in: On Request Data Input15
video-in: On Request Data Input16
type: nvidia::deepstream::NvDsRtspOut
- name: On Request Data Input15
type: nvidia::deepstream::NvDsOnRequestInput
- name: On Request Data Input16
type: nvidia::deepstream::NvDsOnRequestInput
name: NvDsRtspOut
ui_property:
position:
x: 825.8868408203125
y: 181.62344360351562
---
components:
- name: Deepstream Data Connection17
parameters:
source: 1-to-N pipe fitting/On Request Data Output10
target: NvDsRtspOut/On Request Data Input16
type: nvidia::deepstream::NvDsConnection
name: node6
---
components:
- name: Deepstream Data Connection18
parameters:
source: 1-to-N pipe fitting/On Request Data Output10
target: Video File Output/Static Data Input13
type: nvidia::deepstream::NvDsConnection
name: node7
ds_test1_container_builder_jetson.yaml
---
# final stage
unique_stage: final_image
base_image: nvcr.io/nvidia/deepstream-l4t:6.0.1-base
stage_model: clean_stage
local_copy_files:
- src: "/opt/nvidia/graph-composer/graph_composer-1.0.0_arm64.deb"
dst: "/tmp/"
- src: "parameters.yaml"
dst: "/workspace/deepstream-test1/" # ends with '/' is folder
- src: "/tmp/rsync_3.1.0-2_arm64.deb"
dst: "/tmp/"
custom_runs:
- "apt install -y libpopt0"
- "dpkg -i /tmp/rsync_3.1.0-2_arm64.deb"
- "apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb"
work_folder: /workspace/deepstream-test1/
env_list:
PATH: "/opt/nvidia/graph-composer/:$PATH"
LD_LIBRARY_PATH: "/opt/nvidia/graph-composer/:$LD_LIBRARY_PATH"
# DISPLAY: ":0"
# Setup entrypoint
entrypoint:
- "gxe"
- "--manifest"
- "deepstream-test1_manifest.yaml"
- "--app"
- "deepstream-test1.yaml,parameters.yaml"
---
# Container Builder Config
container_builder: main # required
graph:
graph_files: [ deepstream-test1.yaml ]
graph_dst: /workspace/deepstream-test1/
manifest_dst: /workspace/deepstream-test1/deepstream-test1_manifest.yaml
ext_install_root: /workspace/deepstream-test1/
docker_build:
image_name: deepstream-test1-jetson_17072024:v1
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
When executing the graph application everything runs as designed.
The Graph Composer application is executed via:
sudo /opt/nvidia/graph-composer/execute_graph.sh deepstream-test1.yaml parameters.yaml -d /opt/nvidia/graph-composer/config/target_aarch64.yaml
Graph Composer Application execution Output
nvidia@ubuntu:/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2$ sudo /opt/nvidia/graph-composer/execute_graph.sh deepstream-test1.yaml parameters.yaml -d /opt/nvidia/graph-composer/config/target_aarch64.yaml
Graphs: deepstream-test1.yaml,parameters.yaml
Target: /opt/nvidia/graph-composer/config/target_aarch64.yaml
===================================================================
Running deepstream-test1.yaml
===================================================================
[INFO] Writing manifest to /tmp/ds.deepstream-test1/manifest.yaml
2024-07-27 00:16:37.806 INFO gxf/gxe/gxe.cpp@98: Creating context
2024-07-27 00:16:38.827 INFO gxf/gxe/gxe.cpp@85: Loading app: '/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2/deepstream-test1.yaml'
2024-07-27 00:16:38.827 INFO gxf/std/yaml_file_loader.cpp@59: Loading GXF entities from YAML file '/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2/deepstream-test1.yaml'...
2024-07-27 00:16:38.842 INFO gxf/gxe/gxe.cpp@85: Loading app: '/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2/parameters.yaml'
2024-07-27 00:16:38.842 INFO gxf/std/yaml_file_loader.cpp@59: Loading GXF entities from YAML file '/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2/parameters.yaml'...
2024-07-27 00:16:38.843 INFO gxf/gxe/gxe.cpp@153: Initializing...
2024-07-27 00:16:38.945 INFO extensions/nvdsbase/nvds_scheduler.cpp@266: This program is linked against GStreamer 1.14.5
2024-07-27 00:16:38.946 INFO extensions/nvdssource/nvurisrcbin.hpp@24: initialize: nvurisrcbin nv_ds_single_src_input3..c518
2024-07-27 00:16:38.947 INFO extensions/nvdsmuxdemux/nvstreammux.hpp@27: initialize: nvstreammux nv_ds_stream_mux6..db00
2024-07-27 00:16:38.947 INFO extensions/nvdsinference/nvinferbin.hpp@24: initialize: nvinferbin nv_ds_infer10..2748
2024-07-27 00:16:38.948 INFO extensions/nvdsvisualization/nvosdbin.hpp@24: initialize: nvosdbin nv_ds_osd14..5978
2024-07-27 00:16:38.949 INFO ./extensions/nvdsbase/tee.hpp@23: initialize: tee 1-to-N pipe fitting8
2024-07-27 00:16:38.949 INFO extensions/nvdsoutputsink/nvvideoencfilesinkbin.hpp@23: initialize: nvvideoencfilesinkbin Video File Output12
2024-07-27 00:16:38.949 INFO extensions/nvdsoutputsink/nvrtspoutsinkbin.hpp@23: initialize: nvrtspoutsinkbin nv_ds_rtsp_out14
2024-07-27 00:16:38.950 INFO gxf/gxe/gxe.cpp@160: Running...
2024-07-27 00:16:38.950 INFO extensions/nvdsbase/nvds_scheduler.cpp@117: Scheduling 7 elements and 1 components
2024-07-27 00:16:38.950 INFO extensions/nvdssource/nvurisrcbin.hpp@32: create_element: nvurisrcbin nv_ds_single_src_input3..c518
2024-07-27 00:16:39.264 INFO extensions/nvdssource/nvurisrcbin.hpp@56: bin_add: nvurisrcbin nv_ds_single_src_input3..c518
2024-07-27 00:16:39.264 INFO extensions/nvdsmuxdemux/nvstreammux.hpp@37: create_element: nvstreammux nv_ds_stream_mux6..db00
2024-07-27 00:16:39.268 INFO extensions/nvdsmuxdemux/nvstreammux.hpp@61: bin_add: nvstreammux nv_ds_stream_mux6..db00
2024-07-27 00:16:39.269 INFO extensions/nvdsinference/nvinferbin.hpp@32: create_element: nvinferbin nv_ds_infer10..2748
2024-07-27 00:16:39.274 INFO extensions/nvdsinference/nvinferbin.hpp@56: bin_add: nvinferbin nv_ds_infer10..2748
2024-07-27 00:16:39.277 INFO extensions/nvdsvisualization/nvosdbin.hpp@32: create_element: nvosdbin nv_ds_osd14..5978
2024-07-27 00:16:39.282 INFO extensions/nvdsvisualization/nvosdbin.hpp@56: bin_add: nvosdbin nv_ds_osd14..5978
2024-07-27 00:16:39.282 INFO ./extensions/nvdsbase/tee.hpp@31: create_element: tee 1-to-N pipe fitting8
2024-07-27 00:16:39.282 INFO ./extensions/nvdsbase/tee.hpp@55: bin_add: tee 1-to-N pipe fitting8
2024-07-27 00:16:39.282 INFO extensions/nvdsoutputsink/nvvideoencfilesinkbin.hpp@31: create_element: nvvideoencfilesinkbin Video File Output12
2024-07-27 00:16:39.283 INFO extensions/nvdsoutputsink/nvvideoencfilesinkbin.hpp@49: bin_add: nvvideoencfilesinkbin Video File Output12
2024-07-27 00:16:39.283 INFO extensions/nvdsoutputsink/nvrtspoutsinkbin.hpp@31: create_element: nvrtspoutsinkbin nv_ds_rtsp_out14
2024-07-27 00:16:39.283 INFO extensions/nvdsoutputsink/nvrtspoutsinkbin.hpp@55: bin_add: nvrtspoutsinkbin nv_ds_rtsp_out14
*** NvDsRtspOut/nv_ds_rtsp_out14: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***
Opening in BLOCKING MODE
Opening in BLOCKING MODE
ERROR: Deserialize engine failed because file path: /tmp/ds.deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_int8.engine open error
0:00:03.092522591 1501 0x7f544e8af0 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/tmp/ds.deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_int8.engine failed
0:00:03.094183973 1501 0x7f544e8af0 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/tmp/ds.deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_int8.engine failed, try rebuild
0:00:03.094234338 1501 0x7f544e8af0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
0:01:13.007931445 1501 0x7f544e8af0 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1947> [UID = 1]: serialize cuda engine to file: /tmp/ds.deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x368x640
1 OUTPUT kFLOAT conv2d_bbox 16x23x40
2 OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40
0:01:13.205847406 1501 0x7f544e8af0 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<nvinfer_bin_nvinfer> [UID 1]: Load new model:/tmp/ds.deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/config_infer_primary.txt sucessfully
Running...
****** NvDsScheduler Runtime Keyboard controls:
p: Pause pipeline
r: Resume pipeline
q: Quit pipeline
2024-07-27 00:17:52.055 INFO extensions/nvdsbase/nvds_scheduler.cpp@358: NvDsScheduler Pipeline ready
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
2024-07-27 00:17:52.483 INFO extensions/nvdsbase/nvds_scheduler.cpp@343: NvDsScheduler Pipeline running
NvMMLiteOpen : Block : BlockType = 4
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
NvMMLiteBlockCreate : Block : BlockType = 4
Source 0: Frame Number = 0 Total objects = 7 [ Car:4 Person:3 ]
Source 0: Frame Number = 1 Total objects = 7 [ Car:4 Person:3 ]
Source 0: Frame Number = 2 Total objects = 7 [ Car:4 Person:3 ]
Source 0: Frame Number = 3 Total objects = 8 [ Car:5 Person:3 ]
Source 0: Frame Number = 4 Total objects = 10 [ Car:5 Person:5 ]
Source 0: Frame Number = 5 Total objects = 8 [ Car:5 Person:3 ]
H264: Profile = 66, Level = 0
avg bitrate=0 for CBR, force to CQP mode
NVMEDIA_ENC: bBlitMode is set to TRUE
H264: Profile = 66, Level = 0
Source 0: Frame Number = 6 Total objects = 5 [ Car:3 Person:2 ]
avg bitrate=0 for CBR, force to CQP mode
Source 0: Frame Number = 7 Total objects = 6 [ Car:4 Person:2 ]
NVMEDIA_ENC: bBlitMode is set to TRUE
Source 0: Frame Number = 8 Total objects = 7 [ Car:4 Person:3 ]
Source 0: Frame Number = 9 Total objects = 7 [ Car:4 Person:3 ]
Source 0: Frame Number = 10 Total objects = 7 [ Car:4 Person:3 ]
... Frames ommited
Source 0: Frame Number = 1432 Total objects = 6 [ Bicycle:0 Car:4 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1433 Total objects = 6 [ Bicycle:0 Car:4 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1434 Total objects = 8 [ Bicycle:0 Car:6 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1435 Total objects = 9 [ Bicycle:0 Car:7 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1436 Total objects = 10 [ Bicycle:0 Car:8 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1437 Total objects = 8 [ Bicycle:0 Car:6 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1438 Total objects = 8 [ Bicycle:0 Car:6 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1439 Total objects = 8 [ Bicycle:0 Car:6 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1440 Total objects = 9 [ Bicycle:0 Car:7 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1441 Total objects = 9 [ Bicycle:0 Car:7 Person:2 Roadsign:0 ]
Source 0: Frame Number = 1442 Total objects = 0 [ Bicycle:0 Car:0 Person:0 Roadsign:0 ]
End of stream
Returned, stopping playback
Deleting pipeline
2024-07-27 00:18:40.191 INFO gxf/gxe/gxe.cpp@182: Deinitializing...
2024-07-27 00:18:40.194 INFO gxf/gxe/gxe.cpp@189: Closing log file...
2024-07-27 00:18:40.194 INFO gxf/gxe/gxe.cpp@204: Destroying context
2024-07-27 00:18:40.196 INFO gxf/gxe/gxe.cpp@211: Done.
*******************************************************************
End deepstream-test1.yaml
*******************************************************************
[INFO] Graph installation directory /tmp/ds.deepstream-test1 and manifest /tmp/ds.deepstream-test1/manifest.yaml retained
nvidia@ubuntu:/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2$
With a successful execution of the Graph Composer Application I proceeded to build the container.
sudo container_builder -c ds_test1_container_builder_jetson.yaml -d /opt/nvidia/graph-composer/config/target_aarch64.yaml
Container build Output:
ubuntu@nvidia-ds6-us1804:/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2$ sudo container_builder -c ds_test1_container_builder_jetson.yaml -d /opt/nvidia/graph-composer/config/target_aarch64.yaml
[sudo] password for ubuntu:
2024-07-26 22:13:25,929 - ContainerBuilder.container_builder.docker_manager - INFO - base_image set in config file nvcr.io/nvidia/deepstream-l4t:6.0.1-base
2024-07-26 22:13:25,929 - ContainerBuilder.container_builder.docker_manager - INFO - start building stage image deepstream-test1-jetson_17072024:v1
2024-07-26 22:13:25,946 - ContainerBuilder.container_builder.docker_templ.docker_templ_maker - INFO - writing docker content to /home/ubuntu/.cache/container_builder/.dockerfile_19879
2024-07-26 22:13:25,946 - ContainerBuilder.container_builder.docker_manager - INFO - docker file /home/ubuntu/.cache/container_builder/.dockerfile_19879 generated
2024-07-26 22:13:25,947 - ContainerBuilder.container_builder.docker_manager - INFO - start preparing work folder: /home/ubuntu/.cache/container_builder/.workfolder_19879
2024-07-26 22:13:25,947 - ContainerBuilder.container_builder.docker_workfolder - INFO - copying local files to work folder
2024-07-26 22:13:25,958 - ContainerBuilder.container_builder.docker_workfolder - INFO - Work folder: /home/ubuntu/.cache/container_builder/.workfolder_19879/final_image is ready
2024-07-26 22:13:27,219 - Registry - INFO - Installing graph ...
2024-07-26 22:13:27,242 - Registry - INFO - Extension update found for extension: NvDsBaseExt
2024-07-26 22:13:27,242 - Registry - INFO - Old version: 0.0.3 New version: 0.0.4
2024-07-26 22:13:27,244 - Registry - INFO -
Requested target configuration:
arch: aarch64 os: linux distribution: ubuntu_18.04 cuda: 10.2 cudnn: 8.0.3 tensorrt: 8.0.1.6 deepstream: 6.0 vpi: 1.1.12
Selected extension variants:
MultimediaExtension
Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04
StandardExtension
Version: 1.0.1 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 cuda: 10.2
NvDsInterfaceExt
Version: 0.0.3 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsBaseExt
Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsSourceExt
Version: 0.0.6 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsMuxDemuxExt
Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsInferenceExt
Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsSampleModelsExt
Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsOutputSinkExt
Version: 0.0.3 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsVisualizationExt
Version: 0.0.3 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
NvDsInferenceUtilsExt
Version: 0.0.5 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0
2024-07-26 22:14:11,762 - ContainerBuilder.container_builder.docker_manager - INFO - work folder: /home/ubuntu/.cache/container_builder/.workfolder_19879 is ready
2024-07-26 22:14:11,762 - ContainerBuilder.container_builder.docker_manager - INFO - start build docker file /home/ubuntu/.cache/container_builder/.dockerfile_19879
Step 1/11 : FROM nvcr.io/nvidia/deepstream-l4t:6.0.1-base AS final_image
---> b7b630713447
Step 2/11 : ARG DEBIAN_FRONTEND=noninteractive
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in d27a99a907c5
Removing intermediate container d27a99a907c5
---> 39f50d534c23
Step 3/11 : COPY ["final_image", "/"]
---> 12eaf46b0a21
Step 4/11 : RUN true
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 0b91e0998ad6
Removing intermediate container 0b91e0998ad6
---> 5245ab79f479
Step 5/11 : ENV PATH="/opt/nvidia/graph-composer/:$PATH"
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 8d724ff69631
Removing intermediate container 8d724ff69631
---> 702607305e0c
Step 6/11 : ENV LD_LIBRARY_PATH="/opt/nvidia/graph-composer/:$LD_LIBRARY_PATH"
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 07faf477f8d1
Removing intermediate container 07faf477f8d1
---> 42dc46e6cca9
Step 7/11 : WORKDIR /workspace/deepstream-test1/
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 0c1f30534fc6
Removing intermediate container 0c1f30534fc6
---> 0750bad5c82c
Step 8/11 : RUN apt install -y libpopt0
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 1534d26eb9a9
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
libpopt0
0 upgraded, 1 newly installed, 0 to remove and 10 not upgraded.
Need to get 22.8 kB of archives.
After this operation, 115 kB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic/main arm64 libpopt0 arm64 1.16-11 [22.8 kB]
Fetched 22.8 kB in 0s (116 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libpopt0:arm64.
(Reading database ... 38036 files and directories currently installed.)
Preparing to unpack .../libpopt0_1.16-11_arm64.deb ...
Unpacking libpopt0:arm64 (1.16-11) ...
Setting up libpopt0:arm64 (1.16-11) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
Removing intermediate container 1534d26eb9a9
---> dcda65329f6d
Step 9/11 : RUN dpkg -i /tmp/rsync_3.1.0-2_arm64.deb
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 5bc58e0c82ad
Selecting previously unselected package rsync.
(Reading database ... 38042 files and directories currently installed.)
Preparing to unpack /tmp/rsync_3.1.0-2_arm64.deb ...
Unpacking rsync (3.1.0-2) ...
Setting up rsync (3.1.0-2) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of restart.
Removing intermediate container 5bc58e0c82ad
---> 4a4aa186b73b
Step 10/11 : RUN apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 8d92fa8721f1
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'graph-composer' instead of '/tmp/graph_composer-1.0.0_arm64.deb'
The following NEW packages will be installed:
graph-composer
0 upgraded, 1 newly installed, 0 to remove and 11 not upgraded.
Need to get 0 B/23.0 MB of archives.
After this operation, 406 MB of additional disk space will be used.
Get:1 /tmp/graph_composer-1.0.0_arm64.deb graph-composer arm64 1.0.0 [23.0 MB]
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package graph-composer.
(Reading database ... 38071 files and directories currently installed.)
Preparing to unpack .../graph_composer-1.0.0_arm64.deb ...
Unpacking graph-composer (1.0.0) ...
Setting up graph-composer (1.0.0) ...
update-alternatives: using /opt/nvidia/graph-composer/libgxf_core.so to provide /usr/lib/aarch64-linux-gnu/libgxf_core.so (gxf_core) in auto mode
update-alternatives: using /opt/nvidia/graph-composer/gxe to provide /usr/bin/gxe (gxe) in auto mode
update-alternatives: using /opt/nvidia/graph-composer/registry to provide /usr/bin/registry (registry) in auto mode
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
Removing intermediate container 8d92fa8721f1
---> 43437098a1a6
Step 11/11 : ENTRYPOINT [ "gxe", "--manifest", "deepstream-test1_manifest.yaml", "--app", "deepstream-test1.yaml,parameters.yaml" ]
---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
---> Running in 938c63a025f4
Removing intermediate container 938c63a025f4
---> c89fd09f6914
Successfully built c89fd09f6914
Successfully tagged deepstream-test1-jetson_17072024:v1
2024-07-26 22:14:44,230 - ContainerBuilder.container_builder.docker_builder - INFO - docker build image on file:/home/ubuntu/.cache/container_builder/.dockerfile_19879, successfully
2024-07-26 22:14:44,230 - ContainerBuilder.container_builder.docker_manager - INFO - build image deepstream-test1-jetson_17072024:v1 on dockerfile /home/ubuntu/.cache/container_builder/.dockerfile_19879 sucessfully
2024-07-26 22:14:44,230 - ContainerBuilder.container_builder.docker_manager - INFO - build final image deepstream-test1-jetson_17072024:v1 sucessfully
2024-07-26 22:14:44,230 - ContainerBuilder.container_builder.docker_manager - INFO - start pruning images
2024-07-26 22:14:44,237 - ContainerBuilder.container_builder.docker_manager - INFO - Container builder build config sucessfully
ubuntu@nvidia-ds6-us1804:/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2$
Than tagged the container
docker tag deepstream-test1-jetson_17072024:v1 xxyyzz.azurecr.io/deepstream-test1-jetson_17072024:v1
Pushed the container
docker push xxyyzz.azurecr.io/deepstream-test1-jetson_17072024:v1
unfortunatelly when I try to execute the container it fails
sudo docker run -it --rm --gpus all -v /tmp/.X11-unix/:/tmp/.X11-unix/ xxyyzz.azurecr.io/deepstream-test1-jetson_17072024:v1
Output:
nvidia@ubuntu:/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2$ sudo docker run -it --rm --gpus all -v /tmp/.X11-unix/:/tmp/.X11-unix/ xxyyzz.azurecr.io/deepstream-test1-jetson_17072024:v1
Unable to find image 'xxyyzz.azurecr.io/deepstream-test1-jetson_17072024:v1' locally
v1: Pulling from deepstream-test1-jetson_17072024
f46992f278c2: Pull complete
d0ec296fcb76: Pull complete
9e18ddc8ca7a: Pull complete
457ba495c8e5: Pull complete
71bca45e35bd: Pull complete
644761cdc735: Pull complete
11628dbc31eb: Pull complete
d364c3700c33: Pull complete
01869d070b2e: Pull complete
cc3009375042: Pull complete
3e182d6364dc: Pull complete
f72feb4812f9: Pull complete
151eb940bbec: Pull complete
0e9dda2495b9: Pull complete
0e78bdc2f297: Pull complete
8dc68d594a4e: Pull complete
dcec34da2e7c: Pull complete
acae7215d8de: Pull complete
644cf9ff81e4: Pull complete
049e0a279055: Pull complete
8e9f2f36a229: Pull complete
49edaf70fd8a: Pull complete
d218a5a3f2fe: Pull complete
3ebdd498e4a1: Pull complete
e9acea715ff6: Pull complete
81db192c3dbe: Pull complete
Digest: sha256:cc7db8e192cd33c7d5a9a195a49ccf442d143fc73db4ad8916355181c849dbc6
Status: Downloaded newer image for atcloudlabjetsoncontainer.azurecr.io/deepstream-test1-jetson_17072024:v1
2024-07-26 22:41:14.408 INFO gxf/gxe/gxe.cpp@98: Creating context
2024-07-26 22:41:14.463 INFO gxf/gxe/gxe.cpp@85: Loading app: 'deepstream-test1.yaml'
2024-07-26 22:41:14.463 INFO gxf/std/yaml_file_loader.cpp@59: Loading GXF entities from YAML file 'deepstream-test1.yaml'...
2024-07-26 22:41:14.474 INFO gxf/gxe/gxe.cpp@85: Loading app: 'parameters.yaml'
2024-07-26 22:41:14.474 INFO gxf/std/yaml_file_loader.cpp@59: Loading GXF entities from YAML file 'parameters.yaml'...
2024-07-26 22:41:14.475 INFO gxf/gxe/gxe.cpp@153: Initializing...
(Argus) Error FileOperationFailed: Connecting to nvargus-daemon failed: No such file or directory (in src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 205)
(Argus) Error FileOperationFailed: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 106)
(gst-plugin-scanner:7): GStreamer-WARNING **: 22:41:16.700: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_inferserver.so': libtritonserver.so: cannot open shared object file: No such file or directory
(gst-plugin-scanner:7): GStreamer-WARNING **: 22:41:17.106: Failed to load plugin '/usr/lib/aarch64-linux-gnu/gstreamer-1.0/deepstream/libnvdsgst_udp.so': librivermax.so.0: cannot open shared object file: No such file or directory
2024-07-26 22:41:17.508 INFO extensions/nvdsbase/nvds_scheduler.cpp@266: This program is linked against GStreamer 1.14.5
2024-07-26 22:41:17.509 INFO extensions/nvdssource/nvurisrcbin.hpp@24: initialize: nvurisrcbin nv_ds_single_src_input3..c518
2024-07-26 22:41:17.510 INFO extensions/nvdsmuxdemux/nvstreammux.hpp@27: initialize: nvstreammux nv_ds_stream_mux6..db00
2024-07-26 22:41:17.511 INFO extensions/nvdsinference/nvinferbin.hpp@24: initialize: nvinferbin nv_ds_infer10..2748
2024-07-26 22:41:17.512 INFO extensions/nvdsvisualization/nvosdbin.hpp@24: initialize: nvosdbin nv_ds_osd14..5978
2024-07-26 22:41:17.513 INFO ./extensions/nvdsbase/tee.hpp@23: initialize: tee 1-to-N pipe fitting8
2024-07-26 22:41:17.514 INFO extensions/nvdsoutputsink/nvvideoencfilesinkbin.hpp@23: initialize: nvvideoencfilesinkbin Video File Output12
2024-07-26 22:41:17.514 INFO extensions/nvdsoutputsink/nvrtspoutsinkbin.hpp@23: initialize: nvrtspoutsinkbin nv_ds_rtsp_out14
2024-07-26 22:41:17.515 INFO gxf/gxe/gxe.cpp@160: Running...
2024-07-26 22:41:17.515 INFO extensions/nvdsbase/nvds_scheduler.cpp@117: Scheduling 7 elements and 1 components
2024-07-26 22:41:17.515 INFO extensions/nvdssource/nvurisrcbin.hpp@32: create_element: nvurisrcbin nv_ds_single_src_input3..c518
2024-07-26 22:41:17.654 INFO extensions/nvdssource/nvurisrcbin.hpp@56: bin_add: nvurisrcbin nv_ds_single_src_input3..c518
2024-07-26 22:41:17.654 INFO extensions/nvdsmuxdemux/nvstreammux.hpp@37: create_element: nvstreammux nv_ds_stream_mux6..db00
2024-07-26 22:41:17.657 INFO extensions/nvdsmuxdemux/nvstreammux.hpp@61: bin_add: nvstreammux nv_ds_stream_mux6..db00
2024-07-26 22:41:17.657 INFO extensions/nvdsinference/nvinferbin.hpp@32: create_element: nvinferbin nv_ds_infer10..2748
2024-07-26 22:41:17.662 INFO extensions/nvdsinference/nvinferbin.hpp@56: bin_add: nvinferbin nv_ds_infer10..2748
2024-07-26 22:41:17.664 INFO extensions/nvdsvisualization/nvosdbin.hpp@32: create_element: nvosdbin nv_ds_osd14..5978
2024-07-26 22:41:17.669 INFO extensions/nvdsvisualization/nvosdbin.hpp@56: bin_add: nvosdbin nv_ds_osd14..5978
2024-07-26 22:41:17.669 INFO ./extensions/nvdsbase/tee.hpp@31: create_element: tee 1-to-N pipe fitting8
2024-07-26 22:41:17.670 INFO ./extensions/nvdsbase/tee.hpp@55: bin_add: tee 1-to-N pipe fitting8
2024-07-26 22:41:17.670 INFO extensions/nvdsoutputsink/nvvideoencfilesinkbin.hpp@31: create_element: nvvideoencfilesinkbin Video File Output12
2024-07-26 22:41:17.670 INFO extensions/nvdsoutputsink/nvvideoencfilesinkbin.hpp@49: bin_add: nvvideoencfilesinkbin Video File Output12
2024-07-26 22:41:17.670 INFO extensions/nvdsoutputsink/nvrtspoutsinkbin.hpp@31: create_element: nvrtspoutsinkbin nv_ds_rtsp_out14
2024-07-26 22:41:17.671 INFO extensions/nvdsoutputsink/nvrtspoutsinkbin.hpp@55: bin_add: nvrtspoutsinkbin nv_ds_rtsp_out14
*** NvDsRtspOut/nv_ds_rtsp_out14: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***
Opening in BLOCKING MODE
Opening in BLOCKING MODE
ERROR: Deserialize engine failed because file path: /workspace/deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_int8.engine open error
0:00:06.503938521 1 0x7f684e4460 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1889> [UID = 1]: deserialize engine from file :/workspace/deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_int8.engine failed
0:00:06.505211571 1 0x7f684e4460 WARN nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1996> [UID = 1]: deserialize backend context from engine from file :/workspace/deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_int8.engine failed, try rebuild
0:00:06.505261363 1 0x7f684e4460 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: INT8 not supported by platform. Trying FP16 mode.
0:01:16.905342171 1 0x7f684e4460 INFO nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<nvinfer_bin_nvinfer> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1947> [UID = 1]: serialize cuda engine to file: /workspace/deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/primary.resnet10.caffemodel_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0 INPUT kFLOAT input_1 3x368x640
1 OUTPUT kFLOAT conv2d_bbox 16x23x40
2 OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40
0:01:17.166728466 1 0x7f684e4460 INFO nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<nvinfer_bin_nvinfer> [UID 1]: Load new model:/workspace/deepstream-test1/909dc1ea-c654-44c1-97a3-0b8fec12141a/0.0.4/config_infer_primary.txt sucessfully
2024-07-26 22:42:31.657 ERROR extensions/nvdsbase/nvds_scheduler.cpp@179: Failed to set GStreamer pipeline to PLAYING
Returned, stopping playback
Deleting pipeline
2024-07-26 22:42:31.670 INFO gxf/gxe/gxe.cpp@182: Deinitializing...
2024-07-26 22:42:31.673 INFO gxf/gxe/gxe.cpp@189: Closing log file...
2024-07-26 22:42:31.673 INFO gxf/gxe/gxe.cpp@204: Destroying context
2024-07-26 22:42:31.676 INFO gxf/gxe/gxe.cpp@211: Done.
nvidia@ubuntu:/opt/nvidia/deepstream/deepstream-6.0/reference_graphs/TEST2$
really appreciate your help and input on this topic.
BR Maximilian