Deepstream 6.0 + Container Builder build for Jetson nano 4GB fails

Ahoi everyone,

I face the following issue with my dGPU VM running the Container Builder according to Exercise - Build and publish cross-platform DeepStream container images - Training | Microsoft Learn

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
GPU
NAME=“Ubuntu”
VERSION=“18.04.6 LTS (Bionic Beaver)”
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=“Ubuntu 18.04.6 LTS”
VERSION_ID=“18.04”
HOME_URL=“https://www.ubuntu.com/
SUPPORT_URL=“https://help.ubuntu.com/
BUG_REPORT_URL=“Bugs : Ubuntu
PRIVACY_POLICY_URL=“Data privacy | Ubuntu
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

• DeepStream Version
Deepstream 6.0 installed per Quickstart Guide — DeepStream 6.0 Release documentation

• JetPack Version (valid for Jetson only)
NA

• TensorRT Version
TensorRT 8.0.1

• NVIDIA GPU Driver Version (valid for GPU only)
image

• Issue Type( questions, new requirements, bugs)
When running the following command to start the Container Builder it fails
CMD executed

sudo container_builder -c ds_test1_container_builder_jetson.yaml -d /opt/nvidia/graph-composer/config/target_aarch64.yaml

Output:

2024-07-15 21:16:27,831 - ContainerBuilder.container_builder.docker_manager - INFO - base_image set in config file nvcr.io/nvidia/deepstream-l4t:6.0.1-samples
2024-07-15 21:16:27,832 - ContainerBuilder.container_builder.docker_manager - INFO - start building stage image deepstream-test1-jetson_15072024
2024-07-15 21:16:27,849 - ContainerBuilder.container_builder.docker_templ.docker_templ_maker - INFO - writing docker content to /home/ubuntu/.cache/container_builder/.dockerfile_9628
2024-07-15 21:16:27,849 - ContainerBuilder.container_builder.docker_manager - INFO - docker file /home/ubuntu/.cache/container_builder/.dockerfile_9628 generated
2024-07-15 21:16:27,849 - ContainerBuilder.container_builder.docker_manager - INFO - start preparing work folder: /home/ubuntu/.cache/container_builder/.workfolder_9628
2024-07-15 21:16:27,849 - ContainerBuilder.container_builder.docker_workfolder - INFO - copying local files to work folder
2024-07-15 21:16:27,881 - ContainerBuilder.container_builder.docker_workfolder - INFO - Work folder: /home/ubuntu/.cache/container_builder/.workfolder_9628/final_image is ready
2024-07-15 21:16:29,106 - Registry - INFO - Installing graph ...
2024-07-15 21:16:29,132 - Registry - INFO - Extension update found for extension: NvDsBaseExt
2024-07-15 21:16:29,132 - Registry - INFO - Old version: 0.0.3 New version: 0.0.4
2024-07-15 21:16:29,135 - Registry - INFO - 
Requested target configuration: 
  arch: aarch64 os: linux distribution: ubuntu_18.04 cuda: 10.2 cudnn: 8.0.3 tensorrt: 8.0.1.6 deepstream: 6.0 vpi: 1.1.12 

Selected extension variants: 
MultimediaExtension 
  Version: 1.0.0 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 
StandardExtension 
  Version: 1.0.1 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 cuda: 10.2 
NvDsInterfaceExt 
  Version: 0.0.3 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsBaseExt 
  Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsSourceExt 
  Version: 0.0.6 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsMuxDemuxExt 
  Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsInferenceExt 
  Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsSampleModelsExt 
  Version: 0.0.4 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsOutputSinkExt 
  Version: 0.0.3 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsVisualizationExt 
  Version: 0.0.3 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 
NvDsInferenceUtilsExt 
  Version: 0.0.5 Variant: arch: aarch64 os: linux distribution: ubuntu_18.04 deepstream: 6.0 

2024-07-15 21:17:13,227 - ContainerBuilder.container_builder.docker_manager - INFO - work folder: /home/ubuntu/.cache/container_builder/.workfolder_9628 is ready
2024-07-15 21:17:13,227 - ContainerBuilder.container_builder.docker_manager - INFO - start build docker file /home/ubuntu/.cache/container_builder/.dockerfile_9628
Step 1/10 : FROM  nvcr.io/nvidia/deepstream-l4t:6.0.1-samples AS final_image
 ---> 0468d5478a64
Step 2/10 : ARG DEBIAN_FRONTEND=noninteractive
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in 9bd4f03bca29
Removing intermediate container 9bd4f03bca29
 ---> 4da9dd06ebc6
Step 3/10 : COPY ["final_image", "/"]
 ---> b65eea9b0f2e
Step 4/10 : RUN true
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in 6b6097d5602f
Removing intermediate container 6b6097d5602f
 ---> 65890afb4265
Step 5/10 : ENV PATH="/opt/nvidia/graph-composer/:$PATH"
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in f538683f7c0c
Removing intermediate container f538683f7c0c
 ---> 6aacf81b4025
Step 6/10 : ENV LD_LIBRARY_PATH="/opt/nvidia/graph-composer/:$LD_LIBRARY_PATH"
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in 99328d625420
Removing intermediate container 99328d625420
 ---> bd9342ec3ea0
Step 7/10 : ENV DISPLAY=":0"
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in 1543ae04dd01
Removing intermediate container 1543ae04dd01
 ---> ac8e96d60957
Step 8/10 : WORKDIR /workspace/deepstream-test1/
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in 3e94f878ccea
Removing intermediate container 3e94f878ccea
 ---> 891230437067
Step 9/10 : RUN apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb
 ---> [Warning] The requested image's platform (linux/arm64) does not match the detected host platform (linux/amd64/v3) and no specific platform was requested
 ---> Running in 1d6c814d13ae
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Note, selecting 'graph-composer' instead of '/tmp/graph_composer-1.0.0_arm64.deb'
The following additional packages will be installed:
  libpopt0 rsync
Suggested packages:
  openssh-client openssh-server
The following NEW packages will be installed:
  graph-composer libpopt0 rsync
0 upgraded, 3 newly installed, 0 to remove and 10 not upgraded.
Need to get 330 kB/23.3 MB of archives.
After this operation, 406 MB of additional disk space will be used.
Get:1 http://ports.ubuntu.com/ubuntu-ports bionic/main arm64 libpopt0 arm64 1.16-11 [22.8 kB]
Err:2 http://ports.ubuntu.com/ubuntu-ports bionic-updates/main arm64 rsync arm64 3.1.2-2.1ubuntu1.2
  404  Not Found [IP: 185.125.190.39 80]
Get:3 /tmp/graph_composer-1.0.0_arm64.deb graph-composer arm64 1.0.0 [23.0 MB]
Fetched 22.8 kB in 1s (34.7 kB/s)   
E: Failed to fetch http://ports.ubuntu.com/ubuntu-ports/pool/main/r/rsync/rsync_3.1.2-2.1ubuntu1.2_arm64.deb  404  Not Found [IP: 185.125.190.39 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb' returned a non-zero code: 100
2024-07-15 21:17:31,221 - ContainerBuilder.container_builder.docker_builder - ERROR - docker build failed on file: /home/ubuntu/.cache/container_builder/.dockerfile_9628, buildError:The command '/bin/sh -c apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb' returned a non-zero code: 100
Traceback (most recent call last):
  File "container_builder/docker_builder.py", line 113, in build_from
  File "container_builder/docker_builder.py", line 72, in _internal_build
  File "container_builder/docker_builder.py", line 53, in _print_chunk
docker.errors.BuildError: The command '/bin/sh -c apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb' returned a non-zero code: 100
2024-07-15 21:17:31,222 - ContainerBuilder.container_builder.docker_manager - ERROR - build image deepstream-test1-jetson_15072024 on dockerfile /home/ubuntu/.cache/container_builder/.dockerfile_9628 failed
2024-07-15 21:17:31,222 - ContainerBuilder.container_builder.docker_manager - ERROR - build stage image deepstream-test1-jetson_15072024 failed
2024-07-15 21:17:31,246 - ContainerBuilder.container_builder.container_builder - ERROR - build config file: ds_test1_container_builder_jetson.yaml failed

This is the ds_test1_container_builder_jetson.yaml file - untouched except the image_name variable

---
# final stage
unique_stage: final_image
base_image: nvcr.io/nvidia/deepstream-l4t:6.0.1-samples
stage_model: clean_stage

local_copy_files:
- src: "/opt/nvidia/graph-composer/graph_composer-1.0.0_arm64.deb"
dst: "/tmp/"
- src: "parameters.yaml"
dst: "/workspace/deepstream-test1/" # ends with '/' is folder

custom_runs:
- "apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb"

work_folder: /workspace/deepstream-test1/
env_list:
PATH: "/opt/nvidia/graph-composer/:$PATH"
LD_LIBRARY_PATH: "/opt/nvidia/graph-composer/:$LD_LIBRARY_PATH"
DISPLAY: ":0"

# Setup entrypoint
entrypoint:
- "gxe"
- "--manifest"
- "deepstream-test1_manifest.yaml"
- "--app"
- "deepstream-test1.yaml,parameters.yaml"
---
# Container Builder Config
container_builder: main # required
graph:
graph_files: [ deepstream-test1.yaml ]
graph_dst: /workspace/deepstream-test1/
manifest_dst: /workspace/deepstream-test1/deepstream-test1_manifest.yaml
ext_install_root: /workspace/deepstream-test1/
docker_build:
image_name: deepstream-test1-jetson_15072024

As this is my Jetson Nano 4GB module
image

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Running

sudo container_builder -c ds_test1_container_builder_jetson.yaml -d /opt/nvidia/graph-composer/config/target_aarch64.yaml

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I already tried man different images from DeepStream | NVIDIA NGC
Also tried to replace the /etc/apt/sources.list with this one /etc/apt/sources.list for Ubuntu 18.04.1 LTS Bionic Beaver · GitHub

It changed the error to:

Get:1 http://at.archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:2 http://at.archive.ubuntu.com/ubuntu bionic-updates InRelease [102 kB]
Get:3 http://at.archive.ubuntu.com/ubuntu bionic-backports InRelease [102 kB]
Err:1 http://at.archive.ubuntu.com/ubuntu bionic InRelease
  Couldn't create temporary file /tmp/apt.conf.5d7ZLg for passing config to apt-key
Get:4 http://at.archive.ubuntu.com/ubuntu bionic-security InRelease [102 kB]
Err:2 http://at.archive.ubuntu.com/ubuntu bionic-updates InRelease
  Couldn't create temporary file /tmp/apt.conf.737pjg for passing config to apt-key
Err:3 http://at.archive.ubuntu.com/ubuntu bionic-backports InRelease
  Couldn't create temporary file /tmp/apt.conf.7E3nsk for passing config to apt-key
Err:4 http://at.archive.ubuntu.com/ubuntu bionic-security InRelease
  Couldn't create temporary file /tmp/apt.conf.BbHo6j for passing config to apt-key
Reading package lists... Done
W: GPG error: http://at.archive.ubuntu.com/ubuntu bionic InRelease: Couldn't create temporary file /tmp/apt.conf.5d7ZLg for passing config to apt-key
E: The repository 'http://at.archive.ubuntu.com/ubuntu bionic InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://at.archive.ubuntu.com/ubuntu bionic-updates InRelease: Couldn't create temporary file /tmp/apt.conf.737pjg for passing config to apt-key
E: The repository 'http://at.archive.ubuntu.com/ubuntu bionic-updates InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://at.archive.ubuntu.com/ubuntu bionic-backports InRelease: Couldn't create temporary file /tmp/apt.conf.7E3nsk for passing config to apt-key
E: The repository 'http://at.archive.ubuntu.com/ubuntu bionic-backports InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://at.archive.ubuntu.com/ubuntu bionic-security InRelease: Couldn't create temporary file /tmp/apt.conf.BbHo6j for passing config to apt-key
E: The repository 'http://at.archive.ubuntu.com/ubuntu bionic-security InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
The command '/bin/sh -c apt update -y' returned a non-zero code: 100
2024-07-15 21:09:34,316 - ContainerBuilder.container_builder.docker_builder - ERROR - docker build failed on file: /home/ubuntu/.cache/container_builder/.dockerfile_8914, buildError:The command '/bin/sh -c apt update -y' returned a non-zero code: 100
Traceback (most recent call last):
  File "container_builder/docker_builder.py", line 113, in build_from
  File "container_builder/docker_builder.py", line 72, in _internal_build
  File "container_builder/docker_builder.py", line 53, in _print_chunk
docker.errors.BuildError: The command '/bin/sh -c apt update -y' returned a non-zero code: 100
2024-07-15 21:09:34,317 - ContainerBuilder.container_builder.docker_manager - ERROR - build image deepstream-test1-jetson on dockerfile /home/ubuntu/.cache/container_builder/.dockerfile_8914 failed
2024-07-15 21:09:34,317 - ContainerBuilder.container_builder.docker_manager - ERROR - build stage image deepstream-test1-jetson failed
2024-07-15 21:09:34,342 - ContainerBuilder.container_builder.container_builder - ERROR - build config file: ds_test1_container_builder_jetson.yaml failed

Really appreciating your help as I go my way learning to get a AI project running.

BR Maximilian

Small update because the error bugged me…

managed to successfully ran container build with the following yaml config:

---
# final stage
unique_stage: final_image
base_image: nvcr.io/nvidia/deepstream-l4t:6.0.1-base
stage_model: clean_stage

local_copy_files:
- src: "/opt/nvidia/graph-composer/graph_composer-1.0.0_arm64.deb"
dst: "/tmp/"
- src: "parameters.yaml"
dst: "/workspace/deepstream-test1/" # ends with '/' is folder
- src: "/tmp/rsync_3.1.0-2_arm64.deb"
dst: "/tmp/"

custom_runs:
- "apt install -y libpopt0"
- "dpkg -i /tmp/rsync_3.1.0-2_arm64.deb"
- "apt install -y /tmp/graph_composer-1.0.0_arm64.deb && rm -rf /tmp/graph-composer-1.0.0_arm64.deb"

work_folder: /workspace/deepstream-test1/

env_list:
PATH: "/opt/nvidia/graph-composer/:$PATH"
LD_LIBRARY_PATH: "/opt/nvidia/graph-composer/:$LD_LIBRARY_PATH"
DISPLAY: ":0"

# Setup entrypoint
entrypoint:
- "gxe"
- "--manifest"
- "deepstream-test1_manifest.yaml"
- "--app"
- "deepstream-test1.yaml,parameters.yaml"
---
# Container Builder Config
container_builder: main # required
graph:
graph_files: [ deepstream-test1.yaml ]
graph_dst: /workspace/deepstream-test1/
manifest_dst: /workspace/deepstream-test1/deepstream-test1_manifest.yaml
ext_install_root: /workspace/deepstream-test1/
docker_build:
image_name: deepstream-test1-jetson_15072024

let’s see how it continues…

Cheers!

Glad to hear that. You can file a new topic if there are new issues when you are using DeepStream.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.