Config file values for deploying NGC models?

clemens.richter · December 20, 2024, 3:49pm

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson AGX Orin
• DeepStream Version
DeepStream 7.1
• JetPack Version (valid for Jetson only)
JetPack 6.0
• TensorRT Version
TensorRT 8.6.2.3-1+cuda12.2
• Issue Type( questions, new requirements, bugs)
Questions

Hello,

i am able to successfully run one of the deepstream sample models on my Jetson on my camera live feed using the following gstreamer pipeline:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw, framerate=30/1, width=3840,height=1080 ! nvvideoconvert src-crop=0:0:1920:1080 ! 'video/x-raw(memory:NVMM),width=1920,height=1080,format=NV12' ! nvvideoconvert ! m.sink_0 nvstreammux name=m batch-size=1 width=1920 height=1080 !  nvinfer config-file-path=configs/dstest1_pgie_config.yml ! nvtracker ll-lib-file=/opt/nvidia/deepstream/deepstream/lib/libnvds_nvmultiobjecttracker.so ! nvvideoconvert ! nvdsosd ! nvvideoconvert ! nv3dsink -e

Now i plan to use different models provided by the NVIDIA NGC Catalog. When downloading i am only provided with 2 files: int8_calibration.txt and model.eltl

My quesion now is: Running the nvinfer plugin in my pipeline requires passing a config, either in .yml or .txt format. How should i know, which values to specify for the configs of models i want to use? For some of the property values of the examples, i cannot even find an entry in the gst-nvinfer documentation (Gst-nvinfer — DeepStream documentation).

I am specifically interested in the models: FaceDetect (FaceDetect | NVIDIA NGC), FacialLandmarks (Facial Landmarks Estimation | NVIDIA NGC) and GazeNet (Gaze Estimation | NVIDIA NGC).
I wouldn’t mind a useful guide applicable for any model though.

Thanks!

junshengy · December 23, 2024, 2:22am

You can refer to the app below, but since version 7.1 no longer supports etlt format models, you need to use DS-7.0

github.com

NVIDIA-AI-IOT/deepstream_tao_apps/blob/release/tao5.3_ds7.0ga/apps/tao_others/deepstream-gaze-app/README.md

## Description
The GazeNet deepstream sample application recognizes a person's eye gaze point of regard (X, Y, Z) and gaze vector (theta and phi). The eye gaze vector can also be derived from eye position and eye gaze points of regard. The input of the GazeNet should be the human face and the faciallandmarks.

The TAO pretrained models used in this sample application:

* [Facial Landmark Estimation](https://ngc.nvidia.com/catalog/models/nvidia:tao:fpenet).
* [FaceNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:facenet)
* [GazeNet](https://ngc.nvidia.com/catalog/models/nvidia:tao:gazenet)

## Prerequisition

* DeepStream SDK 6.0 GA and above

* gst-nvdsvideotemplate plugin

  Since the GazeNet is a multi-input and multi-branch network and it depends on the facial landmark estimation model output which is not the standard deepstream object meta, the gst-nvinfer plugin can not support GazeNet inference.

  The gst-nvdsvideotemplate plugin is used in this sample to do the GazeNet inference. The libnvds_gazeinfer.so is the customized library for gst-nvdsvideotemplate to infer the batched iamges with the Nvidia proprietary inference library.

## Application Pipeline

This file has been truncated. show original

system · February 3, 2025, 3:01pm

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Gaze Estimation App on deepstream 6.0: Engine File Error DeepStream SDK tensorrt , deepstream	5	32	October 25, 2024
Run Faciallandmarks model with deepstream on Jetson nano 2GB DeepStream SDK tensorrt , jetson-inference	6	929	April 4, 2022
FaceDetect Pre-Trained model implementation using DS DeepStream SDK	26	915	July 30, 2023
Looking for a simple deepstream pipeline (python) which does both super-resolution and detection of objects DeepStream SDK opencv , jetson-inference , gstreamer , python	16	1869	May 24, 2023
Cannot run gaze demo DeepStream SDK deepstream	4	24	March 20, 2025
Sample for simple classification deepstream6 app DeepStream SDK	8	806	May 3, 2022
Deepstream-gaze-app: Results are not good DeepStream SDK tensorrt , opencv , cuda , gstreamer , deepstream	10	56	December 5, 2024
Deepstream GazeNet segmentation fault Frameworks deepstream	1	891	November 30, 2022
Face Recognition Deepstream App DeepStream SDK	7	1298	October 12, 2021
DeepStream 7.1 docker python GazeNet pipeline DeepStream SDK docker , jetson , deepstream , jetson-orin	5	116	March 21, 2025

Config file values for deploying NGC models?

Related topics