Failed to load GazeNet model in TRT

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Nano 4 GB
• DeepStream Version 6.0
• JetPack Version (valid for Jetson only) 4.6
• TensorRT Version 8.0
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs) questions/bugs
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

Hello.

I’m going to use GazeNet model in Python pipeline. Build pipeline structure, using this example

  1. Download model from NGC
  2. Create engine using tao-converter:
    tao-converter -k nvidia_tlt -p input_left_images:0,1x1x224x224,1x1x224x224,1x1x224x224 -p input_right_images:0,1x1x1x224x224,1x1x1x224x224,1x1x1x224x224 -p input_face_images:0,1x1x1x224x224,1x1x1x224x224,1x1x1x224x224 -p input_facegrid:0,1x1x1x625x1,1x1x1x625x1,1x1x1x625x1 -b 4 -m 4 -t fp16 gazenet_facegrid.etlt -e gazenet_facegrid_b4_gpu0_fp16.engine
  3. Use it in pipeline:
gaze_identifier.set_property('customlib-name', "~/deepstream_tao_apps/apps/tao_others/deepstream-gaze-app/gazeinfer_impl/libnvds_gazeinfer.so")
    gaze_identifier.set_property('customlib-props', "config-file:./sample_gazenet_model_config.txt")

sample_gazenet_model_config:

enginePath=~/deepstream_tao_apps/models/gazenet/gazenet_facegrid_b4_gpu0_fp16.engine
etltPath=~/deepstream_tao_apps/models/gazenet/gazenet_facegrid.etlt
etltKey=nvidia_tlt
## networkMode can be int8, fp16 or fp32
networkMode=fp16
batchSize=4

Got the following output, when running the pipeline:

Library Opened Successfully
Setting custom lib properties # 1
Adding Prop: config-file : ./sample_gazenet_model_config.txt
Inside Custom Lib : Setting Prop Key=config-file Value=./sample_gazenet_model_config.txt
0:00:04.858322868 22736      0x8892700 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<secondary-inference> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1900> [UID = 2]: deserialized trt engine from :/home/seth/ds_meshbi/models/model_fpenet.etlt_b4_gpu0_fp16.engine
INFO: [FullDims Engine Info]: layers num: 4
0   INPUT  kFLOAT input_face_images 1x80x80         min: 1x1x80x80       opt: 4x1x80x80       Max: 4x1x80x80       
1   OUTPUT kFLOAT conv_keypoints_m80 80x80x80        min: 0               opt: 0               Max: 0               
2   OUTPUT kFLOAT softargmax      80x2            min: 0               opt: 0               Max: 0               
3   OUTPUT kFLOAT softargmax:1    80              min: 0               opt: 0               Max: 0               

ERROR: [TRT]: 3: Cannot find binding of given name: softargmax,softargmax:1,conv_keypoints_m80
0:00:04.858535320 22736      0x8892700 WARN                 nvinfer gstnvinfer.cpp:635:gst_nvinfer_logger:<secondary-inference> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1868> [UID = 2]: Could not find output layer 'softargmax,softargmax:1,conv_keypoints_m80' in engine
0:00:04.858573550 22736      0x8892700 INFO                 nvinfer gstnvinfer.cpp:638:gst_nvinfer_logger:<secondary-inference> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2004> [UID = 2]: Use deserialized engine model: /home/seth/ds_meshbi/models/model_fpenet.etlt_b4_gpu0_fp16.engine
0:00:04.910418647 22736      0x8892700 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary-inference> [UID 2]: Load new model:faciallandmark_sgie_config.txt sucessfully

Deserializing engine from: /home/seth/deepstream_tao_apps/models/gazenet/gazenet_facegrid_b4_gpu0_fp16.engineThe logger passed into createInferRuntime differs from one already provided for an existing builder, runtime, or refitter. TensorRT maintains only a single logger pointer at any given time, so the existing value, which can be retrieved with getLogger(), will be used instead. In order to use a new logger, first destroy all existing builder, runner or refitter objects.

ERROR: [TRT]: 3: [executionContext.cpp::setBindingDimensions::969] Error Code 3: Internal Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::969, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [8,1,224,224] for bindings[0] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 8.
)
ERROR: [TRT]: 3: [executionContext.cpp::setBindingDimensions::969] Error Code 3: Internal Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::969, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [8,1,224,224] for bindings[1] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 8.
)
ERROR: [TRT]: 3: [executionContext.cpp::setBindingDimensions::969] Error Code 3: Internal Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::969, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [8,1,625,1] for bindings[2] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 8.
)
ERROR: [TRT]: 3: [executionContext.cpp::setBindingDimensions::969] Error Code 3: Internal Error (Parameter check failed at: runtime/api/executionContext.cpp::setBindingDimensions::969, condition: profileMaxDims.d[i] >= dimensions.d[i]. Supplied binding dimension [8,1,224,224] for bindings[3] exceed min ~ max range at index 0, maximum dimension in profile is 1, minimum dimension in profile is 1, but supplied dimension is 8.
)

Why it cannot find bindings name?

Hi,

Did you run it with deepstream-app?
Have you tried our gaze-app before? If not, would you mind giving it a try?

Thanks.

Hi,

I just tried it, same error:

@AastaLLL Any ideas?

Hi,

Thanks for the feedback.

We are going to check this internally.
Will share more information with you later.

Thank you, looking forward to your response

Hi,

Based on your error, the device seems running out of memory.
Would you mind validating this with tegrastats?

Thanks.

Hi,

Hello guys.

I have updates. I tried use another model for this purpose (GitHub - OverEuro/deep-head-pose-lite: A lite-version hopenet for head pose estimation with PyTorch) and convert them from pytorch → onnx → trt. When I added it via nvinfer as sgie to my pipeline, the same error occured. Then I commented line with output-blob-names= and issue was gone. Then I tried the same on previous model (GazeNet) and issue was gone to.

Now I’m trying to get metadata from gazenet and deep-head-pose-lite models and check if I will get models output or not.

BTW, when I ran these models via trtexec, I don’t got any errors too.

Hi,

Thanks for the update.
We also confirmed that the GazeNet sample can work in our environment.

But the default configure is too heavy for Nano, we modify the batch to make it work.
Below is our change for your reference:

diff --git a/configs/facial_tao/config_infer_primary_facenet.txt b/configs/facial_tao/config_infer_primary_facenet.txt
index 3d44e40..0cf6588 100644
--- a/configs/facial_tao/config_infer_primary_facenet.txt
+++ b/configs/facial_tao/config_infer_primary_facenet.txt
@@ -26,8 +26,8 @@ net-scale-factor=0.0039215697906911373
 tlt-model-key=nvidia_tlt
 tlt-encoded-model=../../models/faciallandmark/facenet.etlt
 labelfile-path=labels_facenet.txt
-int8-calib-file=../../models/faciallandmark/facenet_cal.txt
-model-engine-file=../../models/faciallandmark/facenet.etlt_b1_gpu0_int8.engine
+#int8-calib-file=../../models/faciallandmark/facenet_cal.txt
+model-engine-file=../../models/faciallandmark/facenet.etlt_b1_gpu0_fp16.engine
 infer-dims=3;416;736
 uff-input-order=0
 uff-input-blob-name=input_1
@@ -35,7 +35,7 @@ batch-size=1
 process-mode=1
 model-color-format=0
 ## 0=FP32, 1=INT8, 2=FP16 mode
-network-mode=1
+network-mode=2
 num-detected-classes=1
 interval=0
 gie-unique-id=1
diff --git a/configs/facial_tao/faciallandmark_sgie_config.txt b/configs/facial_tao/faciallandmark_sgie_config.txt
index 62bb70f..80668d2 100644
--- a/configs/facial_tao/faciallandmark_sgie_config.txt
+++ b/configs/facial_tao/faciallandmark_sgie_config.txt
@@ -48,14 +48,14 @@
 
 [property]
 gpu-id=0
-model-engine-file=../../models/faciallandmark/faciallandmarks.etlt_b32_gpu0_int8.engine
+model-engine-file=../../models/faciallandmark/faciallandmarks.etlt_b1_gpu0_fp16.engine
 tlt-model-key=nvidia_tlt
 tlt-encoded-model=../../models/faciallandmark/faciallandmarks.etlt
 int8-calib-file=../../models/faciallandmark/fpenet_cal.txt
 #dynamic batch size
-batch-size=32
+batch-size=1
 ## 0=FP32, 1=INT8, 2=FP16 mode
-network-mode=1
+network-mode=2
 num-detected-classes=1
 output-blob-names=softargmax,softargmax:1,conv_keypoints_m80
 #0=Detection 1=Classifier 2=Segmentation 100=other
diff --git a/configs/gaze_tao/sample_gazenet_model_config.txt b/configs/gaze_tao/sample_gazenet_model_config.txt
index e0eaf42..14f9256 100644
--- a/configs/gaze_tao/sample_gazenet_model_config.txt
+++ b/configs/gaze_tao/sample_gazenet_model_config.txt
@@ -1,6 +1,6 @@
-enginePath=../../../../models/gazenet/gazenet_facegrid.etlt_b8_gpu0_fp16.engine
+enginePath=../../../../models/gazenet/gazenet_facegrid.etlt_b1_gpu0_fp16.engine
 etltPath=../../../../models/gazenet/gazenet_facegrid.etlt
 etltKey=nvidia_tlt
 ## networkMode can be int8, fp16 or fp32
 networkMode=fp16
-batchSize=16
+batchSize=1

Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.