Network input order NHWC is not handled correctly

• Hardware Platform: GPU
• DeepStream Version: 6.3
• TensorRT Version: 8.5.3.1
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type: Bug

We have tested the same DeepSORT model that is working correctly in nvtracker in nvinfer, but the NHWC network input order of the model is not handled correctly by nvinfer. The following configuration is used:

[property]
gpu-id=0
gie-unique-id=2
process-mode=2
operate-on-gie-id=1
batch-size=100
model-color-format=0
network-mode=0
maintain-aspect-ratio=1
force-implicit-batch-dim=1
model-engine-file=/networks/enginefiles/tensorrt-8-5-3-1/NVIDIA_TITAN_V/deepsort/deepsort-7_0-3072-4718592-49152-98304-512-80-0-batchsize_100.engine
output-blob-names=features
uff-file=/networks/models/alfastreamer/deepsort/mars-small128.uff
uff-input-order=1
network-input-order=1
infer-dims=128;64;3
uff-input-blob-name=images
network-type=1
output-tensor-meta=1
offsets=0;0;0
workspace-size=1000

When running this as a secondary network in deepstream-app in the 6.3-gc-triton-devel image the following is logged:

0:00:00.987817124   355 0x55d3fdafd980 WARN                 nvinfer gstnvinfer.cpp:887:gst_nvinfer_start:<secondary_gie_0> warning: NvInfer output-tensor-meta is enabled but init_params auto increase memory (auto-inc-mem) is disabled. The bufferpool will not be automatically resized.
0:00:00.988113105   355 0x55d3fdafd980 INFO                 nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2002> [UID = 2]: Trying to create engine from model files
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
0:01:02.734271508   355 0x55d3fdafd980 INFO                 nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2034> [UID = 2]: serialize cuda engine to file: /output/nhwc-repro/mars-small128.uff_b100_gpu0_fp32.engine successfully
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0   INPUT  kFLOAT images          128x64x3        
1   OUTPUT kFLOAT features        128x1x1         

0:01:02.737354390   355 0x55d3fdafd980 WARN                 nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1944> [UID = 2]: backend can not support dims:64x3x128
0:01:02.737449405   355 0x55d3fdafd980 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2052> [UID = 2]: deserialized backend context :/output/nhwc-repro/mars-small128.uff_b100_gpu0_fp32.engine failed to match config params
0:01:02.807061424   355 0x55d3fdafd980 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2108> [UID = 2]: build backend context failed
0:01:02.807099834   355 0x55d3fdafd980 ERROR                nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1282> [UID = 2]: generate backend failed, check config file settings
0:01:02.807174377   355 0x55d3fdafd980 WARN                 nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<secondary_gie_0> error: Failed to create NvDsInferContext instance
0:01:02.807183737   355 0x55d3fdafd980 WARN                 nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<secondary_gie_0> error: Config file path: /output/nhwc-repro/config_infer_secondary_deepsort.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: <main:716>: Failed to set pipeline to PAUSED
Quitting
ERROR from secondary_gie_0: Failed to create NvDsInferContext instance
Debug info: gstnvinfer.cpp(898): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:secondary_gie_bin/GstNvInfer:secondary_gie_0:
Config file path: /output/nhwc-repro/config_infer_secondary_deepsort.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
App run failed

Looking at the log above the engine file seems to be built with correct input dimensions, but then nvinfer reports that the model it just created is not correct. We have also tested using nvtracker to build the engine file and then using the engine file in nvinfer. This is also not working although the engine file definitely is correct in that case because it can be used by the tracker.

Is there any way to change the nvinfer configuration for the DeepSORT model to make this work or is this just a bug in nvinfer?

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Seems the infer-dims is wrong. infer-dims should be channel; height; width; input-order: Gst-nvinfer — DeepStream 6.3 Release documentation

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.