• Hardware Platform: GPU
• DeepStream Version: 6.3
• TensorRT Version: 8.5.3.1
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type: Bug
We have tested the same DeepSORT model that is working correctly in nvtracker in nvinfer, but the NHWC network input order of the model is not handled correctly by nvinfer. The following configuration is used:
[property]
gpu-id=0
gie-unique-id=2
process-mode=2
operate-on-gie-id=1
batch-size=100
model-color-format=0
network-mode=0
maintain-aspect-ratio=1
force-implicit-batch-dim=1
model-engine-file=/networks/enginefiles/tensorrt-8-5-3-1/NVIDIA_TITAN_V/deepsort/deepsort-7_0-3072-4718592-49152-98304-512-80-0-batchsize_100.engine
output-blob-names=features
uff-file=/networks/models/alfastreamer/deepsort/mars-small128.uff
uff-input-order=1
network-input-order=1
infer-dims=128;64;3
uff-input-blob-name=images
network-type=1
output-tensor-meta=1
offsets=0;0;0
workspace-size=1000
When running this as a secondary network in deepstream-app in the 6.3-gc-triton-devel image the following is logged:
0:00:00.987817124 355 0x55d3fdafd980 WARN nvinfer gstnvinfer.cpp:887:gst_nvinfer_start:<secondary_gie_0> warning: NvInfer output-tensor-meta is enabled but init_params auto increase memory (auto-inc-mem) is disabled. The bufferpool will not be automatically resized.
0:00:00.988113105 355 0x55d3fdafd980 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2002> [UID = 2]: Trying to create engine from model files
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: The implicit batch dimension mode has been deprecated. Please create the network with NetworkDefinitionCreationFlag::kEXPLICIT_BATCH flag whenever possible.
0:01:02.734271508 355 0x55d3fdafd980 INFO nvinfer gstnvinfer.cpp:682:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2034> [UID = 2]: serialize cuda engine to file: /output/nhwc-repro/mars-small128.uff_b100_gpu0_fp32.engine successfully
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0 INPUT kFLOAT images 128x64x3
1 OUTPUT kFLOAT features 128x1x1
0:01:02.737354390 355 0x55d3fdafd980 WARN nvinfer gstnvinfer.cpp:679:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Warning from NvDsInferContextImpl::checkBackendParams() <nvdsinfer_context_impl.cpp:1944> [UID = 2]: backend can not support dims:64x3x128
0:01:02.737449405 355 0x55d3fdafd980 ERROR nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:2052> [UID = 2]: deserialized backend context :/output/nhwc-repro/mars-small128.uff_b100_gpu0_fp32.engine failed to match config params
0:01:02.807061424 355 0x55d3fdafd980 ERROR nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:2108> [UID = 2]: build backend context failed
0:01:02.807099834 355 0x55d3fdafd980 ERROR nvinfer gstnvinfer.cpp:676:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 2]: Error in NvDsInferContextImpl::initialize() <nvdsinfer_context_impl.cpp:1282> [UID = 2]: generate backend failed, check config file settings
0:01:02.807174377 355 0x55d3fdafd980 WARN nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<secondary_gie_0> error: Failed to create NvDsInferContext instance
0:01:02.807183737 355 0x55d3fdafd980 WARN nvinfer gstnvinfer.cpp:898:gst_nvinfer_start:<secondary_gie_0> error: Config file path: /output/nhwc-repro/config_infer_secondary_deepsort.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
** ERROR: <main:716>: Failed to set pipeline to PAUSED
Quitting
ERROR from secondary_gie_0: Failed to create NvDsInferContext instance
Debug info: gstnvinfer.cpp(898): gst_nvinfer_start (): /GstPipeline:pipeline/GstBin:secondary_gie_bin/GstNvInfer:secondary_gie_0:
Config file path: /output/nhwc-repro/config_infer_secondary_deepsort.txt, NvDsInfer Error: NVDSINFER_CONFIG_FAILED
App run failed
Looking at the log above the engine file seems to be built with correct input dimensions, but then nvinfer reports that the model it just created is not correct. We have also tested using nvtracker to build the engine file and then using the engine file in nvinfer. This is also not working although the engine file definitely is correct in that case because it can be used by the tracker.
Is there any way to change the nvinfer configuration for the DeepSORT model to make this work or is this just a bug in nvinfer?