DS4.0 with custom onnx working but DS5.0 not

Description

Hi, I am using Deepstream5.0 with my custom onnx model on x86 machine. While running the inference there is an error regarding TensorRT reshape…

I had successfully ran the same model with Deepstream4.0.2 previously and did same upgrades with DS5.0.

The error is -

create pipeline
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: (Unnamed Layer* 233) [Shuffle]: reshaping failed for tensor: 879
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 19840 2} {9920 2}
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: (Unnamed Layer* 233) [Shuffle]: reshaping failed for tensor: 879
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 19840 2} {9920 2}
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: (Unnamed Layer* 233) [Shuffle]: reshaping failed for tensor: 879
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 19840 2} {9920 2}
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: (Unnamed Layer* 233) [Shuffle]: reshaping failed for tensor: 879
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 19840 2} {9920 2}
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: (Unnamed Layer* 233) [Shuffle]: reshaping failed for tensor: 879
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 19840 2} {9920 2}
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: (Unnamed Layer* 233) [Shuffle]: reshaping failed for tensor: 879
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 19840 2} {9920 2}
0:00:01.052997295 5611 0x55ccb9839640 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1701> [UID = 1]: deserialized trt engine from :/home/inkers/rahul/inkFR2.0/launchconfigs/Primary_UpperBody/meanfixed640_372/mnetv2_372x640_simplified.onnx_b2_gpu0_fp32.engine
INFO: …/nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 7
0 INPUT kFLOAT image 3x372x640 min: 1x3x372x640 opt: 2x3x372x640 Max: 2x3x372x640
1 OUTPUT kFLOAT face_bbox -1x5 min: 0 opt: 0 Max: 0
2 OUTPUT kFLOAT face_landm -1x15 min: 0 opt: 0 Max: 0
3 OUTPUT kFLOAT person_bbox -1x5 min: 0 opt: 0 Max: 0
4 OUTPUT kFLOAT person_landm -1x15 min: 0 opt: 0 Max: 0
5 OUTPUT kFLOAT face_cls 9920x2 min: 0 opt: 0 Max: 0
6 OUTPUT kFLOAT person_cls 9920x2 min: 0 opt: 0 Max: 0

0:00:01.053130285 5611 0x55ccb9839640 INFO nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1805> [UID = 1]: Use deserialized engine model: /home/inkers/rahul/inkFR2.0/launchconfigs/Primary_UpperBody/meanfixed640_372/mnetv2_372x640_simplified.onnx_b2_gpu0_fp32.engine
inkfr3-app: nvdsinfer_context_impl.cpp:1301: NvDsInferStatus nvdsinfer::NvDsInferContextImpl::allocateBuffers(): Assertion `bindingDims.numElements > 0’ failed.

Environment

TensorRT Version: 7.2.1
GPU Type: Descrete
Nvidia Driver Version: 450.102.04
CUDA Version: 10.2
CUDNN Version: 7.6.5
Operating System + Version: Ubuntu18.04 running on x86.
Python Version (if applicable): python3.6
TensorFlow Version (if applicable): Not used
PyTorch Version (if applicable): Not used
Baremetal or Container (if container which image + tag): not used

If required i can share the changes done in files.
Thanks.

Hi @sharma.rahul98912,

Could you please try force-implicit-batch-dim = true to force ONNX to old shape(batching).

Thank you

Hi thanks for the reply.

I tried adding mentioned flag “force-implicit-batch-dim = true”, but then the error is -

ERROR: ModelImporter.cpp:474 In function importModel:
[4] Assertion failed: !_importer_ctx.network()->hasImplicitBatchDimension() && “This version of the ONNX parser only supports TensorRT INetworkDefinitions with an explicit batch dimension. Please ensure the network was created using the EXPLICIT_BATCH NetworkDefinitionCreationFlag.”
ERROR: …/nvdsinfer/nvdsinfer_model_builder.cpp:390 Failed to parse onnx file
ERROR: …/nvdsinfer/nvdsinfer_model_builder.cpp:1039 failed to build network since parsing model errors.
ERROR: …/nvdsinfer/nvdsinfer_model_builder.cpp:872 failed to build network.

I know that TRT>=7 does support explicit batch bydefault with any onnx model so
Also i did check with trtexec , and the model is passed successfully-
trtexec --explicitBatch --onnx=onnx_model.onnx

Not sure what is the issue.

Hi .
Update-

I used trtexec --explicitBatch --onnx=onnx_model.onnx --saveEngine to store the engine file and then generated file is working.

Hi,
One more doubt regarding “pgie_pad_buffer_probe()”.

To get tensor_data out i am updating the following changes inside “create_common_elements()” in deepstream_app.c-

if (config->primary_gie_config.enable) {
if (!create_primary_gie_bin (&config->primary_gie_config,
&pipeline->common_elements.primary_gie_bin)) {
goto done;
}
gst_bin_add (GST_BIN (pipeline->pipeline),
pipeline->common_elements.primary_gie_bin.bin);
if (*sink_elem) {
NVGSTDS_LINK_ELEMENT (pipeline->common_elements.primary_gie_bin.bin,
*sink_elem);
}
*sink_elem = pipeline->common_elements.primary_gie_bin.bin;
if (!*src_elem) {
src_elem = pipeline->common_elements.primary_gie_bin.bin;
}
NVGSTDS_ELEM_ADD_PROBE(pipeline->common_elements.primary_tensor_buffer_probe_id,
pipeline->common_elements.tracker_bin.queue, “src”,
pgie_pad_buffer_probe, GST_PAD_PROBE_TYPE_BUFFER,
&pipeline->common_elements);
/

NVGSTDS_ELEM_ADD_PROBE (pipeline->common_elements.primary_bbox_buffer_probe_id,
pipeline->common_elements.primary_gie_bin.bin, “src”,
gie_primary_processing_done_buf_prob, GST_PAD_PROBE_TYPE_BUFFER,
pipeline->common_elements.appCtx);
*/
}

It is giving me segmentation fault.

I have enabled “output-tensor-meta”.

Please let me know if some correction is required.
Thanks in advance.

Hi @sharma.rahul98912,

This looks like DeepStream related issue. Please post in the related forum.

Thank you.