Issues Running classifier trained in pytorch in Deepstream?

[beefshepherd]

Hardware RTX 3060
DS version 5.1

This has been an open issue since long,

I’m reattaching relevant files with the error logs here,

I’ve attached the following files in the same,

1. Exported Onnx File
2. Pytorch trained Classifier
3. Model Config File used in deepstream pipeline.

resnet18.onnx (42.6 MB)
resnet18.pt (42.7 MB)

[property]
gpu-id=0
# preprocessing parameters: These are the same for all classification models generated by TLT.
net-scale-factor=1.0
offsets=123.67;116.28;103.53
model-color-format=0
batch-size=64

labelfile-path=labels.txt
onnx-file=resnet18.onnx
infer-dims=3;224;224 # where c = number of channels, h = height of the model input, w = width of model input, 0: implies CHW format.
uff-input-blob-name=actual_input
output-blob-names=output

## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
# process-mode: 2 - inferences on crops from primary detector, 1 - inferences on whole frame
process-mode=2
interval=0
network-type=1 # defines that the model is a classifier.
gie-unique-id=4
classifier-threshold=0.51
secondary-reinfer-interval=0

I’ve also been able to verify the following commands,

trtexec --onnx=resnet18.onnx --explicitBatch, This command runs perfectly fine with the following log, [02/25/2022-23:28:29] [I] total compute time: 2.99227 s &&&& PASSED TensorRT.trtexec # /usr/src/tensorrt/bin/trtexec --onnx=resnet18.onnx --explicitBatch

Please let me know if you need anything else. I feel the shared files should be more than enuff to replicate on your end.
The problem with DS pipeline is that the DS pipeline builds the engine successfuly but fails immediately after it with the same log attached in previous post,

8.onnx_b64_gpu0_fp16.engine successfully
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Flatten_47: reshaping failed for tensor: 189
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: …/nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{64 512 1 1} {1 512}
INFO: …/nvdsinfer/nvdsinfer_model_builder.cpp:685 [FullDims Engine Info]: layers num: 2
0 INPUT kFLOAT actual_input 3x224x224 min: 1x3x224x224 opt: 64x3x224x224 Max: 64x3x224x224
1 OUTPUT kFLOAT output 2 min: 0 opt: 0 Max: 0

iris-app: nvdsinfer_context_impl.cpp:1302: NvDsInferStatus nvdsinfer::NvDsInferContextImpl::allocateBuffers(): Assertion `bindingDims.numElements > 0’ failed.
Aborted (core dumped)

Would love if this issue could be addressed as soon as possible. Thanks.

Edit: labels file is a binary classifier, yes;no

[kayccc]

Sorry for the late response, have you managed to get issue resolved or still need the support? Thanks

[beefshepherd]

@kayccc no the issue hasn’t been resolved. Would love if there was a fix for this issue. Thanks. I’ve attached relevant files to reproduce. Can you please check. Thanks.

[mchi]

@beefshepherd

batch-size=64

Hi @beefshepherd ,
I checked your onnx model, it has fixed shape with batch size == 1, so it could fail if you run it with “batch-size=64”.
Could you run below command to modify it to be dynamic shape and use the simplied_dynamic_batch_resnet18.onnx then try again?

// onnx-simplifier · PyPI
$ pip3 install onnx-simplifier
$ python3 -m onnxsim resnet18.onnx simplied_dynamic_batch_resnet18.onnx --dynamic-input-shape --input-shape input:1,3,224,224

[beefshepherd]

DS Pipeline started however crashed with the following log,

ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Flatten_47: reshaping failed for tensor: 189
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: shapeMachine.cpp (160) - Shape Error in executeReshape: reshape would change volume
ERROR: ../nvdsinfer/nvdsinfer_func_utils.cpp:33 [TRT]: Instruction: RESHAPE{1 512 1 1} {64 512}
ERROR: nvdsinfer_backend.cpp:460 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1533 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR
0:00:03.085597178 66477 0x56368c45e180 WARN                 nvinfer gstnvinfer.cpp:1225:gst_nvinfer_input_queue_loop:<secondary_gie_2> error: Failed to queue input batch for inferencing
ERROR from secondary_gie_2: Failed to queue input batch for inferencing
Debug info: gstnvinfer.cpp(1225): gst_nvinfer_input_queue_loop (): /GstPipeline:pipeline/GstBin:secondary_gie_bin/GstNvInfer:secondary_gie_2

[mchi]

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Have you changed your model to be dynamic batch onnx ?