• Hardware (NVIDIA GeForce RTX 3060]
• Network Type (ActionRecognition]
• Ubuntu 20.04 Deepstream-6.1
So I trained a custom 3d model of action recognition net but while deploying it in deepstream I’m facing this error -
divya@divya-GF65-Thin-10UE:/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-3d-action-recognition$ ./deepstream-3d-action-recognition -c deepstream_action_recognition_config.txt
ERROR: Batch-size in network-input-shape should be atleast Sum Total of ROIs
num-sources = 2
Now playing: file:///home/divya/Downloads/ride.mp4, file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_ride_bike.mov,
WARNING: ../nvdsinfer/nvdsinfer_model_builder.cpp:1482 Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-3d-action-recognition/./resnet18_3d_rgb_hmdb5_32.etlt_b4_gpu0_fp16.engine open error
0:00:00.760950828 6105 0x5652d74f6120 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1888> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-3d-action-recognition/./resnet18_3d_rgb_hmdb5_32.etlt_b4_gpu0_fp16.engine failed
0:00:00.773897864 6105 0x5652d74f6120 WARN nvinfer gstnvinfer.cpp:643:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1993> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-6.1/sources/apps/sample_apps/deepstream-3d-action-recognition/./resnet18_3d_rgb_hmdb5_32.etlt_b4_gpu0_fp16.engine failed, try rebuild
0:00:00.773908617 6105 0x5652d74f6120 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1914> [UID = 1]: Trying to create engine from model files
WARNING: [TRT]: onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
0:00:11.642285793 6105 0x5652d74f6120 INFO nvinfer gstnvinfer.cpp:646:gst_nvinfer_logger:<primary-nvinference-engine> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1946> [UID = 1]: serialize cuda engine to file: /home/divya/computer_vision/cv_samples_v1.4.0/action_recognition_net/results/export/rgb_resnet18_3.etlt_b1_gpu0_fp16.engine successfully
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [Implicit Engine Info]: layers num: 2
0 INPUT kFLOAT input_rgb 3x3x224x224
1 OUTPUT kFLOAT fc_pred 2
0:00:11.659249247 6105 0x5652d74f6120 INFO nvinfer gstnvinfer_impl.cpp:328:notifyLoadModelStatus:<primary-nvinference-engine> [UID 1]: Load new model:config_infer_primary_3d_action.txt sucessfully
Running...
ERROR from element preprocess-plugin: Configuration file parsing failed
Error details: gstnvdspreprocess.cpp(372): gst_nvdspreprocess_start (): /GstPipeline:preprocess-test-pipeline/GstNvDsPreProcess:preprocess-plugin:
Config file path: config_preprocess_3d_custom.txt
Returned, stopping playback
this is the config_preprocess_3d_custom.txt file -
[property]
enable=1
target-unique-ids=1
# network-input-shape: batch, channel, sequence, height, width
# 3D sequence of 64 images
#network-input-shape= 4;3;32;224;224
# 3D sequence of 32 images
network-input-shape= 1;3;16;112;112
# 0=RGB, 1=BGR, 2=GRAY
network-color-format=0
# 0=NCHW, 1=NHWC, 2=CUSTOM
network-input-order=2
# 0=FP32, 1=UINT8, 2=INT8, 3=UINT32, 4=INT32, 5=FP16
tensor-data-type=0
tensor-name=0
processing-width=112
processing-height=112
# 0=NVBUF_MEM_DEFAULT 1=NVBUF_MEM_CUDA_PINNED 2=NVBUF_MEM_CUDA_DEVICE
# 3=NVBUF_MEM_CUDA_UNIFIED 4=NVBUF_MEM_SURFACE_ARRAY(Jetson)
scaling-pool-memory-type=0
# 0=NvBufSurfTransformCompute_Default 1=NvBufSurfTransformCompute_GPU
# 2=NvBufSurfTransformCompute_VIC(Jetson)
scaling-pool-compute-hw=0
# Scaling Interpolation method
# 0=NvBufSurfTransformInter_Nearest 1=NvBufSurfTransformInter_Bilinear 2=NvBufSurfTransformInter_Algo1
# 3=NvBufSurfTransformInter_Algo2 4=NvBufSurfTransformInter_Algo3 5=NvBufSurfTransformInter_Algo4
# 6=NvBufSurfTransformInter_Default
scaling-filter=0
# model input tensor pool size
tensor-buf-pool-size=8
custom-lib-path=/opt/nvidia/deepstream/deepstream/lib/libnvds_custom_sequence_preprocess.so
#custom-lib-path=./custom_sequence_preprocess/libnvds_custom_sequence_preprocess.so
custom-tensor-preparation-function=CustomSequenceTensorPreparation
# 3D conv custom params
[user-configs]
channel-scale-factors=0.007843137;0.007843137;0.007843137
channel-mean-offsets=110.79;103.3;96.26
stride=1
subsample=0
[group-0]
src-ids=0;1
process-on-roi=1
roi-params-src-0=0;0;1280;720
roi-params-src-1=0;0;1280;720
#roi-params-src-2=0;0;1280;720
#roi-params-src-3=0;0;1280;720
config_infer_primary_3d_action.txt -
[property]
gpu-id=0
tlt-encoded-model=/home/divya/computer_vision/cv_samples_v1.4.0/action_recognition_net/results/export/rgb_resnet18_3.etlt
tlt-model-key=nvidia_tao
model-engine-file=./resnet18_3d_rgb_hmdb5_32.etlt_b4_gpu0_fp16.engine
labelfile-path=labels.txt
batch-size=1
process-mode=1
# requries preprocess metadata input
input-tensor-from-meta=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
gie-unique-id=1
# 1: classifier, 100: custom
network-type=1
# Let application to parse the inference tensor output
output-tensor-meta=1
tensor-meta-pool-size=8