How should I write a config file when configuring PoseClassificationNet with nvinfer?
I’ve tried, but it says BatchDims which doesn’t support it.
Here’s the process I tried:
I’m using the deepstream-6.2-devel container in a x86_64/Ubuntu 20.04 environment.
-
I imported a deepstream_reference_apps containing the bodypose3d app into git clone.
-
I copied the deepstream-bodypose-3d directory to deepstream-bodypose-3d-classification (cp -r).
-
Without modifying the code, I built and ran with make from the sources directory to see nothing wrong.
-
I downloaded the PoseClassification deployable version from NGC
-
I downloaded tao-converter from NGC as version v4.0.0_trt8.5.2.2_x86.
-
I converted ETLT file to engine file with tao-converter. The command used at this time is as follows.
tao-converter st-gcn_3dbp_nvidia.etlt \
-k nvidia_tao \
-d 3,300,34,1 \
-p input,1x3x300x34x1,4x3x300x34x1,16x3x300x34x1 \
-o fc_pred \
-t fp16 \
-m 16 \
-e st-gcn_3dbp_nvidia.etlt_b16_gpu0_fp16.engine
- deepstream_pose_estimation_app.cpp modified the code to add a second SGIE to the pipeline.
GstElement* sgie2 = gst_element_factory_make("nvinfer", "secondary-nvinference-engine2");
if (!sgie2) {
g_printerr ("Secondary nvinfer could not be created. Exiting.\n");
return -1;
}
//---Set sgie2 properties---
/* Configure the nvinfer element using the nvinfer config file. */
g_object_set(G_OBJECT(sgie2),
"output-tensor-meta", TRUE,
"input-tensor-meta", TRUE,
"config-file-path", SGIE2_CONFIG_FILE,
NULL);
/* Override the batch-size set in the config file with the number of sources. */
guint sgie2_batch_size = 0;
g_object_get(G_OBJECT(sgie2), "batch-size", &sgie2_batch_size, NULL);
if (sgie2_batch_size < num_sources) {
g_printerr
("WARNING: Overriding infer-config batch-size (%d) with number of sources (%d)\n",
sgie2_batch_size, num_sources);
g_object_set(G_OBJECT(sgie2), "batch-size", num_sources, NULL);
}
gst_bin_add_many(GST_BIN(pipeline),
nvvideoconvert_enlarge, capsFilter_enlarge,
pgie, tracker, sgie, sgie2, tee,
queue_nvvidconv, nvvidconv, nvosd, filesink, nvdslogger,
nvvideoconvert_reduce, capsFilter_reduce, NULL);
// Link elements
if (!gst_element_link_many(streammux_pgie,
nvvideoconvert_enlarge, capsFilter_enlarge,
pgie, tracker, sgie, sgie2, nvdslogger, tee, NULL))
{
g_printerr ("Elements could not be linked. Exiting.\n");
return -1;
}
- The config file for the second SGIE is written as follows.
[property]
gpu-id=0
net-scale-factor=1.0
## Accuracy mode: _mode0_; Performance mode: _mode1_
tlt-model-key=nvidia_tao
tlt-encoded-model=../models/poseclassficiationnet_vdeployable_v1.0/st-gcn_3dbp_nvidia.etlt
model-engine-file=../models/poseclassficiationnet_vdeployable_v1.0/st-gcn_3dbp_nvidia.etlt_b16_gpu0_fp16.engine
labelfile-path=../models/poseclassficiationnet_vdeployable_v1.0/labels.txt
infer-dims=3;300;34
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
## 0=Detection 1=Classifier 2=Segmentation 100=other
network-type=100
num-detected-classes=6
interval=0
gie-unique-id=1
output-blob-names=fc_pred
classifier-threshold=0.7
operate-on-class-ids=0
## Integer 0:NCHW 1:NHWC
network-input-order=0
# Enable tensor metadata output
output-tensor-meta=1
## 1-Primary 2-Secondary
process-mode=2
## 0=RGB 1=BGR 2=GRAY
model-color-format=1
maintain-aspect-ratio=0
symmetric-padding=0
scaling-filter=1
- First of all, at this stage, we checked to see if the model is loaded. I succeeded in building using make, but got the below error on execution.
root@42d3a8fd48e6:/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/custom/deepstream_reference_apps/deepstream-bodypose-3d-classification/sources# ./deepstream-pose-estimation-app --input rtsp://<MY_RTSP_SOURCE> --fps --output rtsp://
*** nv-filesink: Launched RTSP Streaming at rtsp://localhost:8554/ds-test ***
Now playing: rtsp://<MY_RTSP_SOURCE>
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
WARNING: [TRT]: CUDA lazy loading is not enabled. Enabling it can significantly reduce device memory usage. See `CUDA_MODULE_LOADING` in https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#env-vars
0:00:03.289958847 2226 0x564d7abc0b00 INFO nvinfer gstnvinfer.cpp:680:gst_nvinfer_logger:<secondary-nvinference-engine2> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1909> [UID = 1]: deserialized trt engine from :/opt/nvidia/deepstream/deepstream-6.2/sources/apps/sample_apps/custom/deepstream_reference_apps/deepstream-bodypose-3d-classification/models/poseclassficiationnet_vdeployable_v1.0/st-gcn_3dbp_nvidia.etlt_b16_gpu0_fp16.engine
INFO: ../nvdsinfer/nvdsinfer_model_builder.cpp:610 [FullDims Engine Info]: layers num: 2
0 INPUT kFLOAT input 3x300x34x1 min: 1x3x300x34x1 opt: 4x3x300x34x1 Max: 16x3x300x34x1
1 OUTPUT kFLOAT fc_pred 6 min: 0 opt: 0 Max: 0
deepstream-pose-estimation-app: nvdsinfer_backend.cpp:135: virtual bool nvdsinfer::TrtBackendContext::canSupportBatchDims(int, const NvDsInferBatchDims&): Assertion `m_AllLayers[bindingIdx].inferDims.numDims == batchDims.dims.numDims' failed.
Aborted (core dumped)
What did I miss?
