I’m trying to build a face-detection-with-landmark based on SCRFD model with deepstream-triton. The pipeline of ensemble models is:
Preprocess (Python: resize, normalize, transpose) → Infer SCRFD (TensorRT with custom NMS plugins) → Post-process (Python: rescale, crop face, normalize) → Face Embedding
The ensemble model work well with triton-inference-server, however, when run with deepstream-triton 6.0, i got this error: Assertion `buf->getTotalBytes() >= bytes’. I’ve found that the error raised when i tried to return dynamic shape tensors by python backend in Post-process stage.
Config Post-process
name: “post_scrfd_nms”
backend: “python”
max_batch_size : 1input [
{
name: “num_detections”
data_type: TYPE_INT32
dims: [1]
},
{
name: “nmsed_boxes”
data_type: TYPE_FP32
dims: [200, 4]
},
{
name: “nmsed_scores”
data_type: TYPE_FP32
dims: [200]
},
{
name: “nmsed_classes”
data_type: TYPE_FP32
dims: [200]
},
{
name: “nmsed_landmarks”
data_type: TYPE_FP32
dims: [200, 10]
},
{
name: “original_image”
data_type: TYPE_FP32
dims: [3, 1080, 1920]
}]
output [
{
name: “res_num_detections”
data_type: TYPE_INT32
dims: [1]
},
{
name: “res_bboxes”
data_type: TYPE_FP32
dims: [-1, 4]
},
{
name: “res_scores”
data_type: TYPE_FP32
dims: [-1]
},
{
name: “res_landmarks”
data_type: TYPE_FP32
dims: [-1, 5, 2]
},
{
name: “res_face_align”
data_type: TYPE_FP32
dims: [-1, 3, 112, 112]
}
]parameters: {
key: “EXECUTION_ENV_PATH”,
value: {string_value: “/deepstream/triton/envs/face_align_38.tar.gz”}
}
Setup:
• GPU: NVIDIA A100
• Container: nvcr.io/nvidia/deepstream:6.0-triton
Note: The batchedNMSCustomPlugin is built by add a landmarks array to original batchedNMSPlugin
Git: GitHub - NNDam/deepstream-face-recognition: Face detection -> alignment -> feature extraction with deepstream
