Deepstream pose estimation: How can I create custom pose parse function?

Hi guys,
I’m trying to create pose parse function like in /opt/nvidia/deepstream/deepstream-5.0/sources/objectDetector_SSD. I see just have sample for object detection and classification. I’m newbie, can anybody help me?
Thanks so much!

Hey, pls share your setup with us.

Yeah, currently DS only support to customzing post processor for detection and classification and isntance segmentaiton model.

I just cloned your deepstream pose estimation and converted the model in pytorch → onnx → tensorrt engine.
Your application is really well but need to improve accuracy. Then I created parse_keypoint.cpp contain NvDsInferParseKeypoints function

bool NvDsInferParseKeypoints(
NvDsInferTensorMeta *tensor_meta,
float threshold,
float link_threshold,
Vec1D <NvDsInferKeypointDetectionInfo &keypointList)

NvDsInferKeypointDetectionInfo is declared in nvdsinfer.h

typedef struct{
float x, y;
} Point;

typedef struct{
std::vector keypoint;
int numOfPoints;
} NvDsInferKeypointDetectionInfo;

Then I created make file and make. It haven’t caused error.
Now I’m gonna test the .so shared lib.
I want to know whether there is a standard form to make custom parse output and whether I can get the output via NvDsInferTensorMeta (I see that all sample get the output layer via NvDsInferLayerInfo)?
**Update: I try using deepstream-app default by command
deepstream-app -c config_file_name but it throw error like this:

Thanks so much!

What accuracy issue are you observing?
For pose model, you only can do the post process via a gstreamer probe .

I got about 40% mAPs (IoU 0.5) in COCO with resnet model and it run not very well with my dataset. How can I retrain it or train a new custom model (with other backbone)?

For how to retrain a model, I think it’s not a deepstream question, maybe you can create a new topic in TLT forum to ask help.

1 Like

Thank you very much!