BodyPoseNet - Animal Pose detection

We are looking to train a pose estimation model for cattle. Could you please point us in the right direction re below:

  1. BodyPoseNet using Tao Toolkit description suggests that it could be used for animals. If you could please share noteworthy points for us to consider if we are to use it for cattle body training.

  2. we are going to use it with deepstream. If we use bottom-up approach, can this model be used as a pgie model with some other sgie models with it? Or should we use our existing yolo model as a pgie and use pose detection as sgie?

  3. keeping in mind derpstream is a must, would you suggest any ready availabyalternatives like mmpose? Can that be fully integrated with deepstream?


BodyPose network is not verified on any animal dataset. Not sure if it works. Do you prepare training dataset which contains animal keypoints?
The BodyPosenet can output the 18 body parts. May I know more detail about the sgie model? I
It is possible to use pose detection as sgie.
The available app is deepstream_tao_apps/apps/tao_others/deepstream-bodypose2d-app at master · NVIDIA-AI-IOT/deepstream_tao_apps · GitHub

Thanks Morganh.

We are planning to use keypoint analysis so that we can detect animal’s eye and nose. This way, the animal is detected and then eyes and nose are available as well. We need to run further image classification on eye and nose of an animal and so keypoint analysis seemed fit. However, we are 100% open to suggestions if you think we should be using another model rather than bodypose.

For pose estimation, we are planning to prepare a custom dataset with key points and it is around 5k images. The 18 key points will be more than sufficient for us. Our aim is to detect mainly ears, eyes, nose, horns and if possible also the legs. At the initial stage, we are looking to train and use this only on cattle.

Since the originally trained weights are on humans, can we still use this as a base weight for our model prepared with a custom keypoint dataset?

Thank you for confirming the ability to use bodypose2d as a sgie model. So, is it right to assume that it cannot be used as pgie? If it can be used as a pgie then we can use it for detection + body pose. In that case, we will train the model with bottom-up approach. Otherwise, we are planning to use a Yolo or Mask R Cnn as a pgie.

Btw, please let us know if any alternative suggestion for the animal pose estimation which works with Deepstream.

Also, is it possible to send some project-specific details in a private message? Some client requirement specific details we are not able to share on a public domain.

There will be fpenet-generic for next tao release. It will detect key points.

For dataset, please prepare left “keypoints” and also right “keypoints” . This is similar to bodypose.

The available pretrained model is trained on a subset of the Google OpenImages dataset. Primary use case for this model is to detect human poses in a given RGB image. BodyPoseNet is commonly used for activity/gesture recognition, fall detection, posture analysis etc. Not sure if it can work well in an animal keypoint dataset.

If BodyPosenet works as pgie, the output are 18 body parts. How will you feed these 18 body parts to sgie?

Fpenet-generic is an option.

Previously, do you or your company have other bridge?

Hi Morganh,

Thank you for the detailed reply. I get your point now. Based on the clarification you provided it looks like we may need to use back-to-back detection instead of using the posenet. Btw, we don’t have any other bridge. For any other comms, what is the best way to send info without making it public?

If you have prepared dataset animal dataset which has left “keypoints” and also right “keypoints” , you can try to train with bodyposenet directly.

You can send me a private message for more details. I will sync with internal team to check if we can help you further.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.