Replacement of Bodypose2d(17) joint points with 3dBodypose(34)

Accelerated Computing Intelligent Video Analytics TAO Toolkit
, ,
1

Hi there,iam currently working on integration of bodypose2d model with PoseClassificationNet model and iam also utilizing- Tao convert_dataset command to convert my bodypose2d.json(17 joints) to numpy arrays but i want to ask that- Can i replace joints with any of 34 points? like in my 17 points of bodypose2d.json can i replace nose with pelvis from 34 points? keeping the total number of joints to be 17?
Please let me know.

3

You can refer to tao_pytorch_backend/nvidia_tao_pytorch/cv/pose_classification/model/st_gcn.py at 9c2d94c0635b1117edfea85a94a6e3d0ead53754 · NVIDIA/tao_pytorch_backend · GitHub and modify it to meet your key points.

Different keypoints are expected to work. There are some dataset mentioned in tao_pytorch_backend/nvidia_tao_pytorch/cv/pose_classification/model/st_gcn.py at 9c2d94c0635b1117edfea85a94a6e3d0ead53754 · NVIDIA/tao_pytorch_backend · GitHub. They have different keypoints.

4

Thanks but this doesn’t answers my question,however let me ask another thing,Iam actually trying to integrate bodypose2d model with PoseClassifcaitionNet model so now tell me will i be able to train PoseClassificationNet model from the output out Boydpose2d model .json file?Since that model basically is trained on COCO 17 points so ofcourse it will output .json file of 17 points only so tell me Can i use that 17 points .json file from bodypose2d for finetuning of PoseCLassifcationNet model?Lastly also tell me that can i use any of 17 points for training of PoseClassificationNet model?

5

Thanks but this doesn’t answers my question,however let me ask another thing,Iam actually trying to integrate bodypose2d model with PoseClassifcaitionNet model so now tell me will i be able to train PoseClassificationNet model from the output out Boydpose2d model .json file?Since that model basically is trained on COCO 17 points so ofcourse it will output .json file of 17 points only so tell me Can i use that 17 points .json file from bodypose2d for finetuning of PoseCLassifcationNet model?Lastly also tell me that can i use any of 17 points for training of PoseClassificationNet model?

6

Yes, you can train PoseCLassifcationNet with these 2d points. But the accuracy may not be as good as 3D keypoints as there will be more ambiguity in actions for 2D keypoints.

8

also tell me that can i use any 17 points of my own choice for training of PoseClassificationNet?Can i replace all 17 points of coco keypoints with any of my own points?

9

The 17 keypoints in the COCO dataset correspond to the following body parts:

  1. Nose
  2. Left Eye
  3. Right Eye
  4. Left Ear
  5. Right Ear
  6. Left Shoulder
  7. Right Shoulder
  8. Left Elbow
  9. Right Elbow
  10. Left Wrist
  11. Right Wrist
  12. Left Hip
  13. Right Hip
  14. Left Knee
  15. Right Knee
  16. Left Ankle
  17. Right Ankle

This structure allows for detailed representation of human poses in 2D space.

You can replace the key points but the model will need to be re-trained.