Training PoseClassificationNet Model

Accelerated Computing Intelligent Video Analytics TAO Toolkit
, , , , ,
1

Hi Team, iam currently working on training of PoseClassifciationNet Model however i extracted the following .json file from BodyPose3d Model:
bodypose_3dbp.zip (1.8 MB)
I have a few questions regarding training of the PoseClassificationNet model as the official documentation says the spec file with have .npy as dataset path and .pkl as label file.
1- I wanted to ask what if there are multiple people and multiple poses in a single video file,how we will be able to create label file for that? As the documentation says the label file will be a list of lists,so how we will pass .json file along with the IDs for label file, so how we will deal when multiple people with multiple poses in same video file occur?

2

The input data for training or inference are formatted as a NumPy array in five dimensions (N, C, T, V, M):

  1. N indicates the number of sequences.
  2. C stands for the number of input channels, which is set as 3 in this example.
  3. T represents the maximum sequence length in frames that is 300 (10 seconds for 30 FPS) in our case.
  4. V defines the number of joint points, set as 34 for the NVIDIA format.
  5. M means the number of persons. Our pre-trained model assumes a single object but it can also support multiple people.

More, you can run with notebook tao_tutorials/notebooks/tao_launcher_starter_kit/pose_classification_net/poseclassificationnet.ipynb at main · NVIDIA/tao_tutorials · GitHub.

We provided an end-to-end inference script based on the Triton server: https://docs.nvidia.com/tao/tao-toolkit/text/pose_classification/pose_classification.html#deploying-the-model

3

Thank you for your response however my question is again that How we will generate .pkl file if there are multiple people along with multiple poses of them.Since the .pkl label file needs to be generated manually so i want to ask how we will deal with multiple people with multiple poses.

4

Please refer to notebook tao_tutorials/notebooks/tao_launcher_starter_kit/pose_classification_net/poseclassificationnet.ipynb at main · NVIDIA/tao_tutorials · GitHub. It mentions how to generate .pkl file.