Not enough documentation about bodypose3dnet for retraining and what hardware is supported by this model

Accelerated Computing Intelligent Video Analytics TAO Toolkit
,
1

Hello

There is a 3d human pose estimator (HPE) that supported by deepstream

and TAO pretrained models for this 3d HPE

Question 1:

How to use bodypose3dnet for training/retraining using custom datasets and what exactly should be format, configuration and procedure in general ?

For instance, what are represent tensors 1, 2, 3 and 4
k_inv
t_form_inv
scale_normalized_mean_limb_lengths
mean_limb_lengths

For instance, for bodyposenet (2d) there’s much more documentation on how to work with it, e.g. prepare training training, correct format, etc.
GPU-optimized AI, Machine Learning, & HPC Software | NVIDIA NGC | NVIDIA NGC**
https://docs.nvidia.com/tao/tao-toolkit/text/bodypose_estimation/bodyposenet.html
Training and Optimizing a 2D Pose Estimation Model with NVIDIA TAO Toolkit, Part 1 | NVIDIA Technical Blog
Training and Optimizing a 2D Pose Estimation Model with NVIDIA TAO Toolkit, Part 2 | NVIDIA Technical Blog

Question 2:
What are the minimum hardware requirements for inference using deepstream_reference_apps/deepstream-bodypose-3d at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub and 2 available models on NGC
bodypose3dnet_performance.etlt
bodypose3dnet_accuracy.etlt
I assume that it should work on Jetson family devices since this repository is located in
GitHub - NVIDIA-AI-IOT/deepstream_reference_apps: Samples for TensorRT/Deepstream for Tesla & Jetson · GitHub
and it is mentioned in description of the repository “Samples for TensorRT/Deepstream for Tesla & Jetson”,
but I am not sure which ones are supported. I am mostly interested in Jetson Xavier NX and Orin NX or possibly even lower end device.

Thank you

Please provide the following information when requesting support.

• Hardware V100 & Xavier NX
• Network Type (bodypose3dnet GPU-optimized AI, Machine Learning, & HPC Software | NVIDIA NGC | NVIDIA NGC)

2

@Morganh possibly you could help me with that, please?

4

Hi,
In BodyPose3DNet | NVIDIA NGC , only deployable files are available. There is no trainable files. Currently, TAO user guide does not contain bodypose3dnet. So users can only run inference with the .etlt models. Retraining is not supported. Usually the retraining needs .tlt model.
The application can run on dgpu devices or Jetson devices.

5

@Morganh thank you for your clarification. 2 things are still unclear

  1. What Jetson Devices are supported ? For instance, Jetson Nano 4GB, Xavier NX (that what we have at the moment)?
  2. What JetPack should be used to run it deepstream-bodypose-3d with bodypose3dnet ? Currently we have in production Jetson Xavier NX with JetPack 4.4 PR (Production Release).

Thanks

6
  1. All the kinds of Jetson Devices.
  2. Refer to deepstream_reference_apps/deepstream-bodypose-3d at master · NVIDIA-AI-IOT/deepstream_reference_apps · GitHub and Quickstart Guide — DeepStream 6.1.1 Release documentation
8

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks