Any complete installation guide for "deepstream_pose_estimation"?

[AK51]

Hi,

I follow this link and I can get the deep stream working, but I stuck at the trt-pose, export fail. (Not json file something…)

GitHub - NVIDIA-AI-IOT/deepstream_pose_estimation: This is a sample...

nvidia@nvidia-desktop:~/trt_pose/trt_pose/utils$ sudo python export_for_isaac.py --input_checkpoint resnet18_baseline_att_224x224_A_epoch_249.pth
[sudo] password for nvidia: 
Traceback (most recent call last):
  File "export_for_isaac.py", line 56, in <module>
    import torch                    # Please refer to: https://pytorch.org/get-started/locally/
ImportError: No module named torch
nvidia@nvidia-desktop:~/trt_pose/trt_pose/utils$ sudo python3 export_for_isaac.py --input_checkpoint resnet18_baseline_att_224x224_A_epoch_249.pth
Input model is not specified, using resnet18_baseline_att as a default.
Input width/height are not specified, using 224x224 as a default.
Output path is not specified, using resnet18_baseline_att_224x224_A_epoch_249.onnx as a default.
Input topology human_pose.json is not a valid (.json) file.
nvidia@nvidia-desktop:~/trt_pose/trt_pose/utils$ 

Is there any tutorial or complete guide of installation?

I saw it can be done, Deepstream_pose_estimation
But I guess may be some library has changed…
Shall I use deepstream 6.0?

Thanks

[AastaLLL]

Hi,

ImportError: No module named torch

Sorry that the example also requires PyTorch library.
You can install it with the information below:

PyTorch for Jetson

Below are pre-built PyTorch pip wheel installers for Jetson Nano, TX1/TX2, Xavier, and Orin with JetPack 4.2 and newer. Download one of the PyTorch binaries from below for your version of JetPack, and see the installation instructions to run on your Jetson. These pip wheels are built for ARM aarch64 architecture, so run these commands on your Jetson (not on a host PC). You can also use the pre-built l4t-pytorch and l4t-ml container images and Dockerfiles. PyTorch pip wheels JetPack 5

PyTo…

Thanks.

[AK51]

I have posted my questions in here, Usb cam? And deepstream pose estimation last part not working

I have read this post, but they can run it, but I cannot, is there any library version issue?

Deepstream pose estimation model: getting error while running

Machine setup below: **• Hardware Platform ( GPU) - GTX 1650 **• DeepStream Version- 5.0.1 **• TensorRT Version: 7.2.1 **• NVIDIA GPU Driver Version (valid for GPU only): 455.45 After creating ONNX file, and running make, When i run the command like below arvind@arvind:/opt/nvidia/deepstream/…/deepstream_pose_estimation$ sudo ./deepstream-pose-estimation-app ./video.mp4 ./ One element could not be created. Exiting. I am getting this error, I replaces the nvosd.so file in lib too. need su…

vnvidia@nvidia-desktop:~/deepstream_pose_estimation$ sudo ./deepstream-pose-estimation-app video.mp4 .
One element could not be created. Exiting.

Here is what I did for a month of trial and error. >_<

Following this link
https://github.com/NVIDIA-AI-IOT/deepstream_pose_estimation#:~:text=Human%20pose%20estimation%20is%20the,a%20video%20or%20a%20photo.

1. Install DeepStream on your platform, verify it is working by running deepstream-app.
Download DeepStream 5.0to my Downloads folder
https://developer.nvidia.com/assets/Deepstream/5.0/ga/secure/deepstream_sdk_5.0_jetson.tbz2

Follow the setup guide "Install Jetson SDK compenents"
https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_Quickstart.html#jetson-setup

nvidia@nvidia-desktop:~$ sudo apt install \
> libssl1.0.0 \
> libgstreamer1.0-0 \
> gstreamer1.0-tools \
> gstreamer1.0-plugins-good \
> gstreamer1.0-plugins-bad \
> gstreamer1.0-plugins-ugly \
> gstreamer1.0-libav \
> libgstrtspserver-1.0-0 \
> libjansson4=2.11-1
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package libgstrtspserver-1.0-0
E: Couldn't find any package by glob 'libgstrtspserver-1.0-0'
E: Couldn't find any package by regex 'libgstrtspserver-1.0-0'
nvidia@nvidia-desktop:~$ 

	Follow this link, https://zoomadmin.com/HowToInstall/UbuntuPackage/libgstrtspserver-1.0-0
	sudo apt-get update -y
	sudo apt-get install -y libgstrtspserver-1.0-0

nvidia@nvidia-desktop:~$ sudo apt install \
> libssl1.0.0 \
> libgstreamer1.0-0 \
> gstreamer1.0-tools \
> gstreamer1.0-plugins-good \
> gstreamer1.0-plugins-bad \
> gstreamer1.0-plugins-ugly \
> gstreamer1.0-libav \
> libgstrtspserver-1.0-0 \
> libjansson4=2.11-1
git clone https://github.com/edenhill/librdkafka.git
cd librdkafka
git reset --hard 7101c2310341ab3f4675fc565f64f0967e135a6a
./configure
make
sudo make install
#need to modify for 5.0
sudo mkdir -p /opt/nvidia/deepstream/deepstream-5.0/lib
sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream-5.0/lib

sudo vi /etc/apt/sources.list.d/nvidia-l4t-apt-source.list
Modify the code to 
	deb https://repo.download.nvidia.com/jetson/common r32.6 main
	deb https://repo.download.nvidia.com/jetson/t194 r32.6 main

sudo apt update
sudo apt install --reinstall nvidia-l4t-gstreamer
sudo apt install --reinstall nvidia-l4t-multimedia
sudo apt install --reinstall nvidia-l4t-core
cd
cd Downloads
sudo tar -xvf deepstream_sdk_5.0_jetson.tbz2 -C /
cd /opt/nvidia/deepstream/deepstream-5.0
sudo ./install.sh
sudo ldconfig
cd /opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps
sudo git clone https://github.com/NVIDIA-AI-IOT/deepstream_pose_estimation.git
cd
#########################################
#Goto trtpose
#Install pytorch and torchyvision
#Follow: https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-10-now-available/72048
wget https://nvidia.box.com/shared/static/p57jwntv436lfrd78inwl7iml6p13fzh.whl -O torch-1.8.0-cp36-cp36m-linux_aarch64.whl
sudo apt-get install python3-pip libopenblas-base libopenmpi-dev 
pip3 install Cython
pip3 install numpy torch-1.8.0-cp36-cp36m-linux_aarch64.whl
##
##torchvision
sudo apt-get install libjpeg-dev zlib1g-dev libpython3-dev libavcodec-dev libavformat-dev libswscale-dev
git clone --branch v0.9.0 https://github.com/pytorch/vision torchvision
cd torchvision
export BUILD_VERSION=0.9.0
python3 setup.py install --user
cd ../
pip install 'pillow<7'
#####################################
# pip not found
apt-get install python-pip
##################################
pip install 'pillow<7'

##########################################################
git clone https://github.com/NVIDIA-AI-IOT/torch2trt
cd torch2trt
sudo python3 setup.py install --plugins
#########################################################

###################################################
#ModuleNotFoundError: No module named 'packaging'
#from this link, https://stackoverflow.com/questions/42222096/no-module-named-packaging
pip3 install --upgrade pip
pip3 install packaging
#####################################################

sudo python3 setup.py install --plugins

sudo pip3 install tqdm cython pycocotools
sudo apt-get install python3-matplotlib


cd
git clone https://github.com/NVIDIA-AI-IOT/trt_pose
cd trt_pose
sudo python3 setup.py install

#Download the two model
https://drive.google.com/file/d/1XYDdCUdiF2xxx4rznmLb62SdOUZuoNbd/view?usp=drive_open
https://drive.google.com/file/d/13FkJkx7evQ1WwP54UmdiDXWyFMY1OxDU/view?usp=drive_open
#copy these two models to Place the downloaded weights in the tasks/human_pose directory
#copy these two models to Place the downloaded weights in the ~trt_pose/trt_pose/utils directory
cd
cd trt_pose/trt_pose/utils
#error
# follow this link
https://forums.developer.nvidia.com/t/conversion-of-model-weights-for-human-pose-estimation-model-to-onnx-results-in-nonsensical-pose-estimation/164417/11
#freeze
#need to increase the swap memory
python3 export_for_isaac.py --input_checkpoint resnet18_baseline_att_224x224_A_epoch_249.pth --input_topology ../../tasks/human_pose/human_pose.json

cd
git clone https://github.com/NVIDIA-AI-IOT/deepstream_pose_estimation.git
#copy the onnx file to deepstream_pose_estimation
#Replace the OSD binaries (x86 or Jetson) in $DEEPSTREAM_DIR/libs with the ones provided in this #repository under bin/. Please note that these are not inter-compatible across platforms.

nvidia@nvidia-desktop:~/deepstream_pose_estimation/bin$ ls
Jetson  x86
nvidia@nvidia-desktop:~/deepstream_pose_estimation/bin$ cd Jetson/
nvidia@nvidia-desktop:~/deepstream_pose_estimation/bin/Jetson$ ls
libnvds_osd.so
nvidia@nvidia-desktop:~/deepstream_pose_estimation/bin/Jetson$ sudo cp libnvds_osd.so /opt/nvidia/deepstream/deepstream-5.0/lib
[sudo] password for nvidia: 
nvidia@nvidia-desktop:~/deepstream_pose_estimation/bin/Jetson$ 

nvidia@nvidia-desktop:~/deepstream_pose_estimation$ sudo make
Package json-glib-1.0 was not found in the pkg-config search path.
Perhaps you should add the directory containing `json-glib-1.0.pc'
to the PKG_CONFIG_PATH environment variable
No package 'json-glib-1.0' found
g++ -c -o deepstream_pose_estimation_app.o -DPLATFORM_TEGRA -I../../apps-common/includes -I../../../includes -I../deepstream-app/ -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=5  deepstream_pose_estimation_app.cpp
In file included from deepstream_pose_estimation_app.cpp:4:0:
post_process.cpp:8:10: fatal error: gst/gst.h: No such file or directory
 #include <gst/gst.h>
          ^~~~~~~~~~~
compilation terminated.
Makefile:44: recipe for target 'deepstream_pose_estimation_app.o' failed
make: *** [deepstream_pose_estimation_app.o] Error 1
nvidia@nvidia-desktop:~/deepstream_pose_estimation$ 
sudo apt-get install libgstreamer1.0-dev
sudo apt install libjson-glib-dev

nvidia@nvidia-desktop:~/deepstream_pose_estimation$ sudo make
g++ -c -o deepstream_pose_estimation_app.o -DPLATFORM_TEGRA -I../../apps-common/includes -I../../../includes -I../deepstream-app/ -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=5 -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/orc-0.4 -I/usr/include/gstreamer-1.0 -I/usr/include/json-glib-1.0 -I/usr/include/glib-2.0 -I/usr/lib/aarch64-linux-gnu/glib-2.0/include deepstream_pose_estimation_app.cpp
In file included from deepstream_pose_estimation_app.cpp:4:0:
post_process.cpp:12:10: fatal error: gstnvdsmeta.h: No such file or directory
 #include "gstnvdsmeta.h"
          ^~~~~~~~~~~~~~~
compilation terminated.
Makefile:44: recipe for target 'deepstream_pose_estimation_app.o' failed
make: *** [deepstream_pose_estimation_app.o] Error 1

#copy all files in /opt/nvidia/deepstream/deepstream-5.1/sources/includes/ to deepstream_pose_estimation
sudo make
#download a crowd video from https://www.pexels.com/search/videos/crowd/
$copy the video.mp4 to :~/deepstream_pose_estimation

sudo ./deepstream-pose-estimation-app test.mp4 
nvidia@nvidia-desktop:~/deepstream_pose_estimation$ sudo ./deepstream-pose-estimation-app test.mp4 .
One element could not be created. Exiting.
nvidia@nvidia-desktop:~/deepstream_pose_estimation$ sudo ./deepstream-pose-estimation-app test.mp4 Pose_Estimation.mp4
One element could not be created. Exiting.
nvidia@nvidia-desktop:~/deepstream_pose_estimation$ 

#edit deepstream_pose_estimation_config.txt
[property]
gpu-id=0
net-scale-factor=0.0174292
offsets=123.675;116.28;103.53
onnx-file=resnet18_baseline_att_224x224_A_epoch_249.onnx
labelfile-path=labels.txt
batch-size=1
process-mode=1
model-color-format=0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=4
interval=0
gie-unique-id=1
model-engine-file=pose_estimation.onnx_b1_gpu0_fp16.engine
network-type=100
workspace-size=3000

[AastaLLL]

Hi,

Do you get it to work with the change shared in the below comment?

Deepstream_pose_estimation - fatal error: gstnvdsmeta.h: No such file or directory

Hi, It requires some update for the JetPack4.6+Deepstream6.0 environment. Please apply the below change and try it again: diff --git a/Makefile b/Makefile index ae2c316..ca64725 100644 --- a/Makefile +++ b/Makefile @@ -9,7 +9,7 @@ APP:= deepstream-pose-estimation-app TARGET_DEVICE = $(shell gcc -dumpmachine | cut -f1 -d -) -NVDS_VERSION:=5.0 +NVDS_VERSION:=6.0 LIB_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/lib/ APP_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(…

Thanks.

[AK51]

I use Deepstream 5.0, I tried deepstream6.0 before but it has error.
For my way, I just copy all header file to deepstream_pose_estimation
Up to now, I stuck at the very last step.

nvidia@nvidia-desktop:~/deepstream_pose_estimation$ ./deepstream-pose-estimation-app test.mp4 .
One element could not be created. Exiting.

Should do the whole thing again using deepstream6.0. >_<
I don’t mind as soon as there is a clear instruction.
I have been working on this for 2 weeks already…
Thx

nvidia@nvidia-desktop:~/deepstream_pose_estimation$ ls
bin
CLA_LICENSE.md
cover_table.hpp
deepstream-pose-estimation-app
deepstream_pose_estimation_app.cpp
deepstream_pose_estimation_app.o
deepstream_pose_estimation_config.bak
deepstream_pose_estimation_config.txt
gstnvdsbufferpool.h
gstnvdsinfer.h
gstnvdsmeta.h
gst-nvdssr.h
gst-nvevent.h
gst-nvmessage.h
gst-nvquery.h
images
LICENSE.md
Makefile
munkres_algorithm.cpp
nvbufsurface.h
nvbufsurftransform.h
nvds_analytics_meta.h
nvds_dewarper_meta.h
nvdsgstutils.h
nvdsinfer_context.h
nvdsinfer_custom_impl.h
nvdsinfer_dbscan.h
nvdsinfer.h
nvdsinferserver_config.proto
nvdsinferserver_plugin.proto
nvdsinfer_tlt.h
nvdsinfer_utils.h
nvds_latency_meta.h
nvds_logger.h
nvds_mask_utils.h
nvdsmeta.h
nvdsmeta_schema.h
nvds_msgapi.h
nvds_obj_encode.h
nvds_opticalflow_meta.h
nvdstracker.h
nvds_tracker_meta.h
nvds_version.h
nvll_osd_api.h
nvll_osd_struct.h
nvmsgbroker.h
pair_graph.hpp
pose_estimation.onnx
post_process.cpp
README.md
resnet18_baseline_att_224x224_A_epoch_249.onnx
test.mp4
video.mp4
nvidia@nvidia-desktop:~/deepstream_pose_estimation$ 

...
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream_pose_estimation$ sudo make
g++ -c -o deepstream_pose_estimation_app.o -DPLATFORM_TEGRA -I../../apps-common/includes -I../../../includes -I../deepstream-app/ -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=5 -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/orc-0.4 -I/usr/include/gstreamer-1.0 -I/usr/include/json-glib-1.0 -I/usr/include/glib-2.0 -I/usr/lib/aarch64-linux-gnu/glib-2.0/include deepstream_pose_estimation_app.cpp
deepstream_pose_estimation_app.cpp: In function ‘GstPadProbeReturn osd_sink_pad_buffer_probe(GstPad*, GstPadProbeInfo*, gpointer)’:
deepstream_pose_estimation_app.cpp:231:77: warning: zero-length gnu_printf format string [-Wformat-zero-length]
     offset = snprintf(txt_params->display_text + offset, MAX_DISPLAY_LEN, "");
                                                                             ^
deepstream_pose_estimation_app.cpp:236:41: warning: ISO C++ forbids converting a string constant to ‘char*’ [-Wwrite-strings]
     txt_params->font_params.font_name = "Mono";
                                         ^~~~~~
g++ -o deepstream-pose-estimation-app deepstream_pose_estimation_app.o -L/opt/nvidia/deepstream/deepstream-5.0/lib/ -lnvdsgst_meta -lnvds_meta -lnvds_utils -lm -lpthread -ldl -Wl,-rpath,/opt/nvidia/deepstream/deepstream-5.0/lib/ -lgstvideo-1.0 -lgstbase-1.0 -lgstreamer-1.0 -lX11 -ljson-glib-1.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0
nvidia@nvidia-desktop:/opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream_pose_estimation$ sudo ./deepstream-pose-estimation-app test.mp4 .
One element could not be created. Exiting.

[AastaLLL]

Hi,

We have verified the sample with Deepstream 6.0.
Please check the below topic for the following comment:

Deepstream_pose_estimation - fatal error: gstnvdsmeta.h: No such file or directory

Hi, I am trying to get the pose-estimation works for the last 2 weeks… I used deepstream-6.0 instead of 5.0… will it cause problem? I have install these also, it does not help 87 sudo apt-get install libgstreamer1.0-dev 88 sudo apt-get install libgstreamer1.0 91 sudo apt install libjson-glib-dev 92 sudo apt-get install libgstreamer-plugins-base1.0-dev nvidia@nvidia-desktop:~/deepstream_pose_estimation$ sudo make g++ -c -o deepstream_pose_estimation_app.o -DPLATFORM_TEGRA -I../../apps…

Thanks.