Cloud_native_demo on JP4.4.1

Andrey1984 · January 12, 2021, 11:03pm

the command is abrupt ^^
the corrected command is below:

sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/jetson-pose:r32.4.2

>>> import torch
>>> import torch2trt
>>> import trt_pose.models
>>> import tensorrt as trt
>>> MODEL = trt_pose.models.densenet121_baseline_att
>>> model = MODEL(18, 42).cuda().eval()
>>> model.load_state_dict(torch.load('/pose/generated/densenet121_baseline_att.pth'))
>>> data = torch.randn((1, 3, 224, 224)).cuda().float()
>>> model_trt = torch2trt.torch2trt(model, [data], fp16_mode=True, max_workspace_size=1 << 25, log_level=trt.Logger.VERBOSE)
>>> torch.save(model_trt.state_dict(), '/pose/generated/densenet121_baseline_att_trt.pth')
>>> exit()

python3 run_pose_pipeline.py /videos/pose_video.mp4

The sequence above allows to recover the pose component to process the video with it, but how to get it running with CSI Raspberry v2 camera?
Also how to patch other containers but the Deepstream & pose container from the cloud native set of four containers?

trying for csi camera

export DISPLAY=:0
xhost +
 docker run -it --rm --privileged --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix  -v /tmp/argus_socket:/tmp/argus_socket --cap-add SYS_PTRACE  nvcr.io/nvidia/jetson-pose:r32.4.2

while the latter command launches the container with the access to cameras, the argument to load inputs from camera by the given script is unknown.
is it supported? like python3 run_pose_pipeline.py /dev/video0 or similar?

AastaLLL · January 13, 2021, 3:08am

Hi,

The sample uses OpenCV to enable the GStreamer pipeline.
So you can modify it for the CSI Raspberry v2 camera with nvarguscamerasrc.

$ sudo docker run -it --rm --net=host --runtime nvidia  -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix  nvcr.io/nvidia/jetson-pose:r32.4.2
> vim /utils/video.py

Thanks.

Andrey1984 · January 13, 2021, 3:34am

the resuling edition works
thanks to @Honey_Patouceul @AastaLLL

  class Video(object):

    def __init__(self, path, width, height, qsize=10, loop=True, codec='h264'):
        self.path = path
        self.width = width
        self.height = height
        self.qsize = qsize
        self.loop = loop
        self.codec = codec
        self.reset()

   def _gst_str(self):
        return 'nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1280, height=720, format=NV12, framerate=30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=BGRx, width={width}, height={height} ! videoconvert ! queue max-size-buffers={qsize} ! video/x-raw, format=BGR ! appsink sync=0'.format(width=self.width, height=self.height, qsize=self.qsize)

    def reset(self):
        if hasattr(self, 'cap'):
            del self.cap
        self.cap = cv2.VideoCapture(self._gst_str(), cv2.CAP_GSTREAMER)

    def read(self):
        re, img = self.cap.read()
        if re:
            return img
        elif self.loop:
            self.reset()
            re, img = self.cap.read()
            if re:
                return img
            else:
                return None
        else:
            return None

    def destroy(self):
        self.cap.release()

can be executed with

 python3 run_pose_pipeline.py 0

But @AastaLLL do we know how to patch the rest of containers - gaze & BERT?
Are there sources - Dockerfiles from that ngc containers been built ?

AastaLLL · January 14, 2021, 5:13am

Hi,

Thanks for the feedback!

BERT is for audio, not for the camera.
Gaze follows the same workflow to read the camera data.
So you can modify the /utils/video.py located in the nvcr.io/nvidia/jetson-gaze:r32.4.2.

The Dockerfile for these containers is not public.

Thanks.

Andrey1984 · January 14, 2021, 6:28am

@AastaLLL yes,
but the complication with BERT/gaze is that they won’t run as is from the repository GitHub - NVIDIA-AI-IOT/jetson-cloudnative-demo: Multi-container demo for Jetson Xavier NX and Jetson AGX Xavier on JP 4.4.1 because of the difference in versions of the container DP4.4 versus jetson OS 4.4.1 environment. That implies that TRT version mismatch prevents .engine to run at least for one of them, as it seems to me
for the pose we used the series of steps above to redo the .engine file, as it seems to me.
probably similar adjustment will be required for gaze also etc.

Step to reproduce the issue - run

./run_demo.sh

from the cloud repository
that fires up 4 containers, but as is of them only works deepstream container
running the gaze container from NGC:

docker run --runtime nvidia -it --rm --network host -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/jetson-gaze:r32.4.2 python3 run_gaze_sequential.py /videos/gaze_video.mp4 --loop --codec=h264

sys: x11 
coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
INVALID_STATE: std::exception
INVALID_CONFIG: Deserialize the cuda engine failed.
Assertion fail in file 'trtNet.cpp' line 146: _engine is null
terminate called after throwing an instance of 'std::exception'
  what():  std::exception

running the triton server from ngc bert:

docker run --runtime nvidia -it --rm --network host     nvcr.io/nvidia/jetson-voice:r32.4.2     trtserver --model-control-mode=none --model-repository=models/repository/jasper-asr-streaming-vad/

feature-extractor-trt-vad-streaming' version 1
I0115 15:34:32.148529 1 model_repository_manager.cc:808] successfully unloaded 'ctc-decoder-cpu-trt-vad-streaming' version 1
error: creating server: INTERNAL - failed to load all models

AastaLLL · January 18, 2021, 5:32am

Hi,

YES.
Since the container is posted for 4.4 DP, you will need to recompile the engine for compatibility.

Thanks.

Andrey1984 · January 18, 2021, 7:10pm

@AastaLLL could you share the exact steps for rebuilding the containers, please?

AastaLLL · January 20, 2021, 5:58am

Hi,

You can find some Dockerfile in dusty’s GitHub:

But not all are available.

Thanks.

Andrey1984 · January 20, 2021, 4:08pm

@AastaLLL
In particular I am looking to rebuild the gaze/ voice contaners so they could run on JP 4.4.1.
Are there directions for doing so?
Thanks

AastaLLL · January 21, 2021, 3:58am

Hi,

Sorry that there is no Dockerfile for these containers.
For gaze, it seems a pure TensorRT environment with some pre-installed models.

Maybe you can find some idea in GitHub below:

Thanks.

Andrey1984 · January 21, 2021, 12:55pm

@AastaLLL, for the pose container we used steps below to re-generate the engine file, right? There should be possibly similar solution for the gaze container?

import torch
>>> import torch2trt
>>> import trt_pose.models
>>> import tensorrt as trt
>>> MODEL = trt_pose.models.densenet121_baseline_att
>>> model = MODEL(18, 42).cuda().eval()
>>> model.load_state_dict(torch.load('/pose/generated/densenet121_baseline_att.pth'))
>>> data = torch.randn((1, 3, 224, 224)).cuda().float()
>>> model_trt = torch2trt.torch2trt(model, [data], fp16_mode=True, max_workspace_size=1 << 25, log_level=trt.Logger.VERBOSE)
>>> torch.save(model_trt.state_dict(), '/pose/generated/densenet121_baseline_att_trt.pth')
>>> exit()

AastaLLL · January 22, 2021, 4:00am

Suppose yes.

eyemovetechnologies · March 22, 2021, 5:04am

Hi @Andrey1984, were you able to resolve the issue and get the gaze container working? We’re running into the same issue.

Igor_Adamov · May 12, 2021, 11:19am

Hello. If it is still relevant I did the following steps:
(I used r32.4.2 version of Gaze Demo container, edit some strings for your version)

Reconfigure the default runtime by adding the following line to the /etc/docker/daemon.json configuration file after first brace:

“default-runtime”: “nvidia”,

(It’s need to acces to CUDA library and e.t.c. when building)

Create the Dockerfile with the following instructions:

FROM nvcr.io/nvidia/jetson-gaze:r32.4.2
WORKDIR /gaze
RUN chmod +x /gaze/setup.sh
WORKDIR /gaze
RUN ./setup.sh

(The setup.sh create new versions of engines for TensorRT version. ‘WORKDIR /gaze’ commands are not required, I think)

Build new image. I used following command to build (open terminal in folder with Dockerfile):

sudo docker build --no-cache --network=host -t nvcr.io/nvidia/jetson-gaze_edited:r32.4.2 .

And run the container

sudo xhost +si:localuser:root

sudo docker run --runtime nvidia -it --rm --network host -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/jetson-gaze_edited:r32.4.2 python3 run_gaze_sequential.py /videos/gaze_video.mp4 --loop --codec=h264

This steps helped in my situation.

Topic		Replies	Views
Gaze Demo for Jetson/L4T - Xavier AGX (L4T R32.4.2) Not Working Jetson AGX Xavier jetson-inference , docker	4	1355	October 18, 2021
Got errors when running the jetson-cloudnative-demo Jetson Xavier NX	6	1142	October 18, 2021
Jetson Xavier NX L4T 32.4.3 Cloud-Native Demo Issues Jetson Xavier NX	4	947	October 18, 2021
How to install Gazebo on Jetson Xavier NX Jetson Xavier NX kernel	23	4378	July 28, 2021
Trt_pose model in docker: ImportError: libnvmedia_tensor.so: cannot open shared object file: No such file or directory Jetson Nano tensorrt , dla	7	947	May 3, 2023
TensorRT v21.12-py3 Docker image cannot work with GPU option on ARM (AGX) device TensorRT tensorrt	13	2487	February 18, 2024
Suggestion to solve Tegra Nvidia-docker issues Jetson Nano cuda , docker	20	3975	October 15, 2021
Torch2trt fail when building container from jetson-containers Jetson AGX Orin tensorrt	8	30	February 5, 2025
Gaze Demo for Jetson/L4T - Xavier AGX (L4T R32.4.2) Not Working TensorRT jetson-inference , docker	1	584	March 18, 2021
Jetson Xavier NX docker-in-docker issue Jetson Xavier NX cuda , docker	7	1646	November 17, 2021

Cloud_native_demo on JP4.4.1

Related topics