Cloud_native_demo on JP4.4.1

the command is abrupt ^^
the corrected command is below:

sudo docker run -it --rm --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/jetson-pose:r32.4.2
>>> import torch
>>> import torch2trt
>>> import trt_pose.models
>>> import tensorrt as trt
>>> MODEL = trt_pose.models.densenet121_baseline_att
>>> model = MODEL(18, 42).cuda().eval()
>>> model.load_state_dict(torch.load('/pose/generated/densenet121_baseline_att.pth'))
>>> data = torch.randn((1, 3, 224, 224)).cuda().float()
>>> model_trt = torch2trt.torch2trt(model, [data], fp16_mode=True, max_workspace_size=1 << 25, log_level=trt.Logger.VERBOSE)
>>> torch.save(model_trt.state_dict(), '/pose/generated/densenet121_baseline_att_trt.pth')
>>> exit()
python3 run_pose_pipeline.py /videos/pose_video.mp4

The sequence above allows to recover the pose component to process the video with it, but how to get it running with CSI Raspberry v2 camera?
Also how to patch other containers but the Deepstream & pose container from the cloud native set of four containers?


trying for csi camera

export DISPLAY=:0
xhost +
 docker run -it --rm --privileged --net=host --runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix  -v /tmp/argus_socket:/tmp/argus_socket --cap-add SYS_PTRACE  nvcr.io/nvidia/jetson-pose:r32.4.2

while the latter command launches the container with the access to cameras, the argument to load inputs from camera by the given script is unknown.
is it supported? like python3 run_pose_pipeline.py /dev/video0 or similar?

Hi,

The sample uses OpenCV to enable the GStreamer pipeline.
So you can modify it for the CSI Raspberry v2 camera with nvarguscamerasrc.

$ sudo docker run -it --rm --net=host --runtime nvidia  -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix  nvcr.io/nvidia/jetson-pose:r32.4.2
> vim /utils/video.py

Thanks.

1 Like

the resuling edition works
thanks to @Honey_Patouceul @AastaLLL

  class Video(object):

    def __init__(self, path, width, height, qsize=10, loop=True, codec='h264'):
        self.path = path
        self.width = width
        self.height = height
        self.qsize = qsize
        self.loop = loop
        self.codec = codec
        self.reset()

   def _gst_str(self):
        return 'nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1280, height=720, format=NV12, framerate=30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=BGRx, width={width}, height={height} ! videoconvert ! queue max-size-buffers={qsize} ! video/x-raw, format=BGR ! appsink sync=0'.format(width=self.width, height=self.height, qsize=self.qsize)

    def reset(self):
        if hasattr(self, 'cap'):
            del self.cap
        self.cap = cv2.VideoCapture(self._gst_str(), cv2.CAP_GSTREAMER)

    def read(self):
        re, img = self.cap.read()
        if re:
            return img
        elif self.loop:
            self.reset()
            re, img = self.cap.read()
            if re:
                return img
            else:
                return None
        else:
            return None

    def destroy(self):
        self.cap.release()

can be executed with

 python3 run_pose_pipeline.py 0

But @AastaLLL do we know how to patch the rest of containers - gaze & BERT?
Are there sources - Dockerfiles from that ngc containers been built ?

Hi,

Thanks for the feedback!

BERT is for audio, not for the camera.
Gaze follows the same workflow to read the camera data.
So you can modify the /utils/video.py located in the nvcr.io/nvidia/jetson-gaze:r32.4.2.

The Dockerfile for these containers is not public.

Thanks.

@AastaLLL yes,
but the complication with BERT/gaze is that they won’t run as is from the repository https://github.com/NVIDIA-AI-IOT/jetson-cloudnative-demo on JP 4.4.1 because of the difference in versions of the container DP4.4 versus jetson OS 4.4.1 environment. That implies that TRT version mismatch prevents .engine to run at least for one of them, as it seems to me
for the pose we used the series of steps above to redo the .engine file, as it seems to me.
probably similar adjustment will be required for gaze also etc.

Step to reproduce the issue - run

./run_demo.sh

from the cloud repository
that fires up 4 containers, but as is of them only works deepstream container
running the gaze container from NGC:

docker run --runtime nvidia -it --rm --network host -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix nvcr.io/nvidia/jetson-gaze:r32.4.2 python3 run_gaze_sequential.py /videos/gaze_video.mp4 --loop --codec=h264

sys: x11 
coreReadArchive.cpp (38) - Serialization Error in verifyHeader: 0 (Version tag does not match)
INVALID_STATE: std::exception
INVALID_CONFIG: Deserialize the cuda engine failed.
Assertion fail in file 'trtNet.cpp' line 146: _engine is null
terminate called after throwing an instance of 'std::exception'
  what():  std::exception

running the triton server from ngc bert:

docker run --runtime nvidia -it --rm --network host     nvcr.io/nvidia/jetson-voice:r32.4.2     trtserver --model-control-mode=none --model-repository=models/repository/jasper-asr-streaming-vad/
feature-extractor-trt-vad-streaming' version 1
I0115 15:34:32.148529 1 model_repository_manager.cc:808] successfully unloaded 'ctc-decoder-cpu-trt-vad-streaming' version 1
error: creating server: INTERNAL - failed to load all models