Streaming data from Jetson Nano to a local computer

[dgiacomellic]

Hello people, how are you doing? Fine I hope.

So, I´m kinda new in this universe of video analytics and AI, so please be patient with me :).

I´m working in a project using Jetson nano (later on we will move to Jetson Xavier). The project consists in reading a camera and a LiDaR sensor and use both inputs to detect an specific condition of an environment. After detecting the condition, I will perform some actions using the GPIOs, such as turning a relay on or off, activate / deactivate a buzzer, blink LED and so on.

For the video processing, I´m using this repository as a baseline: GitHub - dusty-nv/jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson., which is helping a lot with the work.

At some point, I need to stream all the data from jetson board to a computer or a tablet. In shorts, I need to stream the video from the camera, the processed video from jetson inference, the LiDaR data and the GPIOs actions. The challenge is to maintain some kind of stream server inside jetson. I do not have a network to stream out all this data.

I started to look if there is any solution for this matter and came up with 2 approaches:

1. nginx: It can build a RTMP server, but the jetson-inference (GitHub - dusty-nv/jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson.) libs has RTP streaming only. Is it possible to use RTP with RTMP? Or it is necessary some kind of protocol parsing?

2. Gstreamer: Could solve the problem, but does GStreamer support any kind of datatype? Or is exclusive to multimedia?

Does anyone solved a similar problem? I´m looking for a direction, but examples are always welcome :).

P.S.: There is no problem in using more than 1 tool to stream all the data, like GStreamer for the videos and a message broker for the rest. I´m trying to find if there is a solution that could do all the streaming.

[DaneLLL]

Hi,
By default the implementation is RTSP. For running RTMP, you would need customize jetson-inference to run flvmux ! rtmpsink:
Jetson Nano USB stream save while streaming to RTMP server with audio - #2 by DaneLLL
Self-reboot when using Gstreamer pull video stream with rtmp

We don’t have much experience about running RTMP. Would need other users to share guidance and suggestion.

[dgiacomellic]

Hello @DaneLLL , sorry for the long time with no reply. I decided to use gstreamer as a bridge between the inference and outside consumer (my computer). And simplified my problem to stream video only.

in this post: Help with viewing RTP stream from Jetson Nano to PC I was able to consume the stream on my computer side using VLC.

But I´m having some difficulties to connect the output of the inference to gstreamer. In other words, I don´t know how to use the jetson.utils.videoOutput(“rtp://<jetson_ip>:1234”) to send data to gstreamer. I tried with gst-launch-1.0 -v udpsrc port=1234… but failed. Is there a way to do it? Did I missed something?

Note: I used tcpdump to monitor the output of the jetson.utils method and I can see the packages being transmitted.

[dusty_nv]

@dgiacomellic

I don´t know how to use the jetson.utils.videoOutput(“rtp://<jetson_ip>:1234”) to send data to gstreamer.

Hi @dgiacomellic, you want to put your laptop/PC’s IP address (or hostname) in there instead of the Jetson IP.

Then run this command on your laptop/PC:
https://github.com/dusty-nv/jetson-inference/blob/master/docs/aux-streaming.md#viewing-rtp-remotely

 $ gst-launch-1.0 -v udpsrc port=1234 \
 caps = "application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264, payload=(int)96" ! \
 rtph264depay ! decodebin ! videoconvert ! autovideosink

[dgiacomellic]

Hello @dusty_nv, thanks for the response. Is it possible to use gstreamer inside jetson? Is it feasible? Allow to give more details. I drew a picture of what I’m trying to achieve in my project:

The idea is similar to a car rear camera. But instead of showing just the camera, I would like to show the dectections or classifications of objects captured by the camera. As output, the video analytics would go to an tablet application.

In the tablet side, it cannot has gstreamer due to project restrictions. In fact, it can only has an application written in flutter. So, the tablet can only access an url like: rtsp://<jetson_ip>:<some_port>. HTTP url can also be used.

Note: in this project, we can replace gstreamer with another with another more suitable tool. Nom problem in that.

[dusty_nv]

@dgiacomellic

Is it possible to use gstreamer inside jetson?

Yes, gstreamer is installed on Jetson by default. You can launch pipelines directly from the command line using gst-launch-1.0 command, or DeepStream uses GStreamer. jetson-inference also uses GStreamer.

@dgiacomellic

In the tablet side, it cannot has gstreamer due to project restrictions. In fact, it can only has an application written in flutter. So, the tablet can only access an url like: rtsp://<jetson_ip>:<some_port>. HTTP url can also be used.

If you want to do RTSP output, I would check out DeepStream because it has built-in support for RTSP server through GStreamer.

jetson-inference can only do RTP output (not RTSP). RTP requires know the IP address or hostname of your tablet (or multicast group the tablet is subscribed to)