HTTP MJPEG Stream as Source to Deepstream

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• NVIDIA Orin AGX 64 GB Dev Kit
• DeepStream 6.2
• Jetpack version 5.1.2-b104
• 8.5.2-1+cuda11.4
• Question/Help

I’m running

deepstream-app -c source1_http_dec_infer_resnet_int8.txt

where the config is just the same as the USB one except for setting:

type=3
uri=http://169.254.1.222:8080/img

However, only the first frame is shown, then it it won’t update anymore.

I can view the stream perfectly fine like this:

gst-launch-1.0 souphttpsrc location=http://169.254.1.222:8080/img is-live=true ! queue ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! xvimagesink sync=false

I already checked these threads:

–

I’m new to Deepstream and would like to know what the best way would be to accomodate that. In particular, I don"t know how to recompile an app and what I would need to change where. Could you help me?

Thanks!

2

Ok, I found a satisfactory solution for me using deepstream_python_apps (1.1.6). Modifying the deepstream-test1-usb example to read from uri and adjusting some gstreamer settings (e.g. setting decoder.set_property(‘mjpeg’, 1) and more).

See attached
deepstream_test_1_poe.txt (9.3 KB)

However, now I would like to do two things:

  • Include a custom tracker (one that tracks more than the bbox location, but also a “distance”). How would I adjust that?
  • Use a WebRTC or websocket sink to stream the video + detections to the browser. Could you help me in how to do that properly?

Thanks!

4

Could you illustrate the “distance” in detail? If you want use your own tracker, you have to implement the interface we provide NvDsTracker API for Low-Level Tracker Library.

You can try using the rtsp sink.

5

  1. By distance I mean a distance value coming from a custom neural network (monocular depth estimation, but for each bounding box). I already have the network and I think I know how to adapt the pt/onnx to get the engine with distances. It’s just that I don’t know how to implement a “custom” tracker. I would be happy with a basic tracker that just passes the distance values along so I can map them appropriately.

  2. The problem with RTSP sink is (1) web does not like RTSP and (2) I would like to have perfectly synked bbox data for each frame (ideally an MJPEG stream with each frame having a bbox package). I want to do custom rendering with the bounding boxes in the frontend. Could you point me to what I should do in that case? Should I rather

Thank you!

6

The tracker can only accept bbox as input currently.

I think you can try the webrtcsink for your scenario. But we don’t have much experience with this plugin either.

7

Ok thanks, I’ll give that a try. And also the appsink through socket. Could you maybe point me to some resource how I can fetch the bbox data to grab to ingest into a sink?

8

You can refer to our nvmsgconv and nvmsgbroker. We have a demo to show you how this works.

sources\apps\sample_apps\deepstream-test5
9

Do you mean this? Is there a Python-based version as well?
How would I combine the video stream with these messages?
In any way, thanks!

10

You can refer to deepstream-test4.

You need to receive the message on your client and then match the source id with the stream. Please refer to our Guide to learn the format of the message.

11

Ok thanks!
Would you say it is synced up to 100ms or is it rather worse?
Thanks!

12

It depends on your actual device board and the network environment, you can try it out first.

13

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.