RTSP in/out with deepstream python improvement with tracker and MQTT

Accelerated Computing Intelligent Video Analytics DeepStream SDK
8

More updates, sorry for the late report:
I had done the tracker and also added ROI overcrowd detection. Then, i have a question about latency and buffering. Before I ask, here’s the RSTP camera specification:

  • resolution: 640x480
  • bitrate: 512 kbps
  • fps: 20
  • encoding: h265

I tried using mplayer to visualize the display to measure the latency. I got aprrox. 2 seconds without deepstream application (which means purely from rtsp). Then, when I used my app, it resulted into 20 seconds lag. However, for the MQTT itself it published almost no lags. For deepstream RTSP output specification:

  • resolution: 640x480
  • bitrate: 400 kbps
  • fps: depends on inference (I’ve reached max fps which is 20 as the source)
  • encoding: h264 or h265

Here are the stats for latency:

| Method    | Latency (s) |
| Pure RSTP | 2           |
| Original  | 20          |
| Modified  | 20          |

Latency aside, I checked again for encoding. I tried to compare h264 and h265. I used mplayer again and vlc from another computer. I’d seen h265 buffers much that the video sometimes greyed out. This one even used queue each pipeline link. Here’s the log while running (mplayer):

[rtsp @ 0x7f88b6ce60]max delay reached. need to consume packet
[rtsp @ 0x7f88b6ce60]RTP: missed 7 packets
V:  15.4   0/  0 13%  0%  0.0% 0 0 
[rtsp @ 0x7f88b6ce60]max delay reached. need to consume packet
[rtsp @ 0x7f88b6ce60]RTP: missed 308 packets
V:  15.5   0/  0 13%  0%  0.0% 0 0 
[rtsp @ 0x7f88b6ce60]max delay reached. need to consume packet
[rtsp @ 0x7f88b6ce60]RTP: missed 2 packets
[hevc @ 0x7f880feab8]Could not find ref with POC 308
V:  16.9   0/  0 12%  0%  0.0% 0 0 
[rtsp @ 0x7f88b6ce60]max delay reached. need to consume packet
[rtsp @ 0x7f88b6ce60]RTP: missed 2 packets
V:  17.3   0/  0 12%  0%  0.0% 0 0 
[rtsp @ 0x7f88b6ce60]max delay reached. need to consume packet
[rtsp @ 0x7f88b6ce60]RTP: missed 87 packets
V:  17.3   0/  0 12%  0%  0.0% 0 0

The code surrounding the encoding are not modified compared to original code, except the IP address and rtsp protocol.

Here are my concerns so far. I think I need more advice regarding latency and buffering issues.