NValhalla - a DeepStream demo in Vala

Autonomous Machines Jetson & Embedded Systems Jetson Projects
1

NValhalla performs live redactions on multiple video streams. It is written in Genie, a Vala dialect. Vala is recommended by the gstreamer team for those who want syntactic sugar on top of their GObject C.

2016×1512 514 KB

Features:

  • a GstRtspServer based sink (output) based on Nvidia’s python example code.
  • redaction model and buffer callback is from Nvidia’s deepstream redaction example
  • uridecodebin based sources, so any uri:// should work as input including file:///path/to/video.mp4
  • creates a config folder at ~/.nvalhalla with an inference config, models and .dot files

Many thanks to DaneLLL, amycao, ChrisDing, carolyuu, and the rest of the DeepStream team I might be forgetting for helping me resolve the various issues I ran into writing this app.

It is not very optimized as of yet, so it will run best on a Xavier, however, the next thing I plan on doing is making an int8 calibration file (right now the model is fp32) which should greatly improve the performance.

Please report any issues you run into on the project’s github issue tracker.

2 Likes
3

v0.1.2 is released.

x86 Docker support is now added. All that is needed is Nvidia Docker and you can do:

docker run --gpus all -p 8554:8554 --rm mdegans/nvalhalla --uri $(youtube-dl -f best -g https://www.youtube.com/watch?v=awdX61DPWf4) --uri $(youtube-dl -f best -g https://www.youtube.com/watch?v=FPs_lU01KoI) --uri $(youtube-dl -f best -g https://www.youtube.com/watch?v=SnMBYMOTwEs) --uri $(youtube-dl -f best -g https://www.youtube.com/watch?v=jYusNNldesc)

(this assumes youtube-dl is installed on the host, otherwise substitute urls such as rtsp streams can be used)

The sink defaults to RTSP in Docker, so you may connect to the output with VLC or with gst-play like this:

gst-play-1.0 rtsp://hostname:8554/nvalhalla

Where “hostname” is the hostname of your docker host.

1 Like
4

Preliminary social distancing support has been added in v0.1.7. It can be enabled with the --kenneth option.

Notes:

  • A timestamped, coded protobuf stream is written to ~/.nvalhalla/meta/ (in docker /var/lib/nvalhalla/.nvalhalla/meta) Included is per-object danger level, frame level danger level, timestamps (pts and ntp), source IDs, tracker IDs, and bbox coordinates among other things. Decoding it will require knowledge of coded protobuf streams and this .proto.
  • .csv is supported internally in limited capacity for compatibility with smart_social_distancing, however this is not exposed in NValhalla without modification to the source.
  • The distancing code and the brokering code is contained in a plugin, so these elements can be reused in any deepstream app. A thread for the plugins themselves will be posted as soon as I finish writing their READMEs. Likewise the metadata could be easily brokered with a modified nvmsgbroker (eg. to send to Kafka).

Known issues:

  • the bbox color is wrong on Tegra (it’s green instead of red).