A Jetson based DeepStream application to identify areas of high risk through intuitive heat maps
The principal mode by which people are infected with SARS-CoV-2 (the virus that causes COVID-19 ✳️) is through exposure to respiratory droplets carrying the infectious virus, which can be avoided to some extent by using facemasks.
Imagine if we could see through the time, It would be so easy to avoid paths where people have crossed recently without wearing a mask. Yes! ✊🏻, there is a way, that’s what this whole project revolves around.
- If a face without mask is detected then the region will be painted on the screen.
- If more and more faces without mask cross through the same region the area will be painted with more intense colors.
- If nobody has crossed for a very long time the color intensity will start fading out.
In other words, a Heat Map will be generated continuously representing regions where faces have been detected recently. Allowing us to see through the time.
The application is containerized and uses DeepStream as the backbone to run TensorRT optimized models for the maximum throughput. Built on top of deepstream-imagedata-multistream sample app.
Steps to run
- I have used Jetson Nano Devkit (2GB), you can set up on any other Jetson device.
- Make sure
nvidia-dockeris installed on the device (it comes pre-installed with JetPack 4.3+).
- clone the maskout repository.
$ git clone https://github.com/imneonizer/maskout.git $ cd maskout
- Build the docker container.
$ sudo docker build . -t maskout
- Run the docker container.
export DISPLAY=:1 xhost + sudo docker run --rm -it --gpus all \ -v /home/$USER/videos:/videos \ -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix --net host \ --name maskout-ds-container --hostname maskout \ maskout bash
- Once inside the container, you can run the DeepStream application using:
python3 maskout_app.py file:///videos/1.mp4or
python3 maskout_app.py rtsp://<user>:<pass>@<camera-ip>.
- To view the Heatmap, open another terminal and execute below command.
sudo docker exec -it maskout-ds-container bash run_ui.sh
This will keep on fetching frames from the DeepStream container and serving to port
5000 . You can keep this application running and restart the DeepStream application with different input streams. visit http://localhost:5000 to view heatmap.
- Under high load, the RAM consumption went up to 1.2 GB + 600 MB used by Linux Kernel.
- The application is containarized and requires 2.6 Gb of disk space.
- While flashing Jetson Nano using SDK manager, unselect
DeepStreamas we are going to use containers, it will save you 800 Mb of space.
- After flashing only few MB’s are left on the SD card even though your SD card has more storage, this can be reclaimed by going to
Diskssetting then extending the unallocated space with root.
- If you run out of
RAMwhile building or running the container, on Jetson Nano (2Gb) you can:
GTKand run Jetson on headless mode.
swap-memoryto get more ram.
- This application only supports single stream processing.
- UI can be run sepearately on another machine, it requires two ports for the communication one for rtsp stream and another for heatmap. using this method you can do some of the processing on client side.
- If you don’t care about drawing bounding boxes, the inference can be boosted and more FPS can be achieved by setting
enable_osd = Falsein first line of the