Home security with Jetson Nano and TX2

Hi,

I would like to share my project: A multi-platform, multi-camera home security system using Jetson Nano, Jetson TX2 and raspberry pis.

Depending on your requirements, you can use:

  • Raspberry pi zero
  • Raspberry pi 3
  • Jetson Nano
  • Jetson TX2

The Pis are used for streaming video to the jetson or for basic motion detection, but the most useful aspects of security are enabled by object detection capabilities of Jetson Nano and TX2.

GitHub: https://github.com/dataplayer12/homesecurity

Please feel free to critique this work and provide suggestions for improvement.

2 Likes

Hi @jaiyamsharma

I really enjoyed your project, i was running on jetson nano. But some error occur

jetson@jetson:~/homesecurity/jetsonano$ python3 securitycamnano.py 
Traceback (most recent call last):
  File "securitycamnano.py", line 10, in <module>
    from config import * #h264_folder, mp4_folder, motion_threshold,log_dir,log_file,countfile
ModuleNotFoundError: No module named 'config'

maybe you haven’t uploaded enough code directories.

Please help

Thanks

Hi henry,

Thank you for your interest in this project. You are launching the script from the homesecurity/jetsonano directory, which is causing the error. The intended way of using the code is to launch the launcher script from the base directory. First, set the device variable in the launcher script to jetsonano with

nano launcher.sh

and then start the security cam with

bash launcher.sh

This script will setup your initial video storage directories and find the other dependencies from other directories. You will also have to provide an email address to send videos to and an email address with password to send the email from. These are to be provided as a text file ‘confidential.txt’ in the base directory of the project. I will update the instructions on GitHub to show how to set up the confidential file. This file will never be tracked or uploaded to your GitHub repo if you are forking the project and developing it further.

Hi henry,

I have just added setup instructions for all devices as well as for setting up the confidential.txt file. Please look at the section about Setup in the readme: https://github.com/dataplayer12/homesecurity/blob/master/README.md#setup

Hi jaiyamsharma

Great,

Thanks so much!

Hi jaiyamsharma,

Can you please share links for hardware used for this project for each combination.

This will help to make sure one is purchasing exact same hardware as a slight change in the model might affect the expected outcome.

Hi gasati,

I purchased the official models for each device, no clones. Some of these were bought from Japan where I currently live and some from India. I don’t know if you would like to source from these places. I have run the project successfully on about 10 different raspberry pis from different vendors. I don’t think the outcome depends on any specific vendor or model. For Jetsons, I believe all the products are the same, so you can’t go wrong if you buy a genuine Jetson board.

Still, for your reference, the following US links should work:

Pi Zero: https://www.amazon.com/Raspberry-Pi-Zero-Wireless-model/dp/B06XFZC3BX/ref=sr_1_4?keywords=raspberry+pi+zero+W&qid=1569220716&s=gateway&sr=8-4

Pi 3B Plus: https://www.amazon.com/ELEMENT-Element14-Raspberry-Pi-Motherboard/dp/B07BDR5PDW/ref=sr_1_3?keywords=raspberry+pi+3B%2B&qid=1569220769&s=gateway&sr=8-3

Jetson Nano: https://www.amazon.com/Seeed-Studio-NVIDIA-Jetson-Developer/dp/B07SGBHDCZ/ref=sr_1_2?keywords=jetson+nano&qid=1569220842&s=gateway&sr=8-2

Jetson TX2: https://www.amazon.com/NVIDIA-945-82771-0000-000-Jetson-TX2-Development/dp/B06XPFH939/ref=sr_1_1?keywords=jetson+tx2&qid=1569220885&s=gateway&sr=8-1

Raspberry pi camera (Pi cam 1): https://www.amazon.com/Arducam-Megapixels-Sensor-OV5647-Raspberry/dp/B012V1HEP4/ref=sr_1_1_sspa?keywords=raspberry+pi+camera&qid=1569220951&s=gateway&sr=8-1-spons&psc=1&spLa=ZW5jcnlwdGVkUXVhbGlmaWVyPUFFRkRHMkpNMEpQVTAmZW5jcnlwdGVkSWQ9QTA5NTExMjUyRVRJNU5LWDlJWEFWJmVuY3J5cHRlZEFkSWQ9QTAzNjQ4MDIyVUVYUDZTS0U2NUdTJndpZGdldE5hbWU9c3BfYXRmJmFjdGlvbj1jbGlja1JlZGlyZWN0JmRvTm90TG9nQ2xpY2s9dHJ1ZQ==

Jetson Nano Camera (Pi cam 2): https://www.amazon.com/Raspberry-Pi-Camera-Module-Megapixel/dp/B01ER2SKFS/ref=sr_1_3?keywords=raspberry+pi+camera&qid=1569220951&s=gateway&sr=8-3

Please note that you cannot use Pi cam 1 with Jetson Nano, but you can use Pi cam 2 with both raspberry pi and Jetson Nano.

Let me know if you need any more information.

Hi jaiyamsharma.

Thank you very much for sharing this.

It seems i have received an error when launching “bash launch.sh”, as shown below:

[Errno 2] No such file or directory: 'common/countfile.txt'
OpenCV Error: Assertion failed (dims <= 2 && step[0] > 0) in locateROI, file /build/opencv-XDqSFW/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp, line 949
Traceback (most recent call last):
  File "jetsonano/securitycamnano.py", line 76, in <module>
    fgmask=cv2.erode(fgmask,kernel,iterations=1)
cv2.error: /build/opencv-XDqSFW/opencv-3.2.0+dfsg/modules/core/src/matrix.cpp:949: error: (-215) dims <= 2 && step[0] > 0 in function locateROI

I cannot seem to find the countfile.txt file in your repository either, and was wondering if you know how to fix this error?

Hi,

countfile.txt is just a text file which stores the number of videos which have been processed. I have added the countfile to the latest commit. Please pull the changes and try again.

Hi jaiyamsharma,

Thank you for the explanation. It now works fine!

Just one thing, I’ve noticed that the securitycamnano.py prints an incorrect time, but the date is correct. Consequently, theres an incorrect timestamp when adding clockoverlay to your code.

Just to make sure, I’ve checked my system clock settings, and printed out the time on a seperate python file. However there is nothing wrong so far.

Do you know how to fix this?

Hi Caldarie,

The timestamp is based on the location setting in common/config.py. The default setting is for ‘Asia/Kolkata’. You can change it to be your location from the list of time zones here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
Alternatively, you could comment out lines 15 and 16 in securitycamnano.py which set the location and timezones for the environment.

Hi jaiyamsharma,

Thank you for the reply. It works now.

However, i now have a problem where the motion detector is too sensitive. Please look at the log below:

GST_ARGUS: PowerService: requested_clock_Hz=1008000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
frame rate=30.160224466697443
Motion detected!!
Video recording started
Analyzing file mp4_videos/video5.mp4 to find human in video
frame rate=57.63114945336818
Motion detected!!
Video recording started
2019-11-22 19:42:06.435584: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7
frame rate=58.93974052093108
Motion detected!!
Video recording started
2019-11-22 19:42:22.707303: W tensorflow/core/common_runtime/bfc_allocator.cc:237] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.02GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-11-22 19:42:26.211926: W tensorflow/core/common_runtime/bfc_allocator.cc:237] Allocator (GPU_0_bfc) ran out of memory trying to allocate 1.02GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
2019-11-22 19:42:29.369057: W tensorflow/core/common_runtime/bfc_allocator.cc:237] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB with freed_by_count=0. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory were available.
frame rate=45.50420845449008

As you can see, it keeps picking up motion every time it finishes recording until the jetson nano runs out of memory. Do you know whats the problem?

In the common directory there is a config file where you can specify how sensitive you want the detctor to be. Please see here:
https://github.com/dataplayer12/homesecurity/blob/2cfdbd2cb4d84f2df5f7491996c03ca2ec4dcb3f/common/config.py#L8

Please increase the motion_threshold depending on the scene you are monitoring.

Im trying to do the same exact home security and I want to know how to follow the steps. I want to have multiple cameras and have the Nvidia tx2 to detect it. Any suggestions and steps to get started.

Hi,
Thank you for your interest in this project. There are a few things to take care of, to do multi-camera detection:

  1. Your cameras should be connected to the same network as the tx2
  2. You can either use multiple raspberry pis as camera or use your own IP cameras, the tx2 doesn’t care. The only requirement is that you should provide the URLs of your streams in jetsontx2/tx2_surveillance.py script. The variables are url1 to url4. Be careful to also provide the resolution of the video streams in (height, width) format. Currently, the code can handle only streams of the same resolution, such as (360, 640) or (480, 640).
  3. Finally, for low latency, please make sure that you have a good network router which can handle all the traffic from the cameras to the tx2. More detailed description of the network considerations are in the readme,

I hope this helps.

1 Like

Is a configuration of four Raspberry Pi 4 devices with connected Raspberry Pi Camera Module v2s streaming video over Ethernet to a Jetson Nano for recording, object detection, and email notifications supported? If so, are any modifications required?

Hi 004,
Yes, this is supported. Although, the initial intention of the project was to use Jetson nano only with one camera, the code written for jetsontx2 also works on jetson nano. Please do the following:

  1. Compile the SSD model on the nano. Run python3 jetsontx2/compile_ssd_mobilenet.py from the base directory of the project. This should be done only once.

  2. Run common/stream_video.py on your raspis. The hostname of your pis should be pi1,pi2,pi3 & pi4. If your hostnames are different, you will need to provide the correct names in jetsontx2/tx2_surveillance.py (on lines 117 to 120).

If you have a display connected to your nano, the 4 camera streams will show up along with bounding box detections. Currently, email support is not enabled for TX2 part of the project, but you can modify the code to do so (using the jetson nano code as a guide). Fee free to get in touch if you need any help in doing it.

1 Like

Thank you for taking the time to provide that guidance. This looks like an incredibly useful project and I look forward to trying it using those instructions.

Hi jaiyamsharma,

Thank you for the share this project

you have any idea about that error

i am using jetson nano

Hi tanna,

Which python are you using? Basically, this is happening because jetsonano/securitycamnano.py script sets up some paths which the jetsonano/file_managernano.py should have access to when it is imported. This worked fine in all python versions that I have tried, but it does not in your case. So, I suggest that you add the following two lines to jetsonano/file_managernano.py after line 4:

import sys
sys.path.insert(0, os.getcwd()+'/common/')

This should fix the error.