Program freezes on TX2

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hi all,

I’m working on a TX2 with Jetpack 4.2. The board is connected with two USB cameras and running some computer vision tasks (using pytorch, dlib and opencv).

When started, all the tasks are working correctly. Memory consumption is around 6g. But after around 10mins, all the programs freeze, memory consumption is still the same but cpu usage drops by half. There is no error message in the terminal, I have no idea what is going on. And when running those programs separately, none of them will freeze.

I first suspect that TX2 is too hot, but tegrastats shows GPU at 58C, which is not very high I suppose. And CPU temperature is even lower.

Another possible cause might be pytorch, since there are two programs doing inferences using pytorch. However, if pytorch is not working properly, I would expect some error message in the terminal, but there is none.

Any help is appreciated, thanks!

2

After debugging, I found that the code is stuck at VideoCapture.read(), which is an opencv function reading frames from my USB cameras.

During runtime, all cpu cores are at around 70-%80%, gpu is almost at 99% since there are two pytorch model running at the same time doing realtime inference. Can cpu be the bottleneck and causing the read() to hang indefinitely? While when running only one program, VideoCapture.read() works just fine.

Is there any other possible causes? Much appreciated!

3

Hi,

Sorry for the late update.

It looks like the application is waiting for the camera data and leading to the freeze.
May I know how do you install the OpenCV package first?
Our default OpenCV doesn’t enable the GStreamer support, which read camera with GPU and have a better performance.

Thanks.

4

Hi,

This is the tutorial I followed to install opencv 3.4.6 from source: Installing OpenCV 3.4.6 on Jetson Nano

I believed I installed opencv with gstreamer support.

I’m not sure if using GPU is a good option for me, since I already have two pytorch models running on GPU.

I’ve been suggested to use gstreamer or media api since they use hardware acceleration. But I couldn’t find the way to use either tool to meet my requirements, which is read in frames from USB camera, do something and send images through an RTMP stream so someone else can get the video RTMP stream.

Thanks.

5

Another thing that might go wrong is when I install opencv from source, I checked some file with

sha1sum -c /etc/nv_tegra_release

and all of the tests passes except this one

/usr/lib/aarch64-linux-gnu/libv4lconvert.so: FAILED

This is also mentioned in this post: https://devtalk.nvidia.com/default/topic/1051752/jetson-tx2/build-opencv-3-4-with-cuda-on-nvidia-jetson-tx2/post/5350043/

As a result, I installed opencv 3.4.6 without fixing this problem since I could not find a solution to this. After all the build is success.

Can this cause the opencv to stuck at VideoCapture().read()?

6

Problem solved, DO NOT use Micro AB port. Two USB 2.0 camera both connected to the USB 3.0 type A port via a hub and my program will not freeze any more.

7

Hi,

Thanks for the feedback.
This information is helpful for others with the same use case.