Cuda different errors CudaErrorIllegalAdress, cuda segmentation fault, error: destroying cuda device: 0

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I am working on the following machine:

Processor: Intel Core i7-8750H 
Memory: 16 GB RAM 
1000 GB | 5400RPM | SATA | -,
256 ssd
GPU: NVIDIA GeForce GTX 1060

My deepstream version

DS: 6.1
Driver Version: 515.76
CUDA Version: 11.7

I’m working with deepstream imagedata multistream with adding some integration with redis to add or delete the RTSP streams (using add-remove streams sample) and kafka-python,

The models been used are (primary: resnet10 and Nvds tracker)

After running the code with 1 RTSP stream for X time (where X is different each time but it’s usually more than half hour), I got those different errors at the below screenshoots, kindly advice me what should we do to make that code running 24/7 with 1:30 streams

image
image937×477 156 KB

image
image904×92 82.7 KB

image
image863×141 27.6 KB

2

Hi @ayanasser,

Does this project utilize OptiX? At first glance, this looks like something that might be better addressed in one of the CUDA channels

–
David.

4

No I don’t use OptiX utilization

7

How much memory is on your GTX 1060? Can you also share the output of “nvidia-smi” when running your program?

8

I mention the specs above, also if you mean the Vram is 6GB

The output of nvidia-smi while it’s working smoothly

image
image701×379 34 KB

9

Can any DeepStream sample applications run without any error in your machine? Can you monitor the GPU status and usage with “nvidia-smi dmon” command during your application running?

10

I am working on imagedata sample combining with (Add-delete-sources) sample, but with some modifications and integration with redis and kafka as I mentioned,
and again this happens after N working hours (where N is unknown number).

the other Deepstream samples are working fine but I only tested them with 5 minute video or smthing like that (not a long time)

here’s the output of nvidia-smi demon as the deepstream app is working fine.

image
image555×607 94.6 KB

11

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one.
Thanks

You need to monitor the GPU and memory status until the error happens.