App crach with NVDSINFER_CUDA_ERROR

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,all.
• Hardware Platform (Jetson / GPU) : NANO，Custom Development Boards
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only) 4.4
• TensorRT Version 7.1.0.16-1
• NVIDIA GPU Driver Version (valid for GPU only) 10.2

We had tested on Nano and Xavier，on the nano we got the an error. (3 Xavier and 5 nano run more then 48 hours, 2 had got the error: one hit between 24 and 25 hours after power on , and one hit between 0 and 1 hours after power on)
The program was complie Independently with the same sources. and auto start after power on configed by the start-stop-daemon

0:34:13.575471491 6797 0x7edc002500 ERROR NVDS_APP deepstream_app.cpp:644:bus_callback: ERROR from infer_2: Failed to dequeue output from inferencing. NvDsInferContext error: NVDSINFER_CUDA_ERROR
0:34:13.575652180 6797 0x7edc002500 ERROR NVDS_APP deepstream_app.cpp:647:bus_callback: Debug info: gstnvinfer.cpp(2121): gst_nvinfer_output_loop (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:infer_bin_2/GstNvInfer:infer_2

the dmesg is attached:
log.txt (96.5 KB)

3

@2251582984

Do you mean both these 2 errors come from Jetson Nano?
Could you please narrow down this issue to a more specific problem?

For example, maybe we need information like what kind of DeepStream pipelines were running, DS pipeline configurations, which DL models are used, trend of memory usage in 48 hours, core files and stack tracing information of core dumps, etc.