Python Yolov4 CreateTextureObj error (Segmentation fault)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Using deepstream5.1-devel docker on ubuntu20.04.

I am using https://github.com/NVIDIA-AI-IOT/deepstream_python_apps/tree/master/apps/deepstream-imagedata-multistream as an example, but I needed custom funcionality, to add ROI before feeding to model.

So I basically added this code (which is based on How to define different nvvideoconvert for each rtsp stream in deepstream python apps? - #3 by LoveNvidia)

....
# Create nvstreammux instance to form batches from one or more sources.
streammux = Gst.ElementFactory.make("nvstreammux", "Stream-muxer")
if not streammux:
    sys.stderr.write(" Unable to create NvStreamMux \n")

pipeline.add(streammux)
for i in range(number_sources):
    frame_count["stream_" + str(i)] = 0
    print("Creating source_bin ", i, " \n ")
    uri_name = DEVICE_SETTINGS[i]['STREAM_URL']
    ROI_CENTERS.append(get_roi_center(DEVICE_SETTINGS[i]['ROI']))
    fps_streams["stream{0}".format(i)] = GETFPS(i)
    if uri_name.find("rtsp://") == 0:
        is_live = True
    source_bin = create_source_bin(i, uri_name)
    if not source_bin:
        sys.stderr.write("Unable to create source bin \n")

    nvvidconv_crop = Gst.ElementFactory.make("nvvideoconvert", f'convertor{i}')
    nvvidconv_crop.set_property('src-crop', DEVICE_SETTINGS[i]['ROI'])

    pipeline.add(source_bin)
    pipeline.add(nvvidconv_crop)

    padname = "sink_%u" % i
    sinkpad = streammux.get_request_pad(padname)
    srcpad = source_bin.get_static_pad("src")

    nvvidconv_srcpad = nvvidconv_crop.get_static_pad("src")
    nvvidconv_sinkpad = nvvidconv_crop.get_static_pad("sink")

    srcpad.link(nvvidconv_sinkpad)
    nvvidconv_srcpad.link(sinkpad)
print("Creating Pgie \n ")
pgie = Gst.ElementFactory.make("nvinfer", "primary-inference")
if not pgie:
    sys.stderr.write(" Unable to create pgie \n")
....

This code snippet is placed instead of deepstream_imagedata-multistream.py lines 290-311. Basically I added nvvidconv after each source bin and linked them.

ROI’s defined as

 DEVICE_SETTINGS = [
   {
    'STREAM_URL': 'file:///test.mp4',
    'DEVICE_ID': 1,
    'ROI': '0:0:600:600'
 },
 {
    'STREAM_URL': 'file:///test.mp4',
    'DEVICE_ID': 2,
    'ROI': '100:200:400:600'
 },
]

Program runs for a while, but eventually this error is thrown:

Cuda failure: status=700 in CreateTextureObj at line 2919
nvbufsurftransform.cpp:2703: => Cuda failure: status=700 in CreateTextureObj at line 2902
Transformation Failed -2

Cuda failure: status=700 in CreateTextureObj at line 2902
nvbufsurftransform.cpp:2703: => Transformation Failed -2

nvbufsurftransform.cpp:2703: => Transformation Failed -2

OR

CUDA_ERROR_ILLEGAL_ADDRESS at /tmp/workspace/_deps/sass_lib-src/src/implicit_gemm_sass.cpp:58
Cuda failure: status=700 in CreateTextureObj at line 2902
nvbufsurftransform.cpp:2703: => Transformation Failed -2

Cuda failure: status=700 in CreateTextureObj at line 2919
nvbufsurftransform.cpp:2703: => Transformation Failed -2

OR (batch_size=4)

Cuda failure: status=700 in DeleteTextureObj at line 2991
nvbufsurftransform.cpp:2703: => Transformation Failed -2

nvbufsurftransform.cpp(3105) : getLastCudaError() CUDA error : Recevied NvBufSurfTransformError_Execution_Error : (700) an illegal memory access was encountered.
Cuda failure: status=700 in CreateTextureObj at line 2919
nvbufsurftransform.cpp:2703: => Transformation Failed -2

nvbufsurftransform.cpp(3105) : getLastCudaError() CUDA error : Recevied NvBufSurfTransformError_Execution_Error : (46) all CUDA-capable devices are busy or unavailable.

I am guessing it should be something with the ROI part that I added, because without ROI I am not getting such errors. What could cause this and how could I debug this?

3

I don’t mind trying this on C, if I had any working reference how to add ROI cropping before feeding to inference model

4

Hi @pyml ,
if you can use c/c++, could you refer to GitHub - NVIDIA-AI-IOT/yolov4_deepstream ?