TensorRT Myelin Division by 0 error with YOLOv8 Segmentation model

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Hi,

I have exactly the same error as the following post:

The error occurs when there is no object detected in a frame.

The error is mentioned here too:

In my case I’m testing it in a x86 PC with nvcr.io/nvidia/deepstream:7.0-triton-multiarch docker image that includes TensorRT 8.6.1.6.
Downgrading to a previous version of DeepStream is not an option for me.

Is there any workaround I can apply until you fix it and I can use the yolov8 segmentation model with DeepStream 7 (and TensorRT 8.6)?

  • Creating a custom TensorRT layer? (I don’t know if it’s possible to override an existing one)
  • Modifying the ONNX?
  • Building nvinfer with try/catch? (as it seems to occur when enqueuing the buffer, but maybe tensorrt context doesn’t recover after this)
ERROR: [TRT]: 1: [runner.cpp::shapeChangeHelper::621] Error Code 1: Myelin (Division by 0 detected in the shape 
graph. Tensor (Divisor) "sp__mye3" is equal to 0.; )
ERROR: nvdsinfer_backend.cpp:507 Failed to enqueue trt inference batch
ERROR: nvdsinfer_context_impl.cpp:1824 Infer context enqueue buffer failed, nvinfer error:NVDSINFER_TENSORRT_ERROR

Any clue, any advice, or should I look for another segmentation model?

Thanks

3

The next version will upgrade TRT. Modifying the model may solve the problem, but I don’t know the details of TRT. You can go here to ask how to modify the model to avoid this problem.

You can try PeopleSegNet

4

Thanks @junshengy,

I cannot use peoplenet from TAO, as I want to train it for another kind of objects (multiple objects) that are not people.
I’ll ask on TRT forum or github repo if I can have a workaround to avoid the crash.

Is there any release date for next DeepStream version and with which version of TRT it will be built?

thanks

5

There is no specific date yet, it is still in the development stage.

1 Like