GPU Assert

[JoeShz]

• Hardware Platform (Jetson / GPU) NVIDIA GeForce RTX 3090
• DeepStream Version 6.3
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.4.0
• NVIDIA GPU Driver Version (valid for GPU only) 535.113.01

Hello,
I am using a deepstream pipeline in Python, and I am getting this issue: GPUassert: invalid argument src/modules/ReID/ReID.cpp 1127

I am using my own bindings to fetch the embeddings/reid of the tracker.

My pipeline consumes around 12.5 GB out of 24 GB, why it asserts?

[yingliu]

Both TensorRT and GPU driver doesn’t match the requirement for DeepStream6.3, please check the dependency here: Quickstart Guide — DeepStream 6.3 Release documentation (nvidia.com)

[JoeShz]

Hmm, actually the pipeline works sometimes. I did tens of trials and figured out that I got this error when I used 4+ streams and the number of detections was high.

My pipeline is mainly; PGIE (detector) → Tracker (I fetch the embedings) → SGIE(Age_gender).

[fanzh]

the TensorRT version does not meet the requirements. please refer to doc.

[fanzh]

1. do you mean the app works fine if stream number is less than or equal to 4?
2. what is the whole media pipeline? are you using rtsp source or local file? can it be reproduced every time?
3. could you provide a simplified code project to reproduce this issue? Thanks! You can use forum private email(click forum avatar-> personal messages->new message).

[JoeShz]

1- I am not sure but I mean that the app works fine sometimes and sometimes I get this assertion. So, I said that’s maybe because of the number of streams or the detections.

2- For now I am testing on local files, but It is expected to be rtsp one day.

3- I’ll send the code and configs in a private mail.

[JoeShz]

I think I know why I am getting this assertion. My tracker has maxTargetsPerStream parameter which was set to 100. Adn I am getting more than 200 object detection to be tracked. When I made changed it to 256, it works fine.
Thanks!!