Exclude TacticSources from IBuilderConfig

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hello!
Since my TensorRT code running on Nano based on: https://github.com/noahmr/yolov5-tensorrt is consuming cca 1 GB RAM I’d like to decrease memory usage by - according your advice - excluding CUDNN from “tactic sources”.
Problem is: I can’t find how exactly to perform this exclusion (neither in TRT doc nor forum)?

I tried it inside /src/yolov5_builder.cpp (which is now part of my app) like this, but doesn’t seem to exclude:

std::unique_ptr<nvinfer1::IBuilderConfig> config(builder->createBuilderConfig()); //LN 218
config->setTacticSources(0 << int(nvinfer1::TacticSource::kCUDNN));                 //MY LINE

Beside CUDNN I’d like to attempt exclusion of CuBLAS/CuBLASLt, so question applies to that as well
Many thanks!

Environment

TensorRT Version : 8.0.1
GPU Type : Jetson Nano (Maxwell)
CUDA Version : 10.2
CUDNN Version : 8.2.1
Operating System + Version : Ubuntu 18.04 (Jetpack 4.6)

2

Hi,

This looks like a Jetson issue. Please refer to the below samples in case useful.

For any further assistance, we will move this post to to Jetson related forum.

Thanks!

4

Sir, not sure if we understood well: inference is implemented and works OK I just want to decrease RAM usage by excluding tactic sources-an approach you suggested in my previous thread. I think I just need (correct) argument(s) for config->setTacticSources() call… And it is C++

5

any opinions on this?