Want to know about programs on Layer and Pointwise fusion and CUDA graph

Autonomous Machines Jetson & Embedded Systems Jetson Orin Nano
1

Hi all, I want to know are there any programs as part of the tensorrt sample programs that make use of Layer and Pointwise fusion and CUDA grpah. Please let me know.

Thanks and Regards

Nagaraj Trivedi

4

Sorry for the late response, is this still an issue to support? Thanks

5

Dear @trivedi.nagaraj,
Using trtexec tool and enabling verbose log option you can know which layers are fused. Also, you can enable using CUDA graph.
Does this help?

6

Hi SivaramaKrishnan, thank you for your response. I want it to be included in the python/c++ programs which I write for inferencing. I don’t want for trtexec. Please clarify it.

Thanks and Regards

Nagaraj Trivedi

7

Dear @trivedi.nagaraj,
Could you check Developer Guide :: NVIDIA Deep Learning TensorRT Documentation if it helps? Also, see https://github.com/NVIDIA/TensorRT/blob/release/8.6/samples/common/sampleInference.cpp to know how trtexec using cuda graphs.

8

Hi SivaRamKrishna, I will refer this and I verify.

Thanks and Regards

Nagaraj Trivedi

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.