DLA and GPU cores at the same time

[Sam]

Hello AastaLLL,

I would like to ask you a question base your provided code.

DLA and GPU cores at the same time

Hi, Here is a sample to run GPU and DLAs at the same time. 1. Please prepare TensorRT engine of GPU and DLA with trtexec first. For example, $ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --saveEngine=gpu.engine $ /usr/src/tensorrt/bin/trtexec --onnx=/usr/src/tensorrt/data/mnist/mnist.onnx --useDLACore=0 --allowGPUFallback --saveEngine=dla.engine 2. Compile TensorRT_sample.zip (3.5 KB) $ unzip TensorRT_sample.zip $ cd TensorRT_sample $ make 3. Test Please…

I profiled your code and I saw the result as in the image:

Based on the image:
After the inference (ExecutionContext::enqueue), there are cudaEventRecord and cudaEventSynchronize.
I would like to ask about what is the total execution time of inference. Does it include the time of cudaEventRecord and cudaEventSynchronize or only time of (ExecutionContext::enqueue) ?

Thank you.