Performance of Neural Network on Jetson

Dear Concern,

I want to breakdown the performance of a neural network on jetson.
Is there any method which basically gives layerwise performance on jetson in terms of time and energy.

Thanks

Hi msee16012, if you are using TensorRT, you could utilize it’s IProfiler interface, which will report the execution time that each layer takes.
See the following documentation for more info about it:

Thanks.

Can I do profiling of a neural network, I mean layer-wise performance without using tensorRT and using NVIDIA Nsight Systems.

You can use the Visual Profiler to measure the execution time of the individual CUDA kernels, but you would have to map these back to the layers. You could instrument the code with markers via the NVTX API, such that the names of the code sections would appear in the profiler tool.

However if you are using a framework, the invocation of the individual layers for inferencing might be abstracted away. I believe some frameworks have built-in profiling functions to serve this purpose.

Hi,

I am able to get information of each layer in terms of execution time using tensorrt.

here are my questions

  1. Tensorrt does some optimization so profiling using tensorrt gives me time for each layer of the optimized model no original model. I want to do profiling of original model

  2. I am getting execution time, is there any way that I can get energy and memory access for each layer ?

Hi,

1. You can profiling the original model with its frameworks directly.
Ex. If a model comes from TensorFlow, you should be able to profile it with TensorFlow directly.

2. We only support execution time profiling in layer-level.
You can get some energy and memory information with tegrastats but not in layer-level.

sudo tegrastats

Thanks.