How to calculate clock rate(frequency) in In Artificial intelligence(AI) or Deep Learning Task?

[user39914]

at the moment lately I have a hobby project about Offloading on Jetson Nano with Python My question is “how to calculate clock rate(frequency) in In Artificial intelligence(AI) or Deep Learning Task?”

classically in Computer science(offloading), executed time(or CPU Time) is Product of CPU Clock cycles and Clock Cycle Time.

1. classically Clock cycle time is inverse of frequency of CPU, but how about AI Task?
   Because I think in AI(or DL) GPU have also big role in Task, So should I add just each frequency?
   For example, CPU has 1.5GHz and GPU has 800MHz, then GPU has lower but in AI GPU is more powerful or?
   What’s your opinion about this?
2. and secondary for estimated execute Time, I have to calculate CPU Clock Cycles,
   but I don’t know now, how I exactly calculate this, have someone opinion about this?

Thank in advance

[AastaLLL]

Hi,

You can use CPU clocks as well.
In general, an AI task needs to wait for the inference to finish to get the final results.
So the CPU execution should already include the GPU execution time.

If you want to know the execution time for GPU only.
Please try to measure it with cudaEvent_t.

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__EVENT.html

You can also refer to the below example for more information.
There are several different timing are measured.

https://github.com/NVIDIA/TensorRT/blob/main/samples/common/sampleReporting.h

struct InferenceTime
{
    ...
    float enq{0};     // Enqueue
    float h2d{0};     // Host to Device
    float compute{0}; // Compute
    float d2h{0};     // Device to Host
    float e2e{0};     // end to end
    ...
};

Thanks.