The profiling tool with Triton Inference Server

Development Tools Nsight Systems Nsight Systems
#1

The profiling tool with Triton Inference Server

Hi experts,

We would like to monitor the following status on A100 with Triton Inference Server.

  • TensorCore is working or not
  • The whole execution time
  • The GPU execution time
  • Memory utilization

Do you have tools to monitor these status?
We know DLProf can not be used on Triton, so we’re looking for the solution.

Best Regards,
NY

#2

Perf Analyzer can help with some of the things you mentioned.

#3

tgerdes-san,

Thank you for your reply.
We’ll check it.

Best Regards,
NY

#4

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.