CUDA Profiler Tools Interface (CUPTI) for CUDA Toolkit 12.8 is now available

Developer Tools CUPTI – CUDA Profiler Tools Interface
1

CUDA Profiler Tools Interface (CUPTI) 12.8 is now available for download in the NVIDIA Registered Developer Program.

The NVIDIA CUDA Profiler Tools Interface (CUPTI), distributed as part of the CUDA Toolkit, is a library that enables the creation of profiling and tracing tools that target CUDA applications. CUPTI provides a set of APIs targeted at ISVs creating profilers and other performance optimization tools:
  • the Activity API,
  • the Callback API,
  • the Host Profiling API,
  • the Range Profiling API,
  • the PC Sampling API,
  • the SASS Metric API,
  • the PM Sampling API,
  • the Checkpoint API,
  • the Profiling API,
  • the Event API,
  • the Metric API, and
  • the Python API (available separately)
Using these CUPTI APIs, independent software developers can create profiling tools that provide low and deterministic profiling overhead on the target system, while giving insight into the CPU and GPU behavior of CUDA applications.

Updates in CUDA Toolkit 12.8 Update 1

    Resolved Issues
    • For graph level tracing, fixed the timestamp issue when memcpy node is the first node in the graph.
    • The PC Sampling API and concurrent kernel tracing have been made compatible, allowing them to be collected in the same pass.
    • Resolved bloat in the size of the BSS section in the static CUPTI library.

Updates in CUDA Toolkit 12.8

    New Features
    • Added support for the Blackwell architecture.
    • Blackwell architecture introduces a new hardware method called as HW events for collecting CUDA kernel timestamps. This method provides more accurate timestamps compared to the traditional SW instrumentation and semaphore based approach, and it is expected to add minimal overhead to the application. The new API cuptiActivityEnableHWTrace enables the collection of kernel timestamps through HW events. In CUDA 12.8, this feature is experimental and subject to certain limitations:
      • It is not supported on MPS, WSL, MCDM, MIG, or vGPU setups.
      • On Windows, only a single context can be profiled at a time, while Linux allows tracing multiple contexts simultaneously.
      • Late attach functionality is not supported.
      These limitations are expected to be addressed and removed in future releases.
    • Added support to provide GPU timestamps for the CUDA event record completion. The activity record CUpti_ActivityCudaEvent is deprecated and has been replaced by the new activity record CUpti_ActivityCudaEvent2. The new field deviceTimestamp provides the device-side timestamp and the field cudaEventSyncId provides a unique ID to associate event synchronization records with the latest CUDA Event record.
    • The activity kind CUPTI_ACTIVITY_KIND_SYNCHRONIZATION doesn t provide records for CUDA synchronization operations which return non-zero CUDA status. The API cuptiActivityEnableAllSyncRecords can be used to enable/disable collection of all CUDA synchronization operations which return non-zero CUDA status.
    • Profiling of NVLINK (nvl*), C2C and PCIe metrics are now supported using Profiler Host API and Range Profiler APIs. These metrics are collected at the device-level. These can t be collected with the metrics which are collected at the context-level.
    • Added a new field copyCount in the memcpy activity record to provide the number of copies performed by the batched memcpy operation. The activity record CUpti_ActivityMemcpy5 is replaced by a new activity record CUpti_ActivityMemcpy6.
    • For PM Sampling, users can configure the buffer append mode to either retain the latest or the oldest records in the event of a buffer overflow. Refer to the CUpti_PmSampling_HardwareBuffer_AppendMode enum for additional details.
    • For unified memory profiling, the bitmask of devices involved in the operation is provided using the field processors. The activity record CUpti_ActivityUnifiedMemoryCounter2 is deprecated and has been replaced by the new activity record CUpti_ActivityUnifiedMemoryCounter3.
    • To provide additional information for graph level trace for device launched graphs, a new activity record CUpti_ActivityDeviceGraphTrace and activity kind CUPTI_ACTIVITY_KIND_DEVICE_GRAPH_TRACE are introduced. The activity kind CUPTI_ACTIVITY_KIND_DEVICE_GRAPH_TRACE can not be directly enabled or disabled. It is enabled when activity kind CUPTI_ACTIVITY_KIND_GRAPH_TRACE is enabled and device graph trace is enabled through the API cuptiActivityEnableDeviceGraph().
    • Added a new overhead kind CUPTI_ACTIVITY_OVERHEAD_UVM_ACTIVITY_INIT to report the overhead incurred while initializing the Unified Memory profiling.
    • Enum CUpti_PcieGen is extended to include PCIe Gen 6.
    Deprecated and dropped features
    • The CUPTI Event API from the header cupti_events.h and the CUPTI Metric API from the header cupti_metrics.h are deprecated in the CUDA 12.8 release and will be removed in a future CUDA release. It is recommended to use the CUPTI Range Profiling API as an alternative. For more information, refer to the section Evolution of the profiling APIs.
    • The PC Sampling Activity API and the source/SASS level metrics from the header cupti_activity.h will not be supported on Blackwell and future GPU architectures. These features were deprecated in earlier releases.
    Resolved Issues
    • Resolved an issue where PM Sampling data was being collected on a different GPU than the intended one in multi-GPU systems.
    • Fixed the issue that activity records for conditional nodes in the graph were not generated for Ampere and later architectures.
    • If the device is in MIG mode, the uuid field in the device and NVLINK records returns the MIG UUID which uniquely identifies the subscribed MIG compute instance. This needs a matching driver from the 12.8 toolkit.
    • CUPTI generates external correlation records for CUDA APIs even when CUDA API tracing is disabled using the cuptiActivityEnableRuntimeApi or cuptiActivityEnableDriverApi API.

Getting Started

Latest PRODUCT INFO

1 Like