According to the specifications, the Jetson Orin Nano Super Development Kit has:
NVIDIA Ampere architecture with 1024 CUDA cores and 32 tensor cores
My goal is to see how the number of active CUDA cores affects my code speed. I want to selectively disable the CUDA cores (e.g. 512 active, 512 disabled) and see how it affects performance. My question is:
Is it possible to disable CUDA cores?
If yes, where can I find documentation on this?
My current understanding:
In this post, @AastaLLL mentions that it is not possible to turn CUDA cores off in the original Jetson Nano.
This post by @SPWorley states that it would require hardware changes. Given that it is from 2010, maybe this option has been enabled since then?
In this post, @Honey_Patouceul mentions that you can slow down the GPU clock for the Jetson TX2.
According to the Jetson Linux Dev Guide, “Capping the memory, CPU, and GPU frequencies, and number of online CPU, GPU TPC, DLA and PVA cores at a prequalified level confines the module to the target mode.“. My understanding to this is that there should be a way to cap the number of online GPU cores.
The Ampere architecture has support for Multi-Instance GPU (MIG) where “each GPU can be partitioned into multiple GPU instances“. This is confirmed by an * article on the Technical Blog, where “It has up to eight streaming multiprocessors (SMs) composed of 1024 CUDA cores and up to 32 Tensor Cores for AI processing.“.
*** Please note that this reply is generated by LLM automatically ***
Based on the provided context, it appears that disabling CUDA cores on the Jetson Orin Nano Super Development Kit is not directly possible. However, there are some alternatives and related features that might be helpful for your goal of selectively disabling CUDA cores and observing the performance impact.
Capping the number of online GPU cores: As mentioned in the Jetson Linux Dev Guide, it is possible to cap the number of online GPU cores at a prequalified level. This might allow you to simulate the effect of disabling CUDA cores, although it’s not exactly the same thing. You can try using the nvpmodel command to set the power mode and adjust the number of online GPU cores.
MultiInstance GPU (MIG): The Ampere architecture, which is used in the Jetson Orin Nano Super, supports MIG. This feature allows you to partition the GPU into multiple instances, each with a specific number of CUDA cores and Tensor Cores. While this doesn’t exactly disable CUDA cores, it can help you create separate instances with different resource allocations. You can explore the MIG documentation and see if it can be used to achieve your goals.
GPU clock frequency adjustment: As mentioned in the post by HoneyPatouceul, you can slow down the GPU clock frequency. This might not be exactly what you’re looking for, but it can help you observe the performance impact of reducing the GPU’s processing power.
NVIDIA’s documentation and support: Since the original Jetson Nano and Jetson TX2 are older models, it’s possible that newer models like the Jetson Orin Nano Super have additional features or documentation that can help you achieve your goals. You can try searching NVIDIA’s official documentation and support resources for more information on disabling or capping CUDA cores.
In summary, while directly disabling CUDA cores might not be possible, there are alternative approaches and features that can help you simulate the effect or achieve similar goals. You can explore the options mentioned above and see if they can be used to meet your requirements.
Here’s a comparison of the alternatives:
Alternative
Description
Effectiveness
Capping online GPU cores
Adjust the number of online GPU cores using nvpmodel
Simulates the effect of disabling CUDA cores, but not exactly the same
MultiInstance GPU (MIG)
Partition the GPU into multiple instances with different resource allocations
Allows for separate instances with different CUDA core allocations, but not exactly disabling cores
GPU clock frequency adjustment
Slow down the GPU clock frequency
Reduces the GPU’s processing power, but not exactly disabling CUDA cores
Keep in mind that these alternatives might not exactly meet your requirements, but they can help you achieve similar goals or simulate the effect of disabling CUDA cores.
*** Please give the thumbs up if you get answers you like. Or provide feedback to help us improve the answer. ***
Capping the number of online GPU cores → How would I cap the number of online GPU cores with nvpmodel?
MultiInstance GPU (MIG) → OK, since MIG is available on the Jetson Orin Nano, thanks to its Ampere architecture, how do I use it?
GPU clock frequency adjustment → This is a undesirable option. I would want to shut off cores, and prevent any calculations on them. This would allow me to perform power profiling more accurately.
NVIDIA’s documentation and support → I have tried to look through the documentation the best I can. Hence, me asking on the NVIDIA Developer Forum 🙂
Since each GPU TPC unit has 2048 cores, does this mean that the Jetson Orin Nano 8GB has (4 x 2048 =) 8192 CUDA cores?
The Jetson Orin Nano 8GB has 4 TPC units. Therefore, if I select 2 TPC does this mean that half the total CUDA (as shown in the PowerEstimator tool) cores are active, but the rest of the architecture remains the same. Is this understanding accurate?
