Delivering the Missing Building Blocks for NVIDIA CUDA Kernel Fusion in Python

Technical Blogs & Events Technical Blog
1

Originally published at: Delivering the Missing Building Blocks for NVIDIA CUDA Kernel Fusion in Python | NVIDIA Technical Blog

C++ libraries like CUB and Thrust provide high-level building blocks that enable NVIDIA CUDA application and library developers to write speed-of-light code that is portable across architectures. Many widely used projects, such as PyTorch, TensorFlow, XGBoost, and RAPIDS, use these abstractions to implement core functionality. The same abstractions are missing in Python. There are high-level…

3

Just installed cuda-cccl, but got the following error when trying to use it:

File "C:\Users\...\Documents\Python\t.py", line 3, in <module>
    import cupy as cp
  File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\cupy\__init__.py", line 16, in <module>
    from cupy import _core  # NOQA
  File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\cupy\_core\__init__.py", line 3, in <module>
    from cupy._core import core  # NOQA
  File "cupy/_core/core.pyx", line 1, in init cupy._core.core
  File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\cupy\cuda\__init__.py", line 9, in <module>
    from cupy.cuda import compiler  # NOQA
  File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\site-packages\cupy\cuda\compiler.py", line 14, in <module>
    from cupy.cuda import device
  File "cupy/cuda/device.pyx", line 105, in init cupy.cuda.device
  File "cupy/_util.pyx", line 52, in cupy._util.memoize.decorator
  File "C:\Users\...\AppData\Local\Programs\Python\Python313\Lib\functools.py", line 57, in update_wrapper
    setattr(wrapper, attr, value)
AttributeError: attribute '__name__' of 'builtin_function_or_method' objects is not writable

How can I get around it?

4

Thanks! It looks like you might be running into Support Cython 3.1 · Issue #9128 · cupy/cupy · GitHub. What version of CuPy do you have installed?

pip list | grep cupy
5

cupy 13.4.1, installed automatically
Windows-11

6

Maybe CUDA doesn’t support Python 3.13 yet.

For what it’s worth, I’m currently using Python 3.12 with CUDA 12.9.1, and cuda.cccl is working fine.

7

After upgrading cupy to 13.5.1, I got another error:

Python 3.13.5 (tags/v3.13.5:6cb20a2, Jun 11 2025, 16:15:46) [MSC v.1943 64 bit (AMD64)] on win32
C:\Users\...\Documents\Python\transform-iterator.py
Traceback (most recent call last):
  File "C:\Users\...\Documents\Python\transform-iterator.py", line 14, in <module>
    import cuda.cccl.parallel.experimental as parallel
ModuleNotFoundError: No module named 'cuda'

The module is named as cuda-cccl, not cuda.cccl, but no hyphen is allowed in import names. Should I change the module name or use importlib.import_module()?

8

Thanks for the update @LaszloHars. It looks like you’re running on bare Windows, and not WSL. Is using WSL an option for you? We don’t test or support Windows right now, and cuda.cccl wheels aren’t published for Windows either. Running within WSL should work fine.

9

I’m excited to use CCCL. I’ve installed. the Python package using

pip install ‘cuda-cccl[cu12]’

because I am running Cuda 12.4. The install seems to work, and I can import stuff in the module from Python. I have tried some of the example programs provided at the CCCL github site. Some work, e.g. merge_sort_basic.py. However, for most (e.g. scans and reductions and unary transform), I get an error message

EXCEPTION in cccl_device_scan_build(): nvJitLink error: 6
EXCEPTION in cccl_device_scan_cleanup(): CUDA error: invalid resource handle
Return code 999 encountered during scan result cleanup

I would greatly appreciate advice/guidance.