Hello, I’ve run into “CUDA error: an illegal memory access was encountered” multiple times regardless of using pytorch/ollama. I’ve tried to cut down my code to the simplest form that still produces this error. Here’s my code: import torch param = torch.randn((150000, 1000), dtype=torch.bfloat16,…

Problem solved. I contacted customer support and changed another 4090.

Hi, @yinqb This is forum to support devtools compute-sanitizer. Do you think anything wrong with compute-sanitizer ? I see your env is 12.6, and pytorch run failed directly. So are you using the right pytorch that compatible with CUDA 12.6 ? Any further question about pytorch, please ask in deep…

Hello, thanks for your reply! I guess the issue I’m running into is the CUDA 900 error “CUDA error: an illegal memory access was encountered“. I was only using compute-sanitizer to try to debug it. Which forum should I redirect this to then? thanks.

Can you try upgrading to driver r590, and compute-sanitizer from CUDA 13.1? Also, can you try adding explicit synchronizations to isolate the issue? Like: import torch param = torch.randn((150000, 1000), dtype=torch.bfloat16, device='cuda:0') torch.cuda.synchronize() # does it fail here? param = …

I’ve done the updates and here’s the screenshots: [image] [image] The driver is 591.59 now and CUDA is 13.1 now. Here’s the output for the code after adding `torch.cuda.synchronize()` (torch131) root@Htzr:~/code# compute-sanitizer python repro_error.py ========= COMPUTE-SANITIZER ========= Pr…

I tried a cuda c++ script and got the same error. #include <iostream> #include <cstdlib> #include <ctime> __global__ void vectorAdd(const float* A, const float* B, float* C, int n) { int i = blockIdx.x * blockDim.x + threadIdx.x; if (i < n) { C[i] = A[i] + B[i]; } } int main(…

Consistent "CUDA error: an illegal memory access was encountered" Error

Developer Tools CUDA Developer Tools Compute Sanitizer

yinqb January 4, 2026, 5:08am 4

Posted here: cudaErrorIllegalAddress Encountered: “CUDA error: an illegal memory access was encountered” - CUDA / CUDA Programming and Performance - NVIDIA Developer Forums

Topic		Replies	Views
cudaErrorIllegalAddress Encountered: "CUDA error: an illegal memory access was encountered" CUDA Programming and Performance cudnn , cublas , cusolver , cufft , cusparse , curand	1	1881	January 20, 2026
Illegal access memory error Linux cuda	0	491	August 19, 2020
cudaMemset: illegal memory access with RTX5090 with 570.86.16 CUDA Programming and Performance llama	24	1718	July 16, 2025
GPU Illegal Memory Access when using pycuda/tensorrt with FP16 input TensorRT	3	1489	December 1, 2022
Illegal Memory Access but memcheck and sanitizer return 0 error CUDA Programming and Performance	1	970	March 2, 2021
Illegal memory access Error Computer Vision & Image Processing tensorrt , pycuda	1	2745	May 3, 2023
Internal Sanitizer Error: The Sanitizer failed to handle a hardware exception Compute Sanitizer	3	551	March 31, 2025
Illegal memory access crash CUDA Programming and Performance	15	4860	January 30, 2022
CUDA error: an illegal memory access was encountered Linux	0	973	October 28, 2020
Compute-sanitizer not catching cudaErrorIllegalAddress CUDA Programming and Performance	16	2422	December 17, 2020

Consistent "CUDA error: an illegal memory access was encountered" Error

Related topics