Hi! I’m running the following script on a Jetson AGX Orin 64GB dev kit via the nemo jetson container and I’m getting a “RuntimeError: CUDA driver error: out of memory”. Any ideas? When I catch the exception and log memory usage (with torch.cuda.memory…), it’s no where near the 60GB available (nothi…

Sure it’s actually NVIDIA’s code that I was trying: [image] nvidia/canary-1b · Hugging Face We’re on a journey to advance and democratize artificial intelligence through open source and open science. To be honest, I had to give up on the Jetson for now and c…

Hi, Please try to increase the memory amount as below comment to see if it works. [image] CSI Camera freezes every time I try image classification Jetson Nano Hi @jdwwhetten2 , can you try the suggestions from this post: https://forums.developer.nvidia.com/t/camera-freez…

I may be missing something but that would restrict the memory amount rather than increase it no? Currently the nemo container has 60GB of memory (+32GB swap) available to it and adding those parameters to docker would limit it to 500MB of memory and 7.5GB of swap.

I was able to transcribe a 30s audio file on the hugging face canary space without any issue, and it’s running on a T4 with 16GB of RAM and 16GB of VRAM. That’s 50% of the memory resources my orin dev kit has. Could this indicate that the Out of Memory (OOM) error I encountered is a misdiagnosis? […

Hi, Swap is not a GPU-accessible memory. Could you check the system status with tegrastats first? The 64GB memory is shared between the CPU and GPU and some might be occupied by the CPU for the OS. Thanks.

In that case, could you clarify why you recommended following instructions to “try adding --memory=500M --memory-swap=8G to your docker run script”. I had checked the system status earlier and mentioned there was plenty of free memory available…

Hi, When you run a PyTorch application, not all the memory is used for GPU tasks. So passing more memory might help in some use cases. Could you try to run the application without using the container? This will help to figure out if this is related to the container resource access. More, could …

I concurrently monitored the device and it’s not fully occupied. Not even close. I can try to run it outside the container. One thing I tried already was allocating empty tensors adding up to 32GB of memory and that worked fine within the container. That would be memory associated with GPU tasks -…

Hi, GPU memory can be allocated via cudaMalloc() directly. Would you mind sharing the testing code with us so we can try the same internally to gather more information. Thanks.

Nemo > Canary 1B > RuntimeError: CUDA driver error: out of memory

Robotics & Edge Computing Jetson Systems Jetson AGX Orin

AastaLLL December 31, 2024, 7:23am 4

Hi,

Please try to increase the memory amount as below comment to see if it works.

Thanks

Topic		Replies	Views
OOM Error Running canary-1b-v2 on Jetson AGX Orin (Memory Not Actually Full) Jetson AGX Orin nemo , generative_ai , speech	2	188	November 3, 2025
CUDA out of memory Jetson Orin Nano cuda	6	628	November 6, 2025
How Jetson allocate memory for GPU? Jetson Orin Nano generative_ai	5	364	January 20, 2026
NanoOwl Generative AI example memory error AGX Orin Jetson AGX Orin generative_ai	3	621	December 5, 2023
Jetson Orin Nano Super insufficient GPU memory Jetson Orin Nano cudnn	20	1100	April 29, 2025
Pytorch 2.4.1 compilation on jetson xavier nx Jetson AGX Xavier cuda , nvbugs , pytorch	12	137	March 24, 2026
How to use Qwen3-ASR-0.6B on jetson orin nano? Jetson Orin Nano cuda , generative_ai , llm	2	266	March 23, 2026
Running large onnx model is getting killed automatically due to insufficient memory? Jetson Orin Nano hw , cuda , onnx	6	717	March 31, 2025
Unable to load large models on Jetson Orin Nano Super despite sufficient RAM Jetson Orin Nano llm	6	631	October 28, 2025
Free up more RAM for Ollama (Jetson Orin Nano Super) Jetson Orin Nano generative_ai	7	1228	May 21, 2025

Nemo > Canary 1B > RuntimeError: CUDA driver error: out of memory

Related topics