Intermittent NvMapMemAlloc error 12 and CUDA allocator crash during PyTorch inference on Jetson Orin Nano

Robotics & Edge Computing Robotics - Isaac Isaac ROS
,
1

Hi,

I’m running a PyTorch YOLO-based inference on a Jetson Orin Nano Super, and I frequently get these errors (not always, but randomly):

NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
Error : NVML_SUCCESS == r INTERNAL ASSERT FAILED at "/opt/pytorch/pytorch/c10/cuda/CUDACachingAllocator.cpp":838, please report a bug to PyTorch.

I tried the following, but the issue still occurs:

  • with torch.no_grad() during inference

  • os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'

  • Full cleanup using torch.cuda.empty_cache(), gc.collect(), and reloading the model

The error isn’t always caught by try/except and sometimes crashes the process.

Setup:

  • Jetson Orin Nano

  • JetPack 6.2.1

  • PyTorch (from NVIDIA SDK): 2.5.0a0+872d972e41.nv24.08

  • Model: YOLO (tracking mode)

Questions:

  1. Is this a PyTorch issue or a Jetson memory allocator issue (NvMapMemAlloc)?

  2. Any known fix or configuration to prevent this intermittent error?

Thanks in advance for any suggestions.

2

Hello @technical12,

Thanks for the post.

This error is typically caused by low-level memory allocation failure on the Jetson due to resource constraints or memory fragmentation. This then causes the crash in the PyTorch memory manager (CUDACachingAllocator). You could either optimize host memory or consider follow the steps Convert the PyTorch model to use TensorRT which is specifically designed to run on the Jetson with maximum efficiency and minimal memory overhead.

Good luck with your implementation!

3

Propably the issue is related the latest jetpack update(r36.4.4->r36.4.7)

4

The following article will be helpfull.

title: Ollama errors orin nano

AastaLLL Moderator

Nov 3

Hi, both

This issue is related to the r36.4.7 update and has a failure rate.
You might see either ‘cudaMalloc failed: out of memory’ or ’ unable to allocate CUDA0 buffer’.
The underlying error for both cases are (in Ollama log):

NvMapMemAllocInternalTagged: 1075072515 error 12

Our internal team is working on the issue.
Will update more information with you later.

Thanks.

title: “unable to allocate CUDA0 buffer” after Updating Ubuntu Packages

AastaLLL Moderator

Nov 5

Hi, @all

Thank you all for the testing and sharing.
We are really sorry about the inconvenience that the r36.4.7 brings.

Although our internal team is still working on the issue, here are some updates about the issue that we can share with you:
The recent update (r38.2.1->r38.2.2, r36.4.4->r36.4.7, 35.6.2->r35.6.3) contains a security fix for CVE-2025-33182 & CVE-2025-33177:

…

The security fix adds a mechanism to prevent the allocation from going into the OOM path (to prevent a denial of service attack).
This led to some limitations in the allocable memory.

We are discussing how to minimize the impact of this security fix.
Will keep you all updated on the latest status.