Intermittent NvMapMemAlloc error 12 and CUDA allocator crash during PyTorch inference on Jetson Orin Nano

Robotics & Edge Computing Robotics - Isaac Isaac ROS
,
1

Hi,

I’m running a PyTorch YOLO-based inference on a Jetson Orin Nano Super, and I frequently get these errors (not always, but randomly):

NvMapMemAllocInternalTagged: 1075072515 error 12
NvMapMemHandleAlloc: error 0
Error : NVML_SUCCESS == r INTERNAL ASSERT FAILED at "/opt/pytorch/pytorch/c10/cuda/CUDACachingAllocator.cpp":838, please report a bug to PyTorch.

I tried the following, but the issue still occurs:

  • with torch.no_grad() during inference

  • os.environ['PYTORCH_CUDA_ALLOC_CONF'] = 'expandable_segments:True'

  • Full cleanup using torch.cuda.empty_cache(), gc.collect(), and reloading the model

The error isn’t always caught by try/except and sometimes crashes the process.

Setup:

  • Jetson Orin Nano

  • JetPack 6.2.1

  • PyTorch (from NVIDIA SDK): 2.5.0a0+872d972e41.nv24.08

  • Model: YOLO (tracking mode)

Questions:

  1. Is this a PyTorch issue or a Jetson memory allocator issue (NvMapMemAlloc)?

  2. Any known fix or configuration to prevent this intermittent error?

Thanks in advance for any suggestions.

2

Hello @technical12,

Thanks for the post.

This error is typically caused by low-level memory allocation failure on the Jetson due to resource constraints or memory fragmentation. This then causes the crash in the PyTorch memory manager (CUDACachingAllocator). You could either optimize host memory or consider follow the steps Convert the PyTorch model to use TensorRT which is specifically designed to run on the Jetson with maximum efficiency and minimal memory overhead.

Good luck with your implementation!

3

Propably the issue is related the latest jetpack update(r36.4.4->r36.4.7)

4

The following article will be helpfull.

title: Ollama errors orin nano

AastaLLL Moderator

Nov 3

Hi, both

This issue is related to the r36.4.7 update and has a failure rate.
You might see either ‘cudaMalloc failed: out of memory’ or ’ unable to allocate CUDA0 buffer’.
The underlying error for both cases are (in Ollama log):

NvMapMemAllocInternalTagged: 1075072515 error 12

Our internal team is working on the issue.
Will update more information with you later.

Thanks.

title: “unable to allocate CUDA0 buffer” after Updating Ubuntu Packages

AastaLLL Moderator

Nov 5

Hi, @all

Thank you all for the testing and sharing.
We are really sorry about the inconvenience that the r36.4.7 brings.

Although our internal team is still working on the issue, here are some updates about the issue that we can share with you:
The recent update (r38.2.1->r38.2.2, r36.4.4->r36.4.7, 35.6.2->r35.6.3) contains a security fix for CVE-2025-33182 & CVE-2025-33177:

…

The security fix adds a mechanism to prevent the allocation from going into the OOM path (to prevent a denial of service attack).
This led to some limitations in the allocable memory.

We are discussing how to minimize the impact of this security fix.
Will keep you all updated on the latest status.

5

Hello, any updates on the fix? I have a new Jetson nano and seeing the same issue. I am trying to train the 124M GPT2 model.

6

I am facing problem while taking inference from a video and its not processing all the frames, giving NvMemoryAlloc cuda error 12 and giving warning “system throttle due to over current”.

Device: Jetson Orin Nano Super developer kit(8GB)

TensorRT version: 10.3

Cuda version: 12.6

Deepstream: 7.1

Jetpack: 6.2.1

I had converted onnx model(in google colab) to .engine (in jetson)

This error was I tested deepstream sample config file.

If their any solution do tell

Thanks!

7

Hello @darshjain8724
There are a few directions you could check first.

  • Check power mode and supply
    Set a higher power mode following the step 3 and 4 of the documentation.

    sudo nvpmodel -q        # see current mode
sudo nvpmodel -m 0      # or another performance mode for your kit
sudo jetson_clocks

    Make sure you’re using the recommended power supply and avoid powering power‑hungry USB devices from the board.

  • Reduce memory load in the DeepStream config while testing

    • Lower the input resolution (width/height) or batch‑size.

    • If you enabled multiple streams, test with one stream first.

    • Try a smaller engine (e.g., lower precision, INT8/FP16 instead of FP32) if possible.

  • Monitor during run

    Run tegrastats in another terminal while DeepStream is running and watch GPU memory and throttling messages.

Hope this would be helpful for you to resolve the issue.

8

Hello,

As there haven’t been any recent updates on this discussion, we’ll be closing the thread to keep the forum organized.

If you continue to experience this issue or have new questions, please start a new topic and include a reference to this discussion for context. This will help our team and other community members assist you more effectively.

Thank you for your participation in the NVIDIA Isaac ROS community.