CUDA Device can't be detected on Jetson Orin Nano running JP7

Robotics & Edge Computing Jetson Systems Jetson Orin Nano
1

Hi, I recently saw that MiniCPM5 is released, so of course I wanted to try it out, but when I run it in Ollama, it was strangely slow, and I didn’t heard any coil whine from the GPU, so I ran ollama ps and this is the result:

NAME                                                                 ID              SIZE      PROCESSOR    CONTEXT    UNTIL              
hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M    6263d7d0bd60    1.8 GB    100% CPU     4096       4 minutes from now

It’s running completely on CPU… and this issue did not only happen on Ollama, other apps that uses CUDA have the same result, for example, LM Studio:

image
image2560×1440 221 KB

but nvidia-smi can detect the GPU?

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.78                 Driver Version: 595.78         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Orin (nvgpu)                  N/A  |   N/A              N/A |                  N/A |
| N/A   N/A  N/A             N/A  /  N/A  | Not Supported          |     N/A          N/A |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

I already tried reinstall the whole jetpack stack using apt but it didn’t do much…

5

Hi,

We run the model with the command shared below:
(replace huggingface-cli with hf)

Ollama can use 100% GPU as below:

$ ollama ps
NAME                  ID              SIZE      PROCESSOR    CONTEXT    UNTIL              
minicpm5-2b:latest    883ef2198ddf    1.9 GB    100% GPU     8192       3 minutes from now    
gemma:2b              b50d6c999e59    2.1 GB    100% GPU     8192       2 minutes from now

Thanks.

6

Hi, I ran the command you provided, but it still says 100% CPU

NAME                                                                 ID              SIZE      PROCESSOR    CONTEXT    UNTIL              
hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M    6263d7d0bd60    1.8 GB    100% CPU     4096       4 minutes from now
7

Hi,

Could you set OLLAMA_DEBUG=2 and share the output log with us?

$ sudo systemctl edit ollama.service

Add

[Service]
Environment="OLLAMA_DEBUG=2"
Environment="OLLAMA_IGPU_ENABLE=1"

Then run the model again and share the output of the command below:

$ journalctl -u ollama

Thanks.

8

Hi, here are the logs

 user   ~    export OLLAMA_DEBUG=2 OLLAMA_IGPU_ENABLE=1
 user   ~    ollama serve
time=2026-10-04T15:24:54.855+08:00 level=INFO source=routes.go:1947 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GGML_VK_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: LLAMA_ARG_FIT: LLAMA_ARG_FIT_TARGET: NO_PROXY: OLLAMA_CONTEXT_LENGTH:0 OLLAMA_DEBUG:DEBUG-4 OLLAMA_DEBUG_LOG_REQUESTS:false OLLAMA_EDITOR: OLLAMA_FLASH_ATTENTION:false OLLAMA_GO_TEMPLATE:true OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_IGPU_ENABLE:1 OLLAMA_KEEP_ALIVE:5m0s OLLAMA_KV_CACHE_TYPE: OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_TRANSFER_STREAMS:4 OLLAMA_MODELS:/home/user/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NO_CLOUD:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://* vscode-webview://* vscode-file://*] OLLAMA_REMOTES:[ollama.com] OLLAMA_SCHED_SPREAD:false OLLAMA_VULKAN:true ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2026-10-04T15:24:54.856+08:00 level=INFO source=routes.go:1949 msg="Ollama cloud disabled: false"
time=2026-10-04T15:24:54.864+08:00 level=INFO source=images.go:883 msg="total blobs: 12"
time=2026-10-04T15:24:54.865+08:00 level=INFO source=images.go:890 msg="total unused blobs removed: 0"
time=2026-10-04T15:24:54.867+08:00 level=DEBUG source=model_recommendations.go:57 msg="starting model recommendations cache" default_recommendations=6 refresh_interval=4h0m0s fetch_timeout=3s
time=2026-10-04T15:24:54.867+08:00 level=DEBUG source=model_show_cache.go:128 msg="starting model show cache"
time=2026-10-04T15:24:54.867+08:00 level=DEBUG source=model_list_cache.go:71 msg="starting model list cache"
time=2026-10-04T15:24:54.867+08:00 level=INFO source=routes.go:2004 msg="Listening on [::]:11434 (version 0.32.0)"
time=2026-10-04T15:24:54.868+08:00 level=DEBUG source=sched.go:221 msg="starting llm scheduler"
time=2026-10-04T15:24:54.869+08:00 level=INFO source=runner.go:60 msg="discovering available GPUs..."
time=2026-10-04T15:24:54.869+08:00 level=DEBUG source=model_recommendations.go:262 msg="loaded model recommendations snapshot" path=/home/user/.ollama/cache/model-recommendations.json count=5
time=2026-10-04T15:24:54.869+08:00 level=DEBUG source=model_recommendations.go:192 msg="refreshing model recommendations from remote" url=https://ollama.com/api/experimental/model-recommendations
time=2026-10-04T15:24:54.870+08:00 level=DEBUG source=gpu.go:63 msg="unrecognized L4T version" nv_tegra_release="# R39 (release), REVISION: 2.1, GCID: 46758480, BOARD: generic, EABI: aarch64, DATE: Fri Aug  7 05:54:22 AM UTC 2026\n# KERNEL_VARIANT: oot\nTARGET_USERSPACE_LIB_DIR=nvidia\nTARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia\n"
time=2026-10-04T15:24:54.871+08:00 level=TRACE source=llama_server.go:71 msg="running llama-server for discovery" cmd=/usr/local/lib/ollama/llama-server libDirs="[/usr/local/lib/ollama /usr/local/lib/ollama/cuda_v13]"
time=2026-10-04T15:24:54.875+08:00 level=INFO source=model_list_cache.go:112 msg="model list cache hydration complete" models=5 failures=0 elapsed=7.685578ms
0.00.342.221 E ggml_cuda_init: failed to initialize CUDA: unknown error
0.00.345.575 I cmn  common_param: common_params_print_info: build 1 (cb295bf59) with Clang 21.1.8 for Linux aarch64
0.00.345.584 I cmn  common_param: common_params_print_info: verbosity = 2147483647 (adjust with the `-lv N` CLI arg)
0.00.347.038 I cmn  common_param: system_info: n_threads = 6 (n_threads_batch = 6) / 6 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | REPACK = 1 | CUDA : ARCHS = 750,800,860,870,890,900,1000,1030,1100,1200,1210 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | 
time=2026-10-04T15:24:55.267+08:00 level=DEBUG source=llama_server.go:121 msg="llama-server discovery: stopped subprocess after collecting GPU info" exit=unknown libDirs="[/usr/local/lib/ollama /usr/local/lib/ollama/cuda_v13]"
0.00.165.746 E ggml_cuda_init: failed to initialize CUDA: unknown error
Available devices:
time=2026-10-04T15:24:55.728+08:00 level=DEBUG source=model_recommendations.go:225 msg="model recommendations refreshed" count=5
time=2026-10-04T15:24:55.734+08:00 level=DEBUG source=model_recommendations.go:302 msg="persisted model recommendations snapshot" path=/home/user/.ollama/cache/model-recommendations.json count=5
time=2026-10-04T15:24:55.734+08:00 level=INFO source=model_recommendations.go:177 msg="model recommendations cache sleep scheduled" wait=3h56m4.222731819s consecutive_failures=0
ggml_cuda_init: failed to initialize CUDA: unknown error
load_backend: loaded CUDA backend from /usr/local/lib/ollama/cuda_v13/libggml-cuda.so
Error: cuInit failed: 999
time=2026-10-04T15:24:55.856+08:00 level=DEBUG source=native_probe.go:259 msg="native GPU discovery failed" error="exit status 1" stderr="ggml_cuda_init: failed to initialize CUDA: unknown error\nload_backend: loaded CUDA backend from /usr/local/lib/ollama/cuda_v13/libggml-cuda.so\nError: cuInit failed: 999\n" libDirs="[/usr/local/lib/ollama /usr/local/lib/ollama/cuda_v13]"
time=2026-10-04T15:24:55.856+08:00 level=DEBUG source=llama_server.go:52 msg="llama-server device discovery took" duration=986.055231ms libDirs="[/usr/local/lib/ollama /usr/local/lib/ollama/cuda_v13]"
time=2026-10-04T15:24:55.857+08:00 level=DEBUG source=runner.go:494 msg="bootstrap discovery took" duration=986.815703ms OLLAMA_LIBRARY_PATH="[/usr/local/lib/ollama /usr/local/lib/ollama/cuda_v13]" extra_envs=map[]
time=2026-10-04T15:24:55.857+08:00 level=DEBUG source=runner.go:131 msg="evaluating which, if any, devices to filter out" initial_count=0
time=2026-10-04T15:24:55.857+08:00 level=TRACE source=runner.go:181 msg="supported GPU library combinations before filtering" supported=map[]
time=2026-10-04T15:24:55.858+08:00 level=DEBUG source=runner.go:42 msg="GPU bootstrap discovery took" duration=989.669581ms
time=2026-10-04T15:24:55.858+08:00 level=INFO source=types.go:50 msg="inference compute" id=cpu library=cpu compute="" name=cpu description=cpu libdirs=ollama driver="" pci_id="" type="" total="7.4 GiB" available="585.6 MiB"
time=2026-10-04T15:24:55.859+08:00 level=INFO source=routes.go:2054 msg="vram-based default context" total_vram="0 B" default_num_ctx=4096
[GIN] 2026/10/04 - 15:25:22 | 200 |    1.813207ms |       127.0.0.1 | HEAD     "/"
[GIN] 2026/10/04 - 15:25:22 | 200 |    6.954228ms |       127.0.0.1 | GET      "/api/tags"
[GIN] 2026/10/04 - 15:25:30 | 200 |      43.169µs |       127.0.0.1 | HEAD     "/"
time=2026-10-04T15:25:30.667+08:00 level=TRACE source=ggml.go:327 msg="key with type not found" key=general.alignment default=32
[GIN] 2026/10/04 - 15:25:30 | 200 |  236.938178ms |       127.0.0.1 | POST     "/api/show"
[GIN] 2026/10/04 - 15:25:30 | 200 |    2.522413ms |       127.0.0.1 | POST     "/api/show"
time=2026-10-04T15:25:31.011+08:00 level=TRACE source=sched.go:247 msg="processing incoming request" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.012+08:00 level=TRACE source=sched.go:281 msg="refreshing GPU list" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.012+08:00 level=DEBUG source=runner.go:272 msg="refreshing free memory"
time=2026-10-04T15:25:31.012+08:00 level=DEBUG source=runner.go:42 msg="overall device VRAM discovery took" duration=385.868µs
time=2026-10-04T15:25:31.012+08:00 level=TRACE source=sched.go:284 msg="refreshing system information" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.012+08:00 level=TRACE source=gpu.go:22 msg="performing system memory discovery"
time=2026-10-04T15:25:31.012+08:00 level=TRACE source=gpu.go:25 msg="system memory discovery completed" duration=177.349µs
time=2026-10-04T15:25:31.012+08:00 level=DEBUG source=sched.go:296 msg="updating default concurrency" OLLAMA_MAX_LOADED_MODELS=3 gpu_count=0
time=2026-10-04T15:25:31.012+08:00 level=TRACE source=sched.go:300 msg="updating free space" gpu_count=0 model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.012+08:00 level=DEBUG source=sched.go:305 msg="loading first model" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.076+08:00 level=TRACE source=ggml.go:327 msg="key with type not found" key=general.alignment default=32
time=2026-10-04T15:25:31.077+08:00 level=INFO source=sched.go:1158 msg="disabling mmap for llama-server load by default" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb reason=cpu
time=2026-10-04T15:25:31.077+08:00 level=INFO source=server.go:109 msg="using llama-server for model" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.077+08:00 level=TRACE source=ggml.go:327 msg="key with type not found" key=llama.nextn_predict_layers default=0
time=2026-10-04T15:25:31.078+08:00 level=INFO source=llama_server.go:433 msg="starting llama-server" cmd="/usr/local/lib/ollama/llama-server --model /home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb --port 39025 --host 127.0.0.1 --no-webui --offline -c 4096 -np 1 --log-verbosity 4 --no-log-prefix --no-log-timestamps --no-mmap --flash-attn auto -b 512 -ub 512 --context-shift --keep 4"
time=2026-10-04T15:25:31.078+08:00 level=DEBUG source=llama_server.go:434 msg=subprocess GGML_BACKEND_PATH=/usr/local/lib/ollama/cuda_v13/libggml-cuda.so LD_LIBRARY_PATH=/usr/local/lib/ollama:/usr/local/lib/ollama/cuda_v13 PATH=/home/user/iOS/netmuxd/target/release:/home/user/KataGo/cpp:/home/user/Arduino-IDE:/home/user/iOS/netmuxd/target/release:/home/user/.opencode/bin:/home/user/.atuin/bin:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/opt/devkitpro/tools/bin:/home/user/.cargo/bin:/home/user/openpilot/.venv/bin:/home/user/.local/bin:/home/user/.nvm/versions/node/v24.12.0/bin:/usr/local/cuda-12.6/bin:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/user/.local/bin:/home/user/.lmstudio/bin:/usr/local/go/bin:/home/user/go/bin
time=2026-10-04T15:25:31.079+08:00 level=INFO source=sched.go:623 msg="system memory" total="7.4 GiB" free="391.1 MiB" free_swap="21.3 GiB"
time=2026-10-04T15:25:31.079+08:00 level=INFO source=llama_server.go:1001 msg="loading model via llama-server" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:31.079+08:00 level=INFO source=llama_server.go:1248 msg="waiting for llama-server to start responding"
time=2026-10-04T15:25:31.080+08:00 level=INFO source=llama_server.go:1303 msg="waiting for llama-server to become available" status="llm server not responding"
0.00.355.251 E ggml_cuda_init: failed to initialize CUDA: unknown error
cmn  common_param: common_params_print_info: build 1 (cb295bf59) with Clang 21.1.8 for Linux aarch64
cmn  common_param: common_params_print_info: verbosity = 4 (adjust with the `-lv N` CLI arg)
cmn  common_param: device_info:
cmn  common_param:   - CPU     : CPU (7546 MiB, 7546 MiB free)
cmn  common_param: system_info: n_threads = 6 (n_threads_batch = 6) / 6 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | DOTPROD = 1 | LLAMAFILE = 1 | REPACK = 1 | CUDA : ARCHS = 750,800,860,870,890,900,1000,1030,1100,1200,1210 | USE_GRAPHS = 1 | PEER_MAX_BATCH_SIZE = 128 | 
srv          init: using 5 threads for HTTP server
srv          init: The UI is disabled
srv          init: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
srv         start: binding port with default address family
srv    load_model: loading model '/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb'
srv    load_model: local path '/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb'
cmn  common_init_: fitting params to device memory ...
cmn  common_init_: (for bugs during this step try to reproduce them with -fit off, or provide --verbose logs if the bug only occurs with -fit on)
common_params_fit_impl: getting device memory data for initial parameters:
time=2026-10-04T15:25:31.586+08:00 level=INFO source=llama_server.go:1303 msg="waiting for llama-server to become available" status="llm server loading model"
common_memory_breakdown_print: | memory breakdown [MiB] | total   free    self   model   context   compute    unaccounted |
common_memory_breakdown_print: |   - Host               |                  368 =   144 +     168 +      56                |
common_memory_breakdown_print: |   - CPU_REPACK         |                 1339 =  1339 +       0 +       0                |
common_params_fit_impl: projected to use 368 MiB of host memory vs. 7546 MiB of total host memory
common_params_fit_impl: will leave 7177 >= 1024 MiB of system memory, no changes needed
common_fit_params: successfully fit params to free device memory
common_fit_params: fitting params to free memory took 0.55 seconds
llama_model_loader: loaded meta data with 76 key-value pairs and 381 tensors from /home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = llama
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                     general.sampling.top_p f32              = 0.950000
llama_model_loader: - kv   3:                      general.sampling.temp f32              = 1.000000
llama_model_loader: - kv   4:                               general.name str              = MiniCPM5 2B Heretic Abliterated
llama_model_loader: - kv   5:                           general.finetune str              = heretic-abliterated
llama_model_loader: - kv   6:                           general.basename str              = MiniCPM5
llama_model_loader: - kv   7:                         general.size_label str              = 2B
llama_model_loader: - kv   8:                            general.license str              = apache-2.0
llama_model_loader: - kv   9:                      general.dataset.count u32              = 8
llama_model_loader: - kv  10:                     general.dataset.0.name str              = Ultra FineWeb
llama_model_loader: - kv  11:             general.dataset.0.organization str              = Openbmb
llama_model_loader: - kv  12:                 general.dataset.0.repo_url str              = https://huggingface.co/openbmb/Ultra-...
llama_model_loader: - kv  13:                     general.dataset.1.name str              = UltraX Preview
llama_model_loader: - kv  14:             general.dataset.1.organization str              = Openbmb
llama_model_loader: - kv  15:                 general.dataset.1.repo_url str              = https://huggingface.co/openbmb/UltraX...
llama_model_loader: - kv  16:                     general.dataset.2.name str              = Ultra FineWeb L3
llama_model_loader: - kv  17:             general.dataset.2.organization str              = Openbmb
llama_model_loader: - kv  18:                 general.dataset.2.repo_url str              = https://huggingface.co/openbmb/Ultra-...
llama_model_loader: - kv  19:                     general.dataset.3.name str              = UltraData Math
llama_model_loader: - kv  20:             general.dataset.3.organization str              = Openbmb
llama_model_loader: - kv  21:                 general.dataset.3.repo_url str              = https://huggingface.co/openbmb/UltraD...
llama_model_loader: - kv  22:                     general.dataset.4.name str              = UltraData Code
llama_model_loader: - kv  23:             general.dataset.4.organization str              = Openbmb
llama_model_loader: - kv  24:                 general.dataset.4.repo_url str              = https://huggingface.co/openbmb/UltraD...
llama_model_loader: - kv  25:                     general.dataset.5.name str              = UltraData SFT 2605
llama_model_loader: - kv  26:                  general.dataset.5.version str              = 2605
llama_model_loader: - kv  27:             general.dataset.5.organization str              = Openbmb
llama_model_loader: - kv  28:                 general.dataset.5.repo_url str              = https://huggingface.co/openbmb/UltraD...
llama_model_loader: - kv  29:                     general.dataset.6.name str              = UltraData SFT Agent 2609
llama_model_loader: - kv  30:                  general.dataset.6.version str              = 2609
llama_model_loader: - kv  31:             general.dataset.6.organization str              = Openbmb
llama_model_loader: - kv  32:                 general.dataset.6.repo_url str              = https://huggingface.co/openbmb/UltraD...
llama_model_loader: - kv  33:                     general.dataset.7.name str              = UltraData RL 2609
llama_model_loader: - kv  34:                  general.dataset.7.version str              = 2609
llama_model_loader: - kv  35:             general.dataset.7.organization str              = Openbmb
llama_model_loader: - kv  36:                 general.dataset.7.repo_url str              = https://huggingface.co/openbmb/UltraD...
llama_model_loader: - kv  37:                               general.tags arr[str,14]      = ["minicpm", "minicpm5", "llama", "tex...
llama_model_loader: - kv  38:                          general.languages arr[str,2]       = ["en", "zh"]
llama_model_loader: - kv  39:                          llama.block_count u32              = 42
llama_model_loader: - kv  40:                       llama.context_length u32              = 131072
llama_model_loader: - kv  41:                     llama.embedding_length u32              = 2048
llama_model_loader: - kv  42:                  llama.feed_forward_length u32              = 6144
llama_model_loader: - kv  43:                 llama.attention.head_count u32              = 16
llama_model_loader: - kv  44:              llama.attention.head_count_kv u32              = 2
llama_model_loader: - kv  45:                       llama.rope.freq_base f32              = 5000000.000000
llama_model_loader: - kv  46:     llama.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  47:                 llama.attention.key_length u32              = 128
llama_model_loader: - kv  48:               llama.attention.value_length u32              = 128
llama_model_loader: - kv  49:                           llama.vocab_size u32              = 130560
llama_model_loader: - kv  50:                 llama.rope.dimension_count u32              = 128
llama_model_loader: - kv  51:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  52:                         tokenizer.ggml.pre str              = minicpm5
llama_model_loader: - kv  53:                      tokenizer.ggml.tokens arr[str,130560]  = ["<s>", "</s>", "<tool_call>", "</too...
llama_model_loader: - kv  54:                  tokenizer.ggml.token_type arr[i32,130560]  = [3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 3, 3, ...
llama_model_loader: - kv  55:                      tokenizer.ggml.merges arr[str,129794]  = ["Ġ t", "h e", "Ġ a", "i n", "Ġ Ġ...
llama_model_loader: - kv  56:                tokenizer.ggml.bos_token_id u32              = 0
llama_model_loader: - kv  57:                tokenizer.ggml.eos_token_id u32              = 1
llama_model_loader: - kv  58:            tokenizer.ggml.unknown_token_id u32              = 130074
llama_model_loader: - kv  59:            tokenizer.ggml.padding_token_id u32              = 1
llama_model_loader: - kv  60:               tokenizer.ggml.add_bos_token bool             = true
llama_model_loader: - kv  61:               tokenizer.ggml.add_sep_token bool             = false
llama_model_loader: - kv  62:                    tokenizer.chat_template str              = {{- bos_token }}{%- if tools %}\n    {...
llama_model_loader: - kv  63:               general.quantization_version u32              = 2
llama_model_loader: - kv  64:                          general.file_type u32              = 15
llama_model_loader: - kv  65:                                general.url str              = https://huggingface.co/mradermacher/M...
llama_model_loader: - kv  66:              mradermacher.quantize_version str              = 2
llama_model_loader: - kv  67:                  mradermacher.quantized_by str              = mradermacher
llama_model_loader: - kv  68:                  mradermacher.quantized_at str              = 2026-09-08T16:43:47+02:00
llama_model_loader: - kv  69:                  mradermacher.quantized_on str              = back
llama_model_loader: - kv  70:                         general.source.url str              = https://huggingface.co/insraq/MiniCPM...
llama_model_loader: - kv  71:                  mradermacher.convert_type str              = hf
llama_model_loader: - kv  72:                      quantize.imatrix.file str              = MiniCPM5-2B-heretic-abliterated-i1-GG...
llama_model_loader: - kv  73:                   quantize.imatrix.dataset str              = imatrix-training-full-3
llama_model_loader: - kv  74:             quantize.imatrix.entries_count u32              = 294
llama_model_loader: - kv  75:              quantize.imatrix.chunks_count u32              = 312
llama_model_loader: - type  f32:   85 tensors
llama_model_loader: - type q4_K:  253 tensors
llama_model_loader: - type q6_K:   43 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 1.45 GiB (4.95 BPW) 
load: 478 unused tokens
load: printing all EOG tokens:
load:   - 1 ('</s>')
load:   - 130073 ('<|im_end|>')
load: special tokens cache size = 510
load: token to piece cache size = 0.8291 MB
print_info: arch                  = llama
print_info: vocab_only            = 0
print_info: no_alloc              = 0
print_info: n_ctx_train           = 131072
print_info: n_embd_inp            = 2048
print_info: n_embd                = 2048
print_info: n_embd_out            = 2048
print_info: n_layer               = 42
print_info: n_layer_all           = 42
print_info: n_head                = 16
print_info: n_head_kv             = 2
print_info: n_rot                 = 128
print_info: n_swa                 = 0
print_info: is_swa_any            = 0
print_info: n_embd_head_k         = 128
print_info: n_embd_head_v         = 128
print_info: n_gqa                 = 8
print_info: n_embd_k_gqa          = 256
print_info: n_embd_v_gqa          = 256
print_info: f_norm_eps            = 0.0e+00
print_info: f_norm_rms_eps        = 1.0e-06
print_info: f_clamp_kqv           = 0.0e+00
print_info: f_max_alibi_bias      = 0.0e+00
print_info: f_logit_scale         = 0.0e+00
print_info: f_attn_scale          = 0.0e+00
print_info: f_attn_value_scale    = 0.0000
print_info: n_ff                  = 6144
print_info: n_expert              = 0
print_info: n_expert_used         = 0
print_info: n_expert_groups       = 0
print_info: n_group_used          = 0
print_info: causal attn           = 1
print_info: pooling type          = -1
print_info: rope type             = 0
print_info: rope scaling          = linear
print_info: freq_base_train       = 5000000.0
print_info: freq_scale_train      = 1
print_info: n_ctx_orig_yarn       = 131072
print_info: rope_yarn_log_mul     = 0.0000
print_info: rope_finetuned        = unknown
print_info: model type            = ?B
print_info: model params          = 2.52 B
print_info: general.name          = MiniCPM5 2B Heretic Abliterated
print_info: vocab type            = BPE
print_info: n_vocab               = 130560
print_info: n_merges              = 129794
print_info: BOS token             = 0 '<s>'
print_info: EOS token             = 1 '</s>'
print_info: EOT token             = 130073 '<|im_end|>'
print_info: UNK token             = 130074 '<unk>'
print_info: PAD token             = 1 '</s>'
print_info: LF token              = 220 'Ċ'
print_info: FIM PRE token         = 5 '<|fim_prefix|>'
print_info: FIM SUF token         = 7 '<|fim_suffix|>'
print_info: FIM MID token         = 6 '<|fim_middle|>'
print_info: EOG token             = 1 '</s>'
print_info: EOG token             = 130073 '<|im_end|>'
print_info: max token length      = 1024
load_tensors: loading model tensors, this can take a while... (mmap = false, direct_io = false)
load_tensors:          CPU model buffer size =   144.10 MiB
load_tensors:   CPU_REPACK model buffer size =  1339.98 MiB
cmn  common_init_: added </s> logit bias = -inf
cmn  common_init_: added <|im_end|> logit bias = -inf
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 4096
llama_context: n_ctx_seq     = 4096
llama_context: n_batch       = 512
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = auto
llama_context: kv_unified    = false
llama_context: freq_base     = 5000000.0
llama_context: freq_scale    = 1
llama_context: n_rs_seq      = 0
llama_context: n_outputs_max = 1
llama_context: n_ctx_seq (4096) < n_ctx_train (131072) -- the full capacity of the model will not be utilized
llama_context:        CPU  output buffer size =     0.50 MiB
llama_kv_cache:        CPU KV buffer size =   168.00 MiB
llama_kv_cache: size =  168.00 MiB (  4096 cells,  42 layers,  1/1 seqs), K (f16):   84.00 MiB, V (f16):   84.00 MiB
llama_kv_cache: attn_rot_k = 0, n_embd_head_k_all = 128
llama_kv_cache: attn_rot_v = 0, n_embd_head_k_all = 128
sched_reserve: reserving ...
sched_reserve: Flash Attention was auto, set to enabled
sched_reserve: resolving fused Gated Delta Net support:
sched_reserve: fused Gated Delta Net (autoregressive) enabled
sched_reserve: fused Gated Delta Net (chunked) enabled
sched_reserve:        CPU compute buffer size =    56.01 MiB
sched_reserve: graph nodes  = 1308
sched_reserve: graph splits = 1
sched_reserve: reserve took 62.74 ms, sched copies = 1
cmn  common_init_: warming up the model with an empty run - please wait ... (--no-warmup to disable)
srv    load_model: initializing, n_slots = 1, n_ctx_slot = 4096, kv_unified = 'false'
spec common_specu: no implementations specified for speculative decoding
slot   load_model: id  0 | task -1 | new slot, n_ctx = 4096
srv    load_model: prompt cache is enabled, size limit: 8192 MiB
srv    load_model: use `--cache-ram 0` to disable the prompt cache
srv    load_model: for more info see https://github.com/ggml-org/llama.cpp/pull/16391
srv    load_model: context checkpoints enabled, max = 32, min spacing = 8192
srv          init: idle slots will be saved to prompt cache upon starting a new task
srv          init: init: chat template, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
<think>

</think>

Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
<think>
'
srv          init: init: chat template, thinking = 1
srv          init: chat template supports preserving reasoning, consider enabling it via --reasoning-preserve
srv  llama_server: model loaded
srv  llama_server: listening on http://127.0.0.1:39025
srv  update_slots: all slots are idle
time=2026-10-04T15:25:52.560+08:00 level=INFO source=llama_server.go:1315 msg="llama-server started in 21.47 seconds"
time=2026-10-04T15:25:52.936+08:00 level=INFO source=images.go:373 msg="template selection" model=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M selected=gguf_chat_template renderer="" parser="" go_template=[completion] chat_template="[tools thinking completion]" harmony=null renderer_parser=null
time=2026-10-04T15:25:52.938+08:00 level=INFO source=sched.go:739 msg="loaded runners" count=1
time=2026-10-04T15:25:52.940+08:00 level=INFO source=llama_server.go:1248 msg="waiting for llama-server to start responding"
time=2026-10-04T15:25:52.944+08:00 level=INFO source=llama_server.go:1315 msg="llama-server started in 21.86 seconds"
time=2026-10-04T15:25:52.944+08:00 level=DEBUG source=sched.go:751 msg="finished setting up" runner.name=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M runner.size="1.7 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=3614723 runner.model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb runner.num_ctx=4096
[GIN] 2026/10/04 - 15:25:52 | 200 | 22.278350667s |       127.0.0.1 | POST     "/api/generate"
time=2026-10-04T15:25:52.956+08:00 level=DEBUG source=sched.go:759 msg="context for request finished"
time=2026-10-04T15:25:52.956+08:00 level=DEBUG source=sched.go:400 msg="runner with non-zero duration has gone idle, adding timer" runner.name=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M runner.size="1.7 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=3614723 runner.model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb runner.num_ctx=4096 duration=5m0s
time=2026-10-04T15:25:52.956+08:00 level=DEBUG source=sched.go:418 msg="after processing request finished event" runner.name=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M runner.size="1.7 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=3614723 runner.model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb runner.num_ctx=4096 refCount=0
time=2026-10-04T15:25:59.421+08:00 level=DEBUG source=sched.go:1394 msg="evaluating already loaded" model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb
time=2026-10-04T15:25:59.464+08:00 level=DEBUG source=llama_server.go:1835 msg="llama-server chat request" messages=1 tools=0
srv    operator(): chat format: peg-native
slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
srv  get_availabl: updating prompt cache
srv          load:  - looking for better prompt, base f_keep = -1.000, sim = 0.000
srv        update:  - cache state: 0 prompts, 0.000 MiB (limits: 8192.000 MiB, 4096 tokens, 8589934592 est)
srv  get_availabl: prompt cache update took 0.33 ms
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> ?min-p -> ?xtc -> temp-ext -> dist 
slot launch_slot_: id  0 | task -1 | sampler params: 
        repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
        dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = 4096
        top_k = 40, top_p = 0.900, min_p = 0.000, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.800
        mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000, adaptive_target = -1.000, adaptive_decay = 0.900
slot launch_slot_: id  0 | task 0 | processing task, is_child = 0
slot   operator(): id  0 | task 0 | new prompt, n_ctx_slot = 4096, n_keep = 4, task.n_tokens = 12
slot   operator(): id  0 | task 0 | cached n_tokens = 0, memory_seq_rm [0, end)
srv  stream_sessi: conv_id= (empty=1)
slot print_timing: id  0 | task 0 | prompt processing, n_tokens =      1, progress = 0.08, t =   8.82 s / 0.11 tokens per second
slot   operator(): id  0 | task 0 | cached n_tokens = 1, memory_seq_rm [1, end)
slot init_sampler: id  0 | task 0 | init sampler, took 4.68 ms, tokens: text = 12, total = 12
[GIN] 2026/10/04 - 15:26:17 | 200 | 18.389665931s |       127.0.0.1 | POST     "/api/chat"
time=2026-10-04T15:26:17.433+08:00 level=DEBUG source=sched.go:495 msg="context for request finished" runner.name=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M runner.size="1.7 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=3614723 runner.model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb runner.num_ctx=4096
time=2026-10-04T15:26:17.444+08:00 level=DEBUG source=sched.go:400 msg="runner with non-zero duration has gone idle, adding timer" runner.name=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M runner.size="1.7 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=3614723 runner.model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb runner.num_ctx=4096 duration=5m0s
time=2026-10-04T15:26:17.445+08:00 level=DEBUG source=sched.go:418 msg="after processing request finished event" runner.name=hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M runner.size="1.7 GiB" runner.vram="0 B" runner.parallel=1 runner.pid=3614723 runner.model=/home/user/.ollama/models/blobs/sha256-86e63a5218a10e095f5ed1dcf2ef1c1a9dd9af6416f4d8e9288c7562f9edd2cb runner.num_ctx=4096 refCount=0
srv          stop: cancel task, id_task = 0
slot      release: id  0 | task 0 | stop processing: n_tokens = 25, truncated = 0
srv  update_slots: all slots are idle
[GIN] 2026/10/04 - 15:26:24 | 200 |    2.911481ms |       127.0.0.1 | HEAD     "/"
[GIN] 2026/10/04 - 15:26:24 | 200 |    2.257318ms |       127.0.0.1 | GET      "/api/ps"

and also this:

 user   ~    ollama ls
NAME                                                                 ID              SIZE      MODIFIED     
hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M    6263d7d0bd60    1.6 GB    2 weeks ago     
gemma4:e2b                                                           7fbdbf8f5e45    7.2 GB    3 weeks ago     
gemma4:e4b                                                           c6eb396dbd59    9.6 GB    3 weeks ago     
minimax-m2.7:cloud                                                   06daa293c105    -         4 months ago    
minimax-m2.5:cloud                                                   c0d5751c800f    -         4 months ago    
 user   ~    ollama run hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M
>>> hello
Thinking...
Hmm, the user said "hello". This is a simple^C>>> /bye
 user   ~    ollama ps
NAME                                                                 ID              SIZE      PROCESSOR    CONTEXT    UNTIL              
hf.co/mradermacher/MiniCPM5-2B-heretic-abliterated-i1-GGUF:Q4_K_M    6263d7d0bd60    1.8 GB    100% CPU     4096       4 minutes from now    
 user   ~   
9

Hi,

According to the message above, please try to validate the CUDA functionality with the below sample first:

Thanks.

10

Hello, here is the result:

 user   deviceQuery   master ≡  ?14    ./deviceQuery 
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 999
-> unknown error
Result = FAIL
 user   deviceQuery   master ≡  ?14    sudo ./deviceQuery
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 999
-> unknown error
Result = FAIL
 user   deviceQuery   master ≡  ?14   
11

Hi,

It looks like your CUDA is not working.
Could you share the way you set up the device?

What is the output of the command below?

$ nvidia-smi

Thanks.

12

Hi cylin577,

Your deviceQuery results confirm that CUDA initialization is failing outside Ollama as well. Since it also fails with sudo, we should check the installation and driver logs before changing more Ollama settings.

To follow up on @AastaLLL question, could you provide the following?

  1. How did you install JetPack 7.2.1—through the Jetson ISO, SDK Manager, or an upgrade of an existing installation? Was CUDA installed separately afterward? Please also share the current output of nvidia-smi, cat /etc/nv_tegra_release, and uname -r.

  2. After rebooting, close LM Studio and stop Ollama if it starts automatically:

    sudo systemctl stop ollama

    Then, from the directory containing your deviceQuery executable, run:

    free -h
./deviceQuery

  3. If it still fails, capture the kernel log immediately afterward:

    sudo journalctl -k -b --no-pager > kernel-current-boot.log

    Please attach that file so we can check for GPU initialization or memory-allocation errors.

If the kernel log does not explain the failure, this additional command will show which CUDA driver library the sample actually loads:

LD_DEBUG=libs,files ./deviceQuery > deviceQuery-loader.log 2>&1

Please attach deviceQuery-loader.log as well. This is more useful than relying on the CUDA version in PATH or the library cache.

These results should help narrow down the failure before deciding whether any packages need replacing.

Best regards,
Alejandro Godínez
Embedded Software Engineer, RidgeRun

Contact us: support@ridgerun.com
Developers Wiki: https://developer.ridgerun.com
Website: https://www.ridgerun.com

14

I use the NVIDIA SDK Manager to flash my Jetson, for the first few month it worked perfectly, but just a few weeks ago this issue starts to happen

 user   ~    nvidia-smi
Mon Oct 12 21:25:05 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.78                 Driver Version: 595.78         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Orin (nvgpu)                  N/A  |   N/A              N/A |                  N/A |
| N/A   N/A  N/A             N/A  /  N/A  | Not Supported          |     N/A          N/A |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
 user   ~   
15
  1. SDK Manager
 user   ~     nvidia-smi
Mon Oct 12 21:27:37 2026       
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 595.78                 Driver Version: 595.78         CUDA Version: 13.2     |
+-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  Orin (nvgpu)                  N/A  |   N/A              N/A |                  N/A |
| N/A   N/A  N/A             N/A  /  N/A  | Not Supported          |     N/A          N/A |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI              PID   Type   Process name                        GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+
 user   ~    cat /etc/nv_tegra_release
# R39 (release), REVISION: 2.1, GCID: 46758480, BOARD: generic, EABI: aarch64, DATE: Fri Aug  7 05:54:22 AM UTC 2026
# KERNEL_VARIANT: oot
TARGET_USERSPACE_LIB_DIR=nvidia
TARGET_USERSPACE_LIB_DIR_PATH=usr/lib/aarch64-linux-gnu/nvidia
 user   ~    uname -r
6.8.12-1021-tegra
 user   ~    cd cuda-samples/cpp/1_Utilities/deviceQuery
 user   deviceQuery   master ≡  ?14    free -h
./deviceQuery
               total        used        free      shared  buff/cache   available
Mem:           7.4Gi       6.6Gi       209Mi        77Mi       906Mi       823Mi
Swap:           31Gi        10Gi        21Gi
./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

cudaGetDeviceCount returned 999
-> unknown error
Result = FAIL
 user   deviceQuery   master ≡  ?14    sudo journalctl -k -b --no-pager > kernel-current-boot.log
 user   deviceQuery   master ≡  ?15    ls -la
total 136
drwxrwxr-x 4 user user  4096 Oct 12 21:28 .
drwxrwxr-x 6 user user  4096 Jul  1 17:32 ..
drwxrwxr-x 3 user user  4096 Jul  1 17:32 CMakeFiles
-rw-rw-r-- 1 user user 13572 Jul  1 17:32 cmake_install.cmake
-rw-rw-r-- 1 user user  1356 Jul  1 17:32 CMakeLists.txt
-rwxrwxr-x 1 user user 72408 Jul  1 17:32 deviceQuery
-rw-rw-r-- 1 user user 15433 Jul  1 17:32 deviceQuery.cpp
-rw-rw-r-- 1 user user 47062 Oct 12 21:28 kernel-current-boot.log
-rw-rw-r-- 1 user user  8161 Jul  1 17:32 Makefile
-rw-rw-r-- 1 user user  1503 Jul  1 17:32 README.md
drwxrwxr-x 2 user user  4096 Jul  1 17:32 .vscode
 user   deviceQuery   master ≡  ?15    ls -lah
total 136K
drwxrwxr-x 4 user user 4.0K Oct 12 21:28 .
drwxrwxr-x 6 user user 4.0K Jul  1 17:32 ..
drwxrwxr-x 3 user user 4.0K Jul  1 17:32 CMakeFiles
-rw-rw-r-- 1 user user  14K Jul  1 17:32 cmake_install.cmake
-rw-rw-r-- 1 user user 1.4K Jul  1 17:32 CMakeLists.txt
-rwxrwxr-x 1 user user  71K Jul  1 17:32 deviceQuery
-rw-rw-r-- 1 user user  16K Jul  1 17:32 deviceQuery.cpp
-rw-rw-r-- 1 user user  46K Oct 12 21:28 kernel-current-boot.log
-rw-rw-r-- 1 user user 8.0K Jul  1 17:32 Makefile
-rw-rw-r-- 1 user user 1.5K Jul  1 17:32 README.md
drwxrwxr-x 2 user user 4.0K Jul  1 17:32 .vscode
 user   deviceQuery   master ≡  ?15    LD_DEBUG=libs,files ./deviceQuery > deviceQuery-loader.log 2>&1
 user   deviceQuery   master ≡  ?16    ls -lah
total 160K
drwxrwxr-x 4 user user 4.0K Oct 12 21:28 .
drwxrwxr-x 6 user user 4.0K Jul  1 17:32 ..
drwxrwxr-x 3 user user 4.0K Jul  1 17:32 CMakeFiles
-rw-rw-r-- 1 user user  14K Jul  1 17:32 cmake_install.cmake
-rw-rw-r-- 1 user user 1.4K Jul  1 17:32 CMakeLists.txt
-rwxrwxr-x 1 user user  71K Jul  1 17:32 deviceQuery
-rw-rw-r-- 1 user user  16K Jul  1 17:32 deviceQuery.cpp
-rw-rw-r-- 1 user user  22K Oct 12 21:28 deviceQuery-loader.log
-rw-rw-r-- 1 user user  46K Oct 12 21:28 kernel-current-boot.log
-rw-rw-r-- 1 user user 8.0K Jul  1 17:32 Makefile
-rw-rw-r-- 1 user user 1.5K Jul  1 17:32 README.md
drwxrwxr-x 2 user user 4.0K Jul  1 17:32 .vscode
 user   deviceQuery   master ≡  ?16   

kernel-current-boot.log (46.0 KB)
deviceQuery-loader.log (21.7 KB)

16

Hi cylin577,

Thanks for collecting the logs. They show a GPU error that is useful to investigate:

At 21:14 the nvgpu driver reported SM exceptions, an invalid pte MMU fault, and FECS errors. Your deviceQuery test was run later, so I cannot yet tell whether that earlier event left the GPU in a bad state or whether CUDA fails from a fresh boot.

The loader log also confirms that deviceQuery is using the Jetson libcuda.so.1 under /opt/nvidia/l4t-gpu-libs/nvgpu/. It then searches for libnvcucompat.so without finding it. Could you try these steps?

  1. Fully shut down and power the Jetson back on. Before opening LM Studio or running Ollama, run ./deviceQuery again and tell us whether it passes.

  2. Check whether the compatibility library exists:

    ls -l /usr/lib/aarch64-linux-gnu/libnvcucompat.so /usr/lib/aarch64-linux-gnu/nvidia/libnvcucompat.so
ldconfig -p | grep libnvcucompat

  3. If the file exists under the nvidia directory but deviceQuery still fails, try this temporary library-path test from the deviceQuery directory:

    LD_LIBRARY_PATH=/usr/lib/aarch64-linux-gnu/nvidia:/usr/local/cuda-13.2/lib64 ./deviceQuery

Please share those results and, if the test still fails after the fresh boot, the new kernel messages from that boot. This will tell us whether to focus on the GPU fault or on the library setup before changing packages.

One more detail that may help: is this the NVIDIA P3768 developer kit carrier or a custom carrier board? If it is custom, please share its model, the BSP/board configuration used when flashing, and whether the carrier, power supply, or BSP changed shortly before the issue began. The kernel log contains GPU faults, so these details would help us check the board integration without assuming it caused the CUDA failure.

Best regards,
Alejandro Godínez
Embedded Software Engineer, RidgeRun

Contact us: support@ridgerun.com
Developers Wiki: https://developer.ridgerun.com
Website: https://www.ridgerun.com