RTX 5060 Ti eGPU (Thunderbolt 4) — CUDA hard-lock, GSP firmware hang, all workarounds failed

Hi NVIDIA team,

I’m experiencing the same CUDA hard-lock issue reported in several other threads (e.g. GitHub #974, #979), but on a different host/enclosure combination, which I hope helps narrow down the root cause.

Configuration

• GPU: NVIDIA GeForce RTX 5060 Ti (GB206), Gigabyte AORUS AI BOX, 16 GB VRAM
• Host: Lenovo ThinkPad, Intel Thunderbolt 4
• Driver: nvidia-open 610.43.03
• OS: Arch Linux (latest stable kernel)

What works

• GPU is detected by the system and visible via boltctl
nvidia-smi reports correct model, VRAM, temperature, and power at idle
• Driver loads without errors

What fails

• Any CUDA operation (Ollama inference, PyTorch, even python3 -c "import torch; torch.zeros(1, device='cuda')") causes an immediate system hard-lock
• No kernel panic, no Xid error, no SysRq response — only a power cycle recovers the system
• Kernel log before the lock shows: kfspWaitForResponse: FSP command timed out

Workarounds tried

NVreg_EnableGpuFirmware=0 — ignored on open kernel modules (confirmed by NVIDIA staff)
• Blacklisting nvidia_drm + nvidia_modeset (compute-only)
• Disabling ASPM, enabling native PCIe ports, disabling Resizable BAR
• Early-loading Thunderbolt module in initramfs
• Multiple driver versions tested

Observation
The pattern appears consistent: TB4 host + Blackwell eGPU = CUDA hard-lock. TB5 + Blackwell reportedly works in some configurations, suggesting the issue may be related to Thunderbolt PCIe tunneling bandwidth or latency rather than the GPU itself.

Request

  1. Is NVIDIA aware of this issue with Blackwell + Thunderbolt on Linux?
  2. Is there an internal tracking bug or ETA for a fix?
  3. Are there any additional debug flags that could help isolate the GSP communication failure over Thunderbolt?

Happy to provide nvidia-bug-report.log.gz or any additional diagnostics.

Thank you.

Blackwell generally rarely works with TB3, TB4 or USB4v1 (on any OS), probably due to its intolerance for signal latency. Among USB-C based adapters, it only works somewhat reliably with TB5, which has its own problems on Linux.

Nvidia shows almost zero interest in fixing standard desktop Linux bugs, so I wouldn’t count on any eGPU related fixes in any foreseeable future.

@guoxh, I’ve noticed on Github, that you’ve managed to make it work using similar workarounds like in case of TB5. Would you care to create a build on egpu.io? This would be an important data point for others. Thanks!

@morgwai666 Yes, confirmed — it works. And notably: no driver patches needed at all.

Hardware: RTX 5060 Ti (GB206, 16GB) + AORUS Gaming Box + Lenovo ThinkPad (TB4)
Driver: stock nvidia-open 610.43.03 (Arch Linux)
Result: stable CUDA inference, GPU-Util 90%, 135W/180W under sustained load, no freeze

Root cause: PCIe bridge LnkCtl2 HW autonomous speed negotiation oscillating Gen3↔️Gen4 → GSP firmware lockdown → hard-lock on any CUDA operation.

Fix (pure system config, zero driver changes):

  1. setpci to lock bridge LnkCtl2 bit5 (HW Autonomous Speed Disable) + Target=Gen3 — must run before nvidia.ko loads
  2. aorus.service (systemd oneshot) — applies the PCIe cap automatically at boot, before the driver loads
  3. Kernel params: iommu=off intel_iommu=off pcie_aspm.policy=performance thunderbolt.clx=0 pcie_port_pm=off
  4. modprobe.d — blacklist nvidia modules and use install nvidia /bin/false to prevent early auto-load
  5. udev rule — disable RTX 5060 Ti HDMI audio function to prevent it from claiming PCIe bandwidth

Step-by-step with all file contents is in the jciolek/aorus-5090-egpu repo ( GitHub - jciolek/aorus-5090-egpu · GitHub ), adapted for RTX 5060 Ti Device ID 0x2d04. Happy to cross-post to egpu.io.

Work in cachyos on firebat mn56 with 8745hs via front usb-c.

Here is script fo this setup what i done with ai after all week of testing.

Bash

#!/usr/bin/env bash

echo "=== 1. Rescanning PCIe bus / Skanowanie magistrali PCIe ==="
echo 1 | sudo tee /sys/bus/pci/rescan
sleep 1

echo "=== 2. Disabling ASPM on PCIe Bridge & eGPU / Wyłączenie ASPM (Oszczędzania energii) ==="
sudo setpci -s 08:00.0 CAP_EXP+10.w=0000
sudo setpci -s 09:00.0 CAP_EXP+10.w=0000

echo "=== 3. Forcing PCIe Gen3 (0003) on BOTH sides / Wymuszenie PCIe Gen3 po OBU stronach ==="
# Parent Bridge / Mostek nadrzędny
sudo setpci -s 08:00.0 CAP_EXP+30.w=0003
sudo setpci -s 08:00.0 CAP_EXP+10.w=0020

# eGPU Card / Karta eGPU (RTX 5060 Ti)
sudo setpci -s 09:00.0 CAP_EXP+30.w=0003
sudo setpci -s 09:00.0 CAP_EXP+10.w=0020
sleep 1

echo "=== 4. LnkCtl2 Registers after configuration / Rejestr LnkCtl2 po zmianie: ==="
echo "Bridge / Mostek 08:00.0: $(sudo setpci -s 08:00.0 CAP_EXP+30.w)"
echo "Card / Karta   09:00.0: $(sudo setpci -s 09:00.0 CAP_EXP+30.w)"

echo "=== 5. Loading NVIDIA Drivers / Ładowanie sterownika NVIDIA ==="
sudo modprobe nvidia nvidia_uvm nvidia_modeset nvidia_drm

echo "=== 6. Setting Power Limits & Disabling Auto-PM / Ustawianie limitów mocy i wyłączenie Auto-PM ==="
sudo nvidia-smi -pm 1
sudo nvidia-smi -pl 150

echo "=== 7. Verifying PCIe Link Speed / Sprawdzić prędkość linku: ==="
sudo lspci -vv -s 09:00.0 | grep -iE "LnkSta:|LnkCtl2:"

powerprofilesctl set performance
sudo nvidia-smi -pl 180
nvidia-smi

echo "=== 8. Detaching iGPU from PCIe bus / Odcinanie iGPU z szyny PCIe ==="
echo 1 | sudo tee /sys/bus/pci/devices/0000:c7:00.0/remove

echo "=== 9. Restoring full eGPU power limit (180W) / Przywracanie pełnej mocy eGPU (180W) ==="
sudo nvidia-smi -pl 180

i-gpu need to be killed because is sharing memory with egpu and fps with igpu are so slow. Egpu is in gen1 mode in desktop and gen3 in games - stable.

Script work for me and i can play games.

STATUS: UNTESTED CONCEPTUAL DRAFT (WIP)

Note: This is an unverified conceptual translation of the AMD fix adapted (by AI) for Intel topology. Not yet validated on physical Intel hardware. Testing and feedback are welcome.

Intel Thunderbolt 3/4 + NVIDIA eGPU Setup & Initializer Guide

⚠️ DISCLAIMER / WARNING (READ FIRST):

  • This guide and the provided scripts are experimental, community-developed workarounds intended for advanced users and technical testing only.

  • Modifying low-level PCIe configuration registers (setpci) and altering module loading behavior can lead to kernel lockups, boot loops, or unbootable systems.

  • DO NOT USE THIS ON YOUR DAILY DRIVER / PRODUCTION MACHINE.

  • Always test these scripts on a separate test installation, dedicated testing partition, or expendable test drive with full backups in place.

  • The author assumes no responsibility for hardware instability, data loss, or system corruption.

System Compatibility

✅ Supported (Mutable / Traditional Linux Distributions):

  • Arch Linux / Arch-based: CachyOS (Recommended), Arch Linux, EndeavourOS, Manjaro (mkinitcpio).

  • Fedora / RHEL-based: Fedora Workstation (traditional), Nobara Linux, RHEL/Rocky (dracut).

  • Debian / Ubuntu-based: Ubuntu Desktop (22.04 / 24.04+), Debian, Pop!_OS, Linux Mint (update-initramfs).

  • openSUSE: Tumbleweed / Leap (dracut).

❌ NOT Supported:

  • Immutable / Atomic Systems: Bazzite, Fedora Silverblue, SteamOS 3.x.

  • Non-systemd / Non-udev distributions: Alpine Linux, Void Linux, Gentoo.

📦 Required Packages:

  • pciutils (provides lspci and setpci)

  • bolt (provides boltctl and boltd for Thunderbolt authorization)

  • nvidia proprietary or open kernel modules (nvidia-open-dkms / nvidia-dkms)

Step 1: System Pre-Configuration (RUN ONCE WITHOUT eGPU CONNECTED)

  • Filename: setup-system-intel.sh

  • When to run: ONCE, BEFORE PLUGGING IN THE eGPU.

  • What it does:

    1. Installs driver interceptors in /etc/modprobe.d/ (install nvidia /bin/false) to prevent the kernel/udev from auto-loading nvidia.ko prematurely during hotplug.

    2. Configures persistent ASPM performance policy.

    3. Enables boltd for Thunderbolt authentication.

    4. Regenerates initramfs so early boot stages respect the driver locks.

Code: setup-system-intel.sh

Bash

#!/usr/bin/env bash
# ==============================================================================
# Step 1: Host System Pre-Configuration for Intel Thunderbolt eGPU
# IMPORTANT: Run this script ONCE WITHOUT the eGPU plugged in.
# ==============================================================================
set -euo pipefail

if [ "$EUID" -ne 0 ]; then
    echo "[-] Error: This script must be run as root: sudo $0" >&2
    exit 1
fi

echo "========================================================"
echo "    Intel eGPU Host System Pre-Configuration Setup      "
echo "========================================================"

# 1. Lock driver auto-loading in modprobe.d
echo "[+] 1/4: Installing driver locks into /etc/modprobe.d/..."
cat << 'EOF' > /etc/modprobe.d/99-egpu-intel-lock.conf
# Block automatic loading of open/proprietary NVIDIA drivers
blacklist nouveau
blacklist nvidia
blacklist nvidia_drm
blacklist nvidia_modeset
blacklist nvidia_uvm

options nouveau modeset=0
options thunderbolt host_reset=0

# Intercept module requests to prevent race condition during hotplug
install nvidia /bin/false
install nvidia-drm /bin/false
install nvidia-modeset /bin/false
install nvidia-uvm /bin/false
EOF

# 2. Configure ASPM performance policy persistence
echo "[+] 2/4: Configuring PCIe ASPM defaults..."
cat << 'EOF' > /etc/tmpfiles.d/pcie-aspm-performance.conf
w- /sys/module/pcie_aspm/parameters/policy - - - - performance
EOF

# 3. Enable boltd for Thunderbolt management
echo "[+] 3/4: Ensuring Thunderbolt daemon (boltd) is active..."
if command -v boltctl >/dev/null 2>&1; then
    systemctl enable --now bolt.service 2>/dev/null || true
fi

# 4. Regenerate initramfs
echo "[+] 4/4: Updating initramfs to prevent driver embedding..."
if command -v mkinitcpio >/dev/null 2>&1; then
    mkinitcpio -P
elif command -v dracut >/dev/null 2>&1; then
    dracut -f --regenerate-all
elif command -v update-initramfs >/dev/null 2>&1; then
    update-initramfs -u -k all
fi

echo "========================================================"
echo "[✓] Pre-configuration complete. Reboot your machine once."
echo "========================================================"

Step 2: Runtime Initialization (RUN AFTER CONNECTING eGPU)

  • Filename: egpu-intel-attach.sh

  • When to run: AFTER CONNECTING THE THUNDERBOLT CABLE.

  • What it does:

    1. Authorizes Thunderbolt link via boltctl.

    2. Rescans PCIe bus.

    3. Traverses the entire upstream bridge chain up to the Intel Root Port.

    4. Disables ASPM (CAP_EXP+10), clears L1 Substates (ECAP_1E), and locks link speed with Hardware Autonomous Speed Disable (HASD Bit 5 in LnkCtl2).

    5. Performs physical Link Retraining on all parent bridges.

    6. Safely loads NVIDIA modules (modprobe --ignore-install nvidia).

Code: egpu-intel-attach.sh

Bash

#!/usr/bin/env bash
# ==============================================================================
# Step 2: eGPU PCIe Link Training & Runtime Initializer (Intel TB4)
# IMPORTANT: Run this script AFTER connecting the Thunderbolt cable.
# ==============================================================================
set -euo pipefail

TARGET_GEN="${1:-3}"    # Default: Gen3 (use "4" for Gen4)
POWER_LIMIT="${2:-180}" # Power limit in Watts

if [ "$EUID" -ne 0 ]; then
    echo "[-] Error: Run as root: sudo $0 [gen 3|4] [power_limit]" >&2
    exit 1
fi

echo "========================================================"
echo "         Initializing Intel Thunderbolt eGPU Link       "
echo "========================================================"

# 1. Force ASPM performance mode
if [ -f /sys/module/pcie_aspm/parameters/policy ]; then
    echo performance > /sys/module/pcie_aspm/parameters/policy 2>/dev/null || true
fi

# 2. Authorize Thunderbolt devices
if command -v boltctl >/dev/null 2>&1; then
    echo "[+] Authorizing Thunderbolt devices..."
    for dev in $(boltctl list | grep "uuid:" | awk '{print $2}'); do
        boltctl enroll "$dev" 2>/dev/null || boltctl authorize "$dev" 2>/dev/null || true
    done
    sleep 0.5
fi

# 3. Rescan PCIe bus
echo "[+] Rescanning PCIe bus..."
echo 1 > /sys/bus/pci/rescan
sleep 1

# 4. Locate NVIDIA GPU Endpoint
EGPU_PCI=$(lspci -D | grep -E 'VGA|3D' | grep -i '10de' | awk '{print $1}' | head -n 1)

if [ -z "$EGPU_PCI" ]; then
    echo "[-] Error: No NVIDIA GPU detected on PCIe bus after rescan." >&2
    exit 1
fi

echo "[+] Detected NVIDIA eGPU endpoint: $EGPU_PCI"

# 5. Robust Upstream Bridge Discovery (Walks sysfs up to Root Port)
echo "[+] Resolving parent bridge hierarchy..."
BRIDGES=()
CUR_PATH="/sys/bus/pci/devices/$EGPU_PCI"

while true; do
    PARENT_PATH=$(dirname "$(readlink -f "$CUR_PATH")")
    PARENT_NAME=$(basename "$PARENT_PATH")

    # Stop when reaching PCI host root domain (e.g. pci0000:00) or non-PCI dir
    if [[ "$PARENT_NAME" =~ ^[0-9a-fA-F]{4}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}\.[0-9a-fA-F]$ ]]; then
        BRIDGES+=("$PARENT_NAME")
        CUR_PATH="$PARENT_PATH"
    else
        break
    fi
done

echo "[+] Upstream bridge chain: ${BRIDGES[*]}"

# Target speed hex translation:
# Bit 5 (0x0020) = Hardware Autonomous Speed Disable (HASD)
# Target=Gen3 (0x0003) -> 0x0023 | Target=Gen4 (0x0004) -> 0x0024
if [ "$TARGET_GEN" = "4" ]; then
    LNKCTL2_VAL="0024"
else
    LNKCTL2_VAL="0023"
fi

echo "[+] Applying low-level registers (Target: Gen${TARGET_GEN}, HASD Bit 5 Enabled)..."

# Apply ASPM and LnkCtl2 across the entire path (bridges + GPU endpoint)
ALL_NODES=("${BRIDGES[@]}" "$EGPU_PCI")

for node in "${ALL_NODES[@]}"; do
    # 1. Disable ASPM L0s/L1 in Link Control (CAP_EXP+10)
    setpci -s "$node" CAP_EXP+10.w=0000 2>/dev/null || true

    # 2. Clear L1 PM Substates if supported (ignore if missing)
    setpci -s "$node" ECAP_1E+04.l=00000000 2>/dev/null || true

    # 3. Lock Target Link Speed + Disable Autonomous Speed Negotiation (LnkCtl2)
    setpci -s "$node" CAP_EXP+30.w="$LNKCTL2_VAL" 2>/dev/null || true
done

# Step 6: Trigger Link Retrain ONLY on Downstream Bridges (Spec violation on Endpoints)
echo "[+] Triggering Link Retrain on parent bridges..."
for bridge in "${BRIDGES[@]}"; do
    setpci -s "$bridge" CAP_EXP+10.w=0020 2>/dev/null || true
done

# Allow PHY link layer training to stabilize
sleep 1

# Step 7: Load NVIDIA Kernel Modules bypass install interceptors
echo "[+] Loading NVIDIA proprietary driver stack..."
modprobe --ignore-install nvidia
modprobe --ignore-install nvidia-modeset
modprobe --ignore-install nvidia-drm
modprobe --ignore-install nvidia-uvm

# Step 8: Apply Power and Performance Locks
if command -v nvidia-smi >/dev/null 2>&1; then
    echo "[+] Applying GPU persistence mode & power limit (${POWER_LIMIT}W)..."
    nvidia-smi -pm 1 2>/dev/null || true
    nvidia-smi -pl "$POWER_LIMIT" 2>/dev/null || true
fi

echo "========================================================"
echo "                  PCIe Link Verification                "
echo "========================================================"
lspci -vv -s "$EGPU_PCI" | grep -iE "LnkSta:|LnkCtl2:" || true
echo "[✓] eGPU successfully initialized and ready for use."

How to Revert All Changes (Uninstallation):

Bash

# Remove configuration files
sudo rm -f /etc/modprobe.d/99-egpu-intel-lock.conf /etc/tmpfiles.d/pcie-aspm-performance.conf

# Rebuild initramfs
sudo mkinitcpio -P         # Arch / CachyOS
sudo dracut -f             # Fedora
sudo update-initramfs -u    # Debian / Ubuntu

sudo reboot