What is Resizable BAR and why should I be interested?
To learn more about this feature and its general system requirements and compatibilities, including which games and apps will benefit most from it, you should check out our comprehensive Blog post on the topic.
There you will also learn how to use a Windows tool to flash your GPU with an updated VBIOS in case your GPU does not support Resizable BAR yet.
But what do you do if you are running Linux and don’t want to boot up Windows on your machine?
I will go through the steps necessary based on the example of Ubuntu 20.04 and an RTX 3060TI FE.
IMPORTANT!
The following instructions are meant for original Founder’s Edition GPUs and the usual disclaimer holds that the provided tool and these instructions are provided as is and without warranty. You perform any Firmware flashing in this context at your own risk.
For OEM GPUs there exist other tools to flash adjusted VBIOS versions as well as original manufactuerer VBIOS files provided by each OEM seperately. I will add section at the end where I give a few pointers.
Let’s get going!
I assume that you have checked the system requirements like BIOS settings and Resizable BAR support of your CPU to make sure the GPU itself is the last missing piece. The NVIDIA driver version does not matter for this procedure.
First of all you should check if your GPU might not already support this. Simply use nvidia-smi to check the GPU features and grep for the BAR section
> nvidia-smi -q | grep -i bar -A 3
which showed this output for my RTX 3060TI:
BAR1 Memory Usage
Total : 256 MiB
Used : 13 MiB
Free : 243 MiB
If this GPU would support Resizable BAR the Total memory would be the whole VRAM size of the card, in this case 8192 MiB. That means I need to update the GPU VBIOS.
Download the Flash tool
You will find the tool here. Put that file into an easily accessible folder. Also make sure that it has the executable flag set.
-rwxr-xr-x 1 user user 101763760 Jan 16 2023 newlinuxupdater
Suspend any GPU usage
To be able to flash the GPU all NVIDIA drivers and kernel modules need to be unloaded. If you are familiar with the procedure to reboot into text console terminal and unloading all NVIDIA kernel modules, feel free to jump to the next section.
Console mode
The way I go about this is usually always the same. Since I am using GRUB I change/add the following lines to /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="text"
GRUB_TERMINAL=console
and run sudo update-grub. After that make sure the terminal target is set to default, which is done by these commands:
sudo systemctl enable multi-user.target --force
sudo systemctl set-default multi-user.target
Your mileage may vary and some of the multi-user.target dependencies might cause a warning message to appear. But it should not influence the outcome.
If you reboot after this your system will start into a terminal session.
To go back to the GUI Desktop after all is said and done, simply revert your changes to /etc/default/grub and go back to the graphical.target
sudo systemctl enable graphical.target --force
sudo systemctl set-default graphical.target
Unloading NVIDIA kernel modules
You can check which modules are loaded by running lsmod | grep -i nvidia. The normal list of modules and how to unload them is below:
sudo rmmod nvidia_drm nvidia_modeset nvidia_uvm nvidia
Now we are ready to go.
Flashing the VBIOS
The flash utility is based on the same tools we use for Windows systems. That means you will be guided through the process and the tool will have safety checks in place to avoid accidental flashing of wrong data or wrong GPUs. Simply execute the tool as super user and follow the steps.
sudo ./newlinuxupdater
After this it will check the new VBIOS against the detected GPU for compatibility and ask for a final confirmation
Then it is just a question of waiting a short time until you are done
During the first steps you might encounter an error like the following:
This is either due to the fact that not all NVIDIA kernel modules were unloaded, or that you forgot to run the flash tool with super user rights.
Final steps
Reboot the system as instructed before reverting back to the GUI. Make a final check that the process was successful:
> nvidia-smi -q | grep -i bar -A 3
which now shows this output for my RTX 3060TI:
BAR1 Memory Usage
Total : 8192 MiB
Used : 13 MiB
Free : 8179 MiB
Congratulations! You can now reap the benefits of Resizable BAR in Linux!
How to handle OEM GPUS from other vendors?
If you look at the link in the beginning pointing to the comprehensive Blog post on the topic you will find links to the most common OEM partners and their respective flash utilities.
It is highly recommended to use these tools on Windows to enable Resizable BAR on your GPU if it is from one of those vendors. Creating a light-weight Windows PE bootable media is all you need and easily created.
In case you still want to risk doing this on Linux…
The vendor utilities usually come as zip files which include the Firmware ROM binary and a flash utility. What you need is the Firmware ROM. Then you need to find a Linux version of the nvflash utility that supports RTX 30 series cards. There might be reverse engineered versions of it in the depth of the Internet.
At this point I must leave the reader to their own devices since this process is not supported by NVIDIA, it might void your warranty and NVIDIA cannot endorse reverse engineering proprietary software from NVIDIA.