To get MIG functionality in an RTX 6000 pro ws card, the “displaymodeselector” utility requires bios >= 98.02.55.00.00. The BIOS on my NVIDIA manufactured RTX 6000 pro ws card is 98.02.52.00.02. NVIDIA tech support said that NVIDIA does not have BIOS upgrades for this card, and that I should put up a post here for help. Might anyone be able to help ?
Hi @alain10, welcome to the NVIDIA developer forums!
I can’t answer this directly, but I tagged someone who might be able to help.
Which tech support did you contact? Consumer support or Enterprise?
Thanks!
depending on where you bought your RTX PRO, mainly whether through an OEM like HP, Dell or Lenovo, or through channel and in which region, so ultimately from like PNY, Leadtek or ELSA JP, the VBIOS matching your product needs to come from the respective vendor. Pls contact their support, they should be able to provide the matching VBIOS and update tool…. greeting -Frank
Thanks Markus, Hi Frank. Can you tell me how do I find the manufacturer? I am using linux - this is the most I have found out so far: PCI Vendor ID: 0x10de , PCI Device ID: 0x2bb1
hmm, so you don’t know what exactly you bought? this is NOT an OEM system from like Dell, HP or Lenovo, which come with the GPU included? If you bought just the GPU as aftermarket option, or assembled into a whitebox from an integrator, the invoice does not have any more details on what exact GPU this is?
The Device IDs are the same for all RTX PRO 6000s… some differentiation may be in the SubSystemVendor IDs, so can you report the other 2x PCI IDs that are identifying your card….
Maybe the command ‘nvidia-smi -q’ could also help identifying the board less ambiguous… Pls send the output over…
Ultimately, there is a small sticker on each board, with a number in the format 900-xxxxx-yyyy-zzz, that would help…
Apologies for the hassle, we should have caught the problem with MIG before we started shipping these boards, but MIG is new for this class of products … :-(.
regards
-Frank
Hi Frank, the seller was Newegg. I received it four days ago. They said their policy was for the purchaser to return the product to the manufacturer for a refund. It appeared to be an OEM box - with a number on it similar to your description: NVD-900-5G144-2200-000.
I will thank you for the instructions here, at the beginning, because the output of nvidia-smi -q is lengthy:
==============NVSMI LOG==============
Timestamp : Tue Aug 26 07:17:56 2025
Driver Version : 580.65.06
CUDA Version : 13.0
Attached GPUs : 1
GPU 00000000:8E:00.0
Product Name : NVIDIA RTX PRO 6000 Blackwell Workstation Edition
Product Brand : NVIDIA RTX
Product Architecture : Blackwell
Display Mode : Requested functionality has been deprecated
Display Attached : No
Display Active : Disabled
Persistence Mode : Enabled
Addressing Mode : HMM
MIG Mode
Current : N/A
Pending : N/A
Accounting Mode : Disabled
Accounting Mode Buffer Size : 4000
Driver Model
Current : N/A
Pending : N/A
Serial Number : redacted
GPU UUID : redacted
GPU PDI : redacted
Minor Number : 0
VBIOS Version : 98.02.52.00.02
MultiGPU Board : No
Board ID : 0x8e00
Board Part Number : 900-5G144-2200-000
GPU Part Number : 2BB1-870-A1
FRU Part Number : N/A
Platform Info
Chassis Serial Number :
Slot Number : 0
Tray Index : 0
Host ID : 1
Peer Type : Direct Connected
Module Id : 1
GPU Fabric GUID : 0x0000000000000000
Inforom Version
Image Version : G144.0520.00.02
OEM Object : 2.1
ECC Object : 7.16
Power Management Object : N/A
Inforom BBX Object Flush
Latest Timestamp : N/A
Latest Duration : N/A
GPU Operation Mode
Current : N/A
Pending : N/A
GPU C2C Mode : Disabled
GPU Virtualization Mode
Virtualization Mode : None
Host VGPU Mode : N/A
vGPU Heterogeneous Mode : N/A
GPU Recovery Action : None
GSP Firmware Version : 580.65.06
IBMNPU
Relaxed Ordering Mode : N/A
PCI
Bus : 0x8E
Device : 0x00
Domain : 0x0000
Base Classcode : 0x3
Sub Classcode : 0x0
Device Id : 0x2BB110DE
Bus Id : 00000000:8E:00.0
Sub System Id : 0x204B10DE
GPU Link Info
PCIe Generation
Max : 5
Current : 1
Device Current : 1
Device Max : 5
Host Max : 5
Link Width
Max : 16x
Current : 16x
Bridge Chip
Type : N/A
Firmware : N/A
Replays Since Reset : 0
Replay Number Rollovers : 0
Tx Throughput : 1270 KB/s
Rx Throughput : 1573 KB/s
Atomic Caps Outbound : N/A
Atomic Caps Inbound : FETCHADD_32 FETCHADD_64 SWAP_32 SWAP_64 CAS_32 CAS_64
Fan Speed : 30 %
Performance State : P8
Clocks Event Reasons
Idle : Not Active
Applications Clocks Setting : Not Active
SW Power Cap : Active
HW Slowdown : Not Active
HW Thermal Slowdown : Not Active
HW Power Brake Slowdown : Not Active
Sync Boost : Not Active
SW Thermal Slowdown : Not Active
Display Clock Setting : Not Active
Clocks Event Reasons Counters
SW Power Capping : 7774172002 us
Sync Boost : 0 us
SW Thermal Slowdown : 0 us
HW Thermal Slowdown : 0 us
HW Power Braking : 0 us
Sparse Operation Mode : N/A
FB Memory Usage
Total : 97887 MiB
Reserved : 640 MiB
Used : 19 MiB
Free : 97230 MiB
BAR1 Memory Usage
Total : 131072 MiB
Used : 19 MiB
Free : 131053 MiB
Conf Compute Protected Memory Usage
Total : 0 MiB
Used : 0 MiB
Free : 0 MiB
Compute Mode : Default
Utilization
GPU : 0 %
Memory : 0 %
Encoder : 0 %
Decoder : 0 %
JPEG : 0 %
OFA : 0 %
Encoder Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
FBC Stats
Active Sessions : 0
Average FPS : 0
Average Latency : 0
DRAM Encryption Mode
Current : Disabled
Pending : Disabled
ECC Mode
Current : Disabled
Pending : Disabled
ECC Errors
Volatile
SRAM Correctable : N/A
SRAM Uncorrectable Parity : N/A
SRAM Uncorrectable SEC-DED : N/A
DRAM Correctable : N/A
DRAM Uncorrectable : N/A
Aggregate
SRAM Correctable : N/A
SRAM Uncorrectable Parity : N/A
SRAM Uncorrectable SEC-DED : N/A
DRAM Correctable : N/A
DRAM Uncorrectable : N/A
SRAM Threshold Exceeded : N/A
Aggregate Uncorrectable SRAM Sources
SRAM L2 : N/A
SRAM SM : N/A
SRAM Microcontroller : N/A
SRAM PCIE : N/A
SRAM Other : N/A
Channel Repair Pending : No
TPC Repair Pending : No
Retired Pages
Single Bit ECC : N/A
Double Bit ECC : N/A
Pending Page Blacklist : N/A
Remapped Rows
Correctable Error : 0
Uncorrectable Error : 0
Pending : No
Remapping Failure Occurred : No
Bank Remap Availability Histogram
Max : 512 bank(s)
High : 0 bank(s)
Partial : 0 bank(s)
Low : 0 bank(s)
None : 0 bank(s)
Temperature
GPU Current Temp : 34 C
GPU T.Limit Temp : 59 C
GPU Shutdown T.Limit Temp : -5 C
GPU Slowdown T.Limit Temp : -2 C
GPU Max Operating T.Limit Temp : 0 C
GPU Target Temperature : N/A
Memory Current Temp : N/A
Memory Max Operating T.Limit Temp : N/A
GPU Power Readings
Average Power Draw : 16.00 W
Instantaneous Power Draw : 16.41 W
Current Power Limit : 600.00 W
Requested Power Limit : 600.00 W
Default Power Limit : 600.00 W
Min Power Limit : 150.00 W
Max Power Limit : 600.00 W
GPU Memory Power Readings
Average Power Draw : N/A
Instantaneous Power Draw : N/A
Module Power Readings
Average Power Draw : N/A
Instantaneous Power Draw : N/A
Current Power Limit : N/A
Requested Power Limit : N/A
Default Power Limit : N/A
Min Power Limit : N/A
Max Power Limit : N/A
Power Smoothing : N/A
Workload Power Profiles
Requested Profiles : N/A
Enforced Profiles : N/A
Clocks
Graphics : 180 MHz
SM : 180 MHz
Memory : 405 MHz
Video : 600 MHz
Applications Clocks
Graphics : 2617 MHz
Memory : 14001 MHz
Default Applications Clocks
Graphics : 2617 MHz
Memory : 14001 MHz
Deferred Clocks
Memory : N/A
Max Clocks
Graphics : 3090 MHz
SM : 3090 MHz
Memory : 14001 MHz
Video : 3090 MHz
Max Customer Boost Clocks
Graphics : N/A
Clock Policy
Auto Boost : N/A
Auto Boost Default : N/A
Fabric
State : N/A
Status : N/A
CliqueId : N/A
ClusterUUID : N/A
Health
Summary : N/A
Bandwidth : N/A
Route Recovery in progress : N/A
Route Unhealthy : N/A
Access Timeout Recovery : N/A
Incorrect Configuration : N/A
Processes
GPU instance ID : N/A
Compute instance ID : N/A
Process ID : 3063
Type : G
Name : /usr/lib/xorg/Xorg
Used GPU Memory : 4 MiB
Capabilities
EGM : disabled
thanks for the details, that number should be ok for me to confirm, what card this is, and where for you to get the necessary VBIOS update…
if its urgent, you may try to contact PNY, one of our channel partners for the enterprise products, They should have an update routine for endusers available… the number confirms you have a channel SKU, not one of the OEM versions of the board…
If not urgent, maybe wait one more day, until I have internal confirmation….
thanks for your patience
-Frank
Hi Frank. Thank you for the sensitivity. Time is a factor for me.
I contacted PNY who had me remove the card and take pictures of all sides and send them to PNY. The card had no PNY markings. The bottom of the card had a label that said NVIDIA. PNY said they have no record of the card’s serial number.
PNY’s best guess was that the card was was NVIDIA manufactured. I am posting the bottom of the card label with the s/n redacted.
Assuming the card was manufactured by NVIDIA, and Newegg based its policy on an arrangement with NVIDIA, could you please tell me the NVIDA address to send the card for a refund?
I’m sorry for your trouble Alain, but if you really think you will want to return the board, that only between you and newegg, you cant return a board directly to Nvidia, nor would we be able to refund you any money …
Newegg seem to have another (than PNY) source in the channel, for the upper of the offers in this screenshot. Newegg need to be able to tell you, where they source that one from, what distributor, and that distributor then will need to be able to provide you with the VBIOS update tool (which indeed comes from Nvidia, distributed to all our direct channel partners, but potentially different versions, depending on when they start certification and qualification of each product…
I’m out of Germany, so not familiar with the US channel partner landscape, so I depend on US colleagues to help me out with information here, which adds latency (on top of me being on vaca this week…)
Pls stay tuned…
thanks
-Frank…
btw: why do you NEED MIG?
Hi Frank,
I went to PNY as you suggested (thank you). The PNY folks were great, and really interested in helping because they could not find the card’s serial number in their systems. They concluded it was likely an NVIDIA brand card. I went back to NVIDIA customer support who finally stated that NVIDIA does NOT take Newegg returns - contrary to Newegg’s published polciy.
I went back to Newegg with NVIDIA’s statement, a picture of the graphic card label showing the S/N and the VBIOS of the card as shipped, and the screen capture of the card’s MIG firmware requirement (shown below). Newegg relented, and sent me a return label.
Since you ask, my current research requires GPU processing speed as opposed to large amounts of card memory. My needs are measured in GPU processing years (not minutes or hours). The MIG capability permits four independent processes to run in parallel. I could use two more such cards which would give me twelve concurrent processes, but can only afford the one. If you are interested in the research direction see: CSDL | IEEE Computer Society
1 Like
I’ll give you 2 replies - first to MIG and the computing resources…:
I don’t want to talk you out of MIG at all, pls don’t get me wrong… with MIG, you have basically 4 GPUs, quarter size each, and an easy way of distribution/scheduling, since you give a job to each instance. will each job max. out each instance? might there be unused resources left over, unusable for the ‘first=only’ job? GPUs certainly are capable of doing multiple jobs ‘at the same time’, have their own scheduling, or you can actually influence the scheduling, so that you can max. out the whole GPU as best as possible, maybe be able to even run a ‘5th’ job on that same, full GPU…? MIG might give you more reliable low latency, if that is important.. MIG with virtualization will give you tenant separation, if that is important, but for running multiple jobs on a single GPU, you don’t NEED MIG….
all the MIG ‘needs’ aside: the product supports MIG, and its JUST a VBIOS update from working…
WE omitted to release the board with MIG properly tested and supported to all our partners (20ish) initially, so all partners received the VBIOS update, and now need to create their own enduser flash routine, an provide that to their websites, or via support to those users who ask…
The issue in your case is that Newegg is not helping you to know, who they bought from, so you an contact the nvidia partner, that sold to Newegg, maybe even with another distributor in between…
That’s where I need input from my US-based colleagues, to get to the right company, who needs to provide the enduserflash routine… (it would have been ‘easy’ in this case, if you received the PNY ‘version’ of the product :-(. ).
again, apologies for all your hassle just because of Nvidia’s omission to have MIG properly enabled and supported in the initial version… (to our excuse, first time we have MIG support in a product that is NOT for datacenter, so coming from different teams and people… )
-Frank
1 Like
Hi Frank,
Thank you. I appreciate the thought you put into it. Those were the first things I tried, much to the frustration of the IT folks.
Data center GPU’s are almost always setup in virtual environments for multiple users, schedulers like SLURM, and the like.
The reason this is not something you could have forseen is because the nature of this current research line is definitely NOT production work. It pushes software to its limits. As it turns out, the inter-operation of virtual environments and their underlying O/S are most often the major contributor to a crash, which then causes the entire system to be rebooted.
This bare-metal O/S + MIG setup causes the least amount of problems and yields the most experiment run time per hour for this particular work.
Please note my prior post - Newegg sent me a return label and I have already sent the card back to them.
Thanks again.
1 Like
Has there been any update on this process? My card is on vbios 98.02.52.00.02 and would not be able to use MIG if I wanted to give it a whirl.
Hi, the ‘issue’ has a solution, in that the partner WE sold the GPUs to is the one that needs to provide you with the VBIOS update matching your GPU… (different partners and OEMs may have slightly different VBIOSes, and all have their own release cycle and procedure)…
Now the issue preventing to get that easily is the problem of finding the partner WHO is the first one, we sold the GPUs to and have the direct contact for like VBIOS update etc…
Depending on region you are in, this could be one of many, some larger of these are like PNY and TDsynnex. But without the reseller you bought from helping you to find out the route the board came from, there is a gap in how to support this issue :-(.
I apologize for this, and am pushing internally for the BU and partner org, to work on solving this process issue…
Best I can recommend for the moment is to ask your reseller for help where he bought the GPU from, and then contact that one, providing the SN of your GPU he can confirm this very GPU went through his warehouse, and he then needs to provide you with the VBIOS update routine…
Again, I feel sorry for the lack of a good customer experience for this problem that is NV induced, no partner to blame really!
thanks for your understanding - Frank
1 Like
Thanks Frank. I merely went through Newegg Business as my reseller so there’s not really a singular point of contact and the box is simply an OEM basic white box so identifying a particular manufacturer is a bit tricky.
I followed the process and received update instructions. Confusingly, the instructions require me to be already at or above the minimum vBios version 98.02.55.00.00. The card came with 98.02.52.00.02 and the update instructions include a link to download version 98.02.81.00.07. Of course I asked for clarification and further instructions, but the board partner is according to my reseller unresponsive. It’s only been about a week, and will certainly get a response eventually - still, maybe someone from Nvidia (Frank?) could confirm that it is safe to upgrade from any vBios version to 98.02.81.00.07 and that the board partner just made a mistake in their instructions.
Also, is there a changelog that is already or could be published somewhere - maybe alongside advisories regarding errata, issues, workarounds, etc.?
Finally, is it really possible to flash an incompatible vBios, or will the flash process abort and/or allow to rollback to my previous version?
Apologies if this info is already somewhere - it’s my first professional GPU purchase and I am a bit surprised that it’s just the plastic-wrapped hardware in very bland cardboxes, but seemingly nothing else.
Many thanks!!
Hi there,
which board partner is this? can you pls share the instructions with me, privately…?
Above in this thread, others also had 98.02.52.00.02 . the xx.xx.55.xx.xx is the minimum version to be able to use MIG !.
The VBIOS numbers you mention match PNY’s and you should be OK to follow their documentation!
Nvidia does NOT manage all the VBIOS versions centrally, but only PER board partner, hence NO: I CAN’T state it OK to flash any version over any other version. Specially downgrades are DANGEROUS!
The enduser routine for our VBIOS flash process (we provide to our board partners for them to build enduser executables) should be configured in a way that it works SOLELY for a specific VBIOS version upwards to another specific VBIOS version, embedded in the executable… ANY other path the executable should error out…
But REALLY for the board partner to support you here with THEIR details….
re the box: you will have purchased the bulk version, not the retail version… bulk being meant for integrators, building many (identical) systems, and not wanting to go through extensive unboxing and massive disposal of packaging material….
I do hope this helps calm your worries?
-Frank
Hi Frank,
thanks - not worried, just inexperienced. For me it’s expensive hardware and I planned to follow all authoritative instructions to avoid RMA/more costs.
Thanks for the warning - I just wasn’t sure whether all those warnings are just out of an abundance of caution and it’s already impossible to brick modern GPUs with an improper VBIOS update. (For example, the card itself could check whether the update it receives is safe, e.g., not a (unsafe) downgrade, compatible with the specific card hardware configuration/certified operation conditions, etc.). → Hence I better wait for that final clarification from the board partner/reseller, even though in this case it is most likely safe to just upgrade.
ad box) makes sense; yeah this was the bulk version.
working with TDsynnex to basically fix the process, and have them provide and EndUser routine to securely flash the VBIOS to MIG -enabled version for these GPUs….
sorry for the hassle and delay..
-Frank
3 Likes