Driver version 516.94 introduces high DPC latency and serious audio problems

Since driver version 516.94, i am experiencing unconditional random audio dropouts, pops and crackles when working with my audio software.
During my research, i have ruled out any of the common reasons for such behaviour.
Finally, i joined a thread at some forum, where a bunch of people already discussed the issue.

From what people are stating, their computers have the following in common:

  • NVIDIA graphics card
  • Intel Chipset (often ASUS branded MB)
  • A current Intel CPU

Here is a screenshot from LatencyMon, which clearly shows the problem.
Driver version 516.94, light load (ASIO)

This is really awful as you can see.
While this may not affect a majority of users (i didn’t notice any problems during gaming or watching videos), it is still a massive problem for creators who work with audio or other realtime-dependent applications.
In my case, i didn’t have any problems like this for years, neither with my audio interface, nor with all the apps i use.

Now i even thought about replacing my audio hardware and reinstalling the OS.
Which is pointless, because a downgrade to driver version 512.95 fixes it.

I am really hoping that NVIDIA can take account of this.
Thanks.

1 Like

I have also been experiencing the similar issues as the OP for the last 1.5 weeks. The main difference for me is that the suggested driver still doesn’t fix the issue for me. Also I’m running the following config:

  • Aorus B550i Pro Ax Wi-Fi
  • AMD Ryzen 7 5800X
  • Nvidia 3060ti Founders Edition
  • 32GB DDR4 3600Mhz RAM
  • 1TB Samsung 980 Pro NVMe M.2 storage.

No amount of reinstallation or Windows 10/11 or driver version remedies the issue unfortunately.

I’m left with awful audio pops, crackles, and latency issues in media, and mouse cursor/keyboard usage. It renders my PC virtually unusable.

I have an identical issue as the user above in terms of audio degradation, which presents itself when the nvidia driver is installed (for my 3060ti card). It seems that the drivers have an nvlddmkm.sys file that exhibits a high DPC which causes these latency/audio degradation issues.

Uninstalling the nvidia driver removes this issue, but then I can’t use the pc with Microsoft Basic Driver Display.

Please can you investigate this issue seriously, as it seems to be quite a prevalent issue affecting users in slightly different ways. Please see the link in OPs post to the nlite website to see how long some users have been pursuing this (over 6+ months). On Reddit and other sites, people have had this issue for over a year, and it remains unresolved!

Update: i have now tested every single driver version that has been released up to now.
Each and every one has the same DPC latency issues.
The only version that is still available and doesn’t suffer from that, is 512.95 (edit: 512.96 also works)

And now the crazy thing: if i start Unreal Engine 5.1 editor on my machine, it tells me this:

GraphicsDriver

That leads me to the thought that Microsoft might have something to do with the problem.

However, it would be amazing to get at least a small statement from some NVIDIA professional on this.

1 Like

I also had this problem. Up to 8mS DPC. Switching to 512.95 seems to have greatly improved things.

First of all hello @mcebis and @jpxdude and welcome to the NVIDIA developer forums!

To start with, I have forwarded this information as an internal ticket to QA/Engineering. But I do need to give you some context to not get your hopes up too soon.

I have been looking at the links you shared and tried to get a bit more information. And I tried the (single) tool that most people are using to measure this. I have used it on one Intel system and did not see any issues. I used it on another and the tool showed no issues in one run and 1-2ms latency in another. I used it on three systems with AMD CPUs and saw no issue at all.

DPC issues can have quite a number of different reasons and of course the NVIDIA kernel driver can be related.
The difficulty in addressing a possible issue with the driver is that it is not 100% guaranteed to cause overly large DPC latency. I am not trying to put anyone off here, the only problem I see is that such an issue is similar to the car you bring to a repair shop “because it makes this weird noise when turning a corner”. It will not make that noise when you are at that shop.

Long story short, it might be a while before you see responses either here or from support.

Thanks

1 Like

Hi Markus,

I appreciate your acknowledgement & attention to the problem. I understand the difficulties of this kind of problem as I used to be a real-time assembly language programmer. It is difficult in a system with many drivers and interactions that can’t be tested or controlled.

If you look at the NTLite thread you can see this is a wide problem for many people. It mostly is noticed by folks doing audio production as the DPC causes audio dropouts in the real time audio drivers. As Nvidia has gaming and studio drivers/ applications it’s going to be an issue for the studio folks and if you’ve bought an NVidia card with studio use in mind it’s likely to be seen an nVIDIA issue as it’s the driver that shows up as the problem even though it’s a systemic problem for a subset of users.

If anyone is investigating from nVIDIA some clues may be in that thread. I don’t think anyone has found a universal solution to the problem. As I see it, unfortunately Windows isn’t designed as a RTOS from the ground up but as a significant number of gamers & studio users are needing it to behave like one, some help or assistance to tweak away the problems is needed. As a RTOS programmer, I know it’s unfortunately not an issue of brute force processing speed, but of timing & tuning. Besides downgrading nVIDIA drivers & disabling hardware that may be causing the interaction problems, LatencyMON and the interrupt tool are all we have to try and tweak.

I hope those comments help.

Regards

1 Like

Hello again,

I appreciate your detailed reply and your understanding!

Engineering have all this now and the links to the NTLite forum as well as some additional things I found out. So for now all we can do is wait.

I regularly check up on bugs like these, so I will give updates as soon as I see them.

Cheers

2 Likes

Here is where I have been able to get to by:

  1. Disabling all unnecessary HW & unplugging unused USB devices
  2. Downgrading to 512.95
  3. Moving the NVIDIA driver interrupt to CPU 2 using the interrupt affinity tool.

I had the 2nd monitor disabled (uses CPU graphics via DVI which is not on my RTX3050) but had re-enabled it for this test.

You can see that the nVIDIA driver is not the biggest culprit any longer. A 5 minute test is the recommended time.

I’ll now start enabling other HW & plugging in USBs.

Probably solved for me other than I am using an old driver.

This may be helpful. I unplugged all unnecessary USB hardware then benchmarked different drivers and interrupt options. I used the studio drivers as presumably that is what most folks noticing this problem may use. All runs were for 5 minutes for comparability and after a reboot.

                                             DPC Count     Highest Exec. (mS)   Total Exec. (mS)

528.24 Interrupt (Core0) 12 487 1.354453 186.96
528.24 Interrupt (Core1) 3 345 0.401488 53.95
512.96 Interrupt (Core0) 11 034 0.567242 160.37
512.96 Interrupt (Core1) 2 960 0.055919 44.55

You can see that on the same core interrupt 512.96 has ~10% fewer DPCs than 528.24 and is ~15% faster in total execution. The 0.0559mS looks impressive but if when I ran it longer I could still get a rare ~0.5mS time.

The big difference is moving the driver to another core than the default core as DPCs and total execution times drop to less than a third for both drivers. It is non-linear in behaviour so the problem is the 1.35mS DPC because with that delay sound is likely to glitch otherwise unlikely to be noticed. The problem is seen to be in the later drivers because the 10% DPC/ 15% execution time penalty with default core interrupts manifests the problem (at least for me).

It would seem wise for me to use the older driver and put the nVIDIA driver interrupt on a non-default core. Every time the driver upgrades, it goes back to the default core.

A good part of my day gone on this so I hope this info helps the nVIDIA team & other users. I have screen shots of the four runs if they may be of any help.

EDIT
As a new user I can’t add a new reply to NetAndifNG’s post so adding here

Good point @NetAndifNG . For the record, I also did the usual ‘real time’ tweaks that you mention like turning off indexing & changing power options. That was prior to discovering the nVIDIA driver DPC issue. The 10% extra DPCs between the two drivers would not seem to be that significant but as I said it’s a non-linear issue so it is and of course, it is possible that the interactions on your system are different to mine. It’s common in software development that the desire to improve drivers causes execution creep issues. Usually there are benchmarking tests before release to check the performance hit of changes. I’m not sure if the DPC performance is something nVIDIA look at between releases.

One thing that surprises me as a former RT programmer is that Windows (& driver writers) put all interrupts on the first core as default. Being able to optimise this on a multi core system would seem to be ideal but I can also see that if there are dependencies between the different interrupt processes, having them on one core allows for more orderly execution which is probably why the DPCs happen. So in that sense DPCs are not bad unless they delay something like ‘real time’ audio from servicing its needs which is what is happening here. I think the problem does not happen on AMD processors but I’m not familiar with the different way they may handle interrupts. In any case, having the nVIDIA driver interrupt on a separate core seems to cause no harm for me.

I have to mention that i did not much “optimization” to my system, other than the usual “tweaks” like setting high-performance power options in Windows and BIOS, as well as resolving IRQ conflicts that had been caused by an onboard audio chipset. I also tried a registry hack, which disabled Spectre & Meltdown investigations, which had been know to drop performance of intel systems.

However, the only thing that had a substantial impact on stability of DPC latency, was the downgrade to driver version 512.95

Meanwhile, i found some evidences that NVIDIA drivers, starting at 516.xx versions caused a remarkable burst of problem reports, not only from audio users, but also gamers.
E.g. google for “nvidia driver 516.40 problems”.

Something seems to have changed in the NVIDIA drivers after 512.95, which does lead to problems for several users with many different systems and apps / games. But the effects, they have in common.

1 Like