Since driver version 516.94, i am experiencing unconditional random audio dropouts, pops and crackles when working with my audio software.
During my research, i have ruled out any of the common reasons for such behaviour.
Finally, i joined a thread at some forum, where a bunch of people already discussed the issue.
From what people are stating, their computers have the following in common:
NVIDIA graphics card
Intel Chipset (often ASUS branded MB)
A current Intel CPU
Here is a screenshot from LatencyMon, which clearly shows the problem.
Driver version 516.94, light load (ASIO)
This is really awful as you can see.
While this may not affect a majority of users (i didnât notice any problems during gaming or watching videos), it is still a massive problem for creators who work with audio or other realtime-dependent applications.
In my case, i didnât have any problems like this for years, neither with my audio interface, nor with all the apps i use.
Now i even thought about replacing my audio hardware and reinstalling the OS.
Which is pointless, because a downgrade to driver version 512.95 fixes it.
I am really hoping that NVIDIA can take account of this.
Thanks.
I have also been experiencing the similar issues as the OP for the last 1.5 weeks. The main difference for me is that the suggested driver still doesnât fix the issue for me. Also Iâm running the following config:
Aorus B550i Pro Ax Wi-Fi
AMD Ryzen 7 5800X
Nvidia 3060ti Founders Edition
32GB DDR4 3600Mhz RAM
1TB Samsung 980 Pro NVMe M.2 storage.
No amount of reinstallation or Windows 10/11 or driver version remedies the issue unfortunately.
Iâm left with awful audio pops, crackles, and latency issues in media, and mouse cursor/keyboard usage. It renders my PC virtually unusable.
I have an identical issue as the user above in terms of audio degradation, which presents itself when the nvidia driver is installed (for my 3060ti card). It seems that the drivers have an nvlddmkm.sys file that exhibits a high DPC which causes these latency/audio degradation issues.
Uninstalling the nvidia driver removes this issue, but then I canât use the pc with Microsoft Basic Driver Display.
Please can you investigate this issue seriously, as it seems to be quite a prevalent issue affecting users in slightly different ways. Please see the link in OPs post to the nlite website to see how long some users have been pursuing this (over 6+ months). On Reddit and other sites, people have had this issue for over a year, and it remains unresolved!
Update: i have now tested every single driver version that has been released up to now.
Each and every one has the same DPC latency issues.
The only version that is still available and doesnât suffer from that, is 512.95 (edit: 512.96 also works)
And now the crazy thing: if i start Unreal Engine 5.1 editor on my machine, it tells me this:
That leads me to the thought that Microsoft might have something to do with the problem.
However, it would be amazing to get at least a small statement from some NVIDIA professional on this.
First of all hello @mcebis and @jpxdude and welcome to the NVIDIA developer forums!
To start with, I have forwarded this information as an internal ticket to QA/Engineering. But I do need to give you some context to not get your hopes up too soon.
I have been looking at the links you shared and tried to get a bit more information. And I tried the (single) tool that most people are using to measure this. I have used it on one Intel system and did not see any issues. I used it on another and the tool showed no issues in one run and 1-2ms latency in another. I used it on three systems with AMD CPUs and saw no issue at all.
DPC issues can have quite a number of different reasons and of course the NVIDIA kernel driver can be related.
The difficulty in addressing a possible issue with the driver is that it is not 100% guaranteed to cause overly large DPC latency. I am not trying to put anyone off here, the only problem I see is that such an issue is similar to the car you bring to a repair shop âbecause it makes this weird noise when turning a cornerâ. It will not make that noise when you are at that shop.
Long story short, it might be a while before you see responses either here or from support.
I appreciate your acknowledgement & attention to the problem. I understand the difficulties of this kind of problem as I used to be a real-time assembly language programmer. It is difficult in a system with many drivers and interactions that canât be tested or controlled.
If you look at the NTLite thread you can see this is a wide problem for many people. It mostly is noticed by folks doing audio production as the DPC causes audio dropouts in the real time audio drivers. As Nvidia has gaming and studio drivers/ applications itâs going to be an issue for the studio folks and if youâve bought an NVidia card with studio use in mind itâs likely to be seen an nVIDIA issue as itâs the driver that shows up as the problem even though itâs a systemic problem for a subset of users.
If anyone is investigating from nVIDIA some clues may be in that thread. I donât think anyone has found a universal solution to the problem. As I see it, unfortunately Windows isnât designed as a RTOS from the ground up but as a significant number of gamers & studio users are needing it to behave like one, some help or assistance to tweak away the problems is needed. As a RTOS programmer, I know itâs unfortunately not an issue of brute force processing speed, but of timing & tuning. Besides downgrading nVIDIA drivers & disabling hardware that may be causing the interaction problems, LatencyMON and the interrupt tool are all we have to try and tweak.
This may be helpful. I unplugged all unnecessary USB hardware then benchmarked different drivers and interrupt options. I used the studio drivers as presumably that is what most folks noticing this problem may use. All runs were for 5 minutes for comparability and after a reboot.
You can see that on the same core interrupt 512.96 has ~10% fewer DPCs than 528.24 and is ~15% faster in total execution. The 0.0559mS looks impressive but if when I ran it longer I could still get a rare ~0.5mS time.
The big difference is moving the driver to another core than the default core as DPCs and total execution times drop to less than a third for both drivers. It is non-linear in behaviour so the problem is the 1.35mS DPC because with that delay sound is likely to glitch otherwise unlikely to be noticed. The problem is seen to be in the later drivers because the 10% DPC/ 15% execution time penalty with default core interrupts manifests the problem (at least for me).
It would seem wise for me to use the older driver and put the nVIDIA driver interrupt on a non-default core. Every time the driver upgrades, it goes back to the default core.
A good part of my day gone on this so I hope this info helps the nVIDIA team & other users. I have screen shots of the four runs if they may be of any help.
EDIT
As a new user I canât add a new reply to NetAndifNGâs post so adding here
Good point @NetAndifNG . For the record, I also did the usual âreal timeâ tweaks that you mention like turning off indexing & changing power options. That was prior to discovering the nVIDIA driver DPC issue. The 10% extra DPCs between the two drivers would not seem to be that significant but as I said itâs a non-linear issue so it is and of course, it is possible that the interactions on your system are different to mine. Itâs common in software development that the desire to improve drivers causes execution creep issues. Usually there are benchmarking tests before release to check the performance hit of changes. Iâm not sure if the DPC performance is something nVIDIA look at between releases.
One thing that surprises me as a former RT programmer is that Windows (& driver writers) put all interrupts on the first core as default. Being able to optimise this on a multi core system would seem to be ideal but I can also see that if there are dependencies between the different interrupt processes, having them on one core allows for more orderly execution which is probably why the DPCs happen. So in that sense DPCs are not bad unless they delay something like âreal timeâ audio from servicing its needs which is what is happening here. I think the problem does not happen on AMD processors but Iâm not familiar with the different way they may handle interrupts. In any case, having the nVIDIA driver interrupt on a separate core seems to cause no harm for me.
I have to mention that i did not much âoptimizationâ to my system, other than the usual âtweaksâ like setting high-performance power options in Windows and BIOS, as well as resolving IRQ conflicts that had been caused by an onboard audio chipset. I also tried a registry hack, which disabled Spectre & Meltdown investigations, which had been know to drop performance of intel systems.
However, the only thing that had a substantial impact on stability of DPC latency, was the downgrade to driver version 512.95
Meanwhile, i found some evidences that NVIDIA drivers, starting at 516.xx versions caused a remarkable burst of problem reports, not only from audio users, but also gamers.
E.g. google for ânvidia driver 516.40 problemsâ.
Something seems to have changed in the NVIDIA drivers after 512.95, which does lead to problems for several users with many different systems and apps / games. But the effects, they have in common.
Same problem for me, I was using the AMD 5700G with integrated video had no problem, I bought a RTX 3050 and after that the sound from the monitor seems to be bursting with echo, using my soundbar on the Optica output I have no problems. I reversed the monitors and put other cables with the same problem, returning to using my integrated graphics the problem disappears, in addition to noticing that when this happens the DCP has high latency!
The crackling and pops were so constant that I thought I had some sort of grounding issue on my room, I went as far as replacing my entire audio system and isolating the machine from any other electric device.
I donât know if this is useful but Iâm able to recreate and âfixâ the crackling issue. @MarkusHoHo
System specs:
I9-12900ks
32GB DDR5
3090 FE.
3 x samsung pro pcie 4.0 nvmes
If I run the ram in stock mode (4800MHz) the audio is bad, I get frequent cracks and pops. The audio gets significantly worse the more you reduce the ram frequency.
In the other hand, if I overclock the RAM modules to 6800MHz, the audio issues are reduced greatly.
From my testing the biggest factors to recreate the audio issues are:
â NVMe drives are under load.
â RAM not keeping up.
I tried this with different RAM sticks and NVMe models from different vendors and the results were consistent across all of them. I was able to get less audio cracks with newer DDR5 sticks that have lower latency.
Another way to replicate the issue is trying to use the system with only 1 stick of ram, the audio will be unbearable despite the overall system being responsive (running IDEs, debuggers and other demanding software).
The audio format does not seem to be that important, at least in my case, changing the sample rate and bit depth didnât help in reducing the latency.
Finally, it could be that my speakers arent the best but I wasnât able to hear cracks when using them ( I didnât have any DPC latency spike either ).
I only get the audio issues consistently while using any sort of headset (with or without DAC).
Hardware acceleration does not seem to affect the issue, it persists whether its on or off.
I posted a way to replicate the issue but akismet removed it.
I was just checking if you tested that, because I have a bug reported (and confirmed) where having hardware GPU scheduler enabled makes AI inference (and possibly training) run slower (i.e. Stable Diffusion image generation is slower, it results in fewer it/s). I wouldnât be surprised if it affected other stuff as well.
From what I saw, the problems are likely to be linked with task scheduling. Limiting the RAM frequency and capacity seems to be the best way to recreate the problem. Slowing down the drives seems to have a significant impact as well.
For example, you can have a nearly idle system with only few tools open, it will appear responsive but the audio will be unbearable and filled with cracks and pops. Setting the priority of the audio services & processes seems to make the experience better but still noticeably bad.
Generally speaking Iâm almost able to get âgoodâ audio when tweaking the entire system and overclocking things here and there, however, it seems ridiculous that in order to reduce audio cracks you need to do all this work and optimizations.
Another important factor is whether the audio is being executed in the background or not, it seems like audios running in the background are more likely to crack.
While playing around with some AI stuff, i tried a variety of CUDA toolkits.
They also include drivers, which are not available from the regular download page.
So i was able to narrow down the versions thing a bit.
Driver version 511.65, which is included in CUDA Toolkit v11.6, seems to be the last version before 512.95, and it also doesnât have the latency problems. DPC counts are comparable or even slightly better.
Driver version 516.01, which is included in CUDA Toolkit v11.7, seems to be the first version that introduces the DPC latency issues. I have verified this using LatencyMon.
And here is a suggestion / idea in order to potentially speed-up the resolution of this.
If NVIDIA wants to solve this problem, they need a test-setup that includes some aspects that seem to trigger the problems.
E.g. they could use an extra audio interface in their lab. I know, this sounds a but naive. Not that i suggest that the problem only arised in these cases. But at least that would be a way to âtriggerâ the problem.
I for myself have a MOTU Traveler MK3 firewire interface, connected using a PCIe 1394-Card, with a Texas Instruments Chip (but that is not related to the DPC issue, since i had a VIA-based card before).
First of all welcome @moslde to the NVIDIA developer forums and this discussion.
Thanks everyone for the valuable input and suggestions.
As some suspected this is not as simple as just the driver doing something it should not. OS, system drivers, audio drivers all play into this as well. Meaning root causing is extremely hard. I am on CC of the internal bug that is addressing this and lately I receive between 10 and 20 messages on the topic every single day.
I just wanted to share this to show we are not ignoring or even taking this lightly.
I agree that itâs a complex scenario, especially since some amd users (without nvidia gpus) have reported similar issues. There are also cases of intel with amd gpu users reporting similar issues.
reddit:
/r/Amd/comments/11no3nd/a_detailed_summary_of_the_crackling_problem/?sort=new
It seems to be an issue with many modern high end systems and the most common ââsolutionââ is limiting the bandwidth of the gpu(s).
With that being said, I get much better audio quality when downgrading to 512.95, I still have some cracks here and there but they are much more uncommon.
I believe the reason this isnât reported as often is because users need to have a âprofessionalâ audio system to notice those cracks, if I change the audio format to a lower quality one, the cracks are non existent.
