OpenGL Shader Disk Cache: Max Size? Garbage Collection?

I have some questions regarding the OpenGL Shader Disk Cache, but first some background:

I have been using it for various games (mostly via Wine). At first I let games store everything in default location ($XDG_CACHE_HOME/.nv/GLCache). However I quickly found out that having multiple games use the same location means that the caches for my games disappeared. Reading up about it, I read that the maximum per cache size is 256mb, and that it gets garbage collected when it gets full. (but right not I can’t find where I found that information, so I’m hoping it is still correct). I also know you have to rebuild the cache when driver changes and/or I think when the game has been updated.

So, I switched to using separate paths per game by setting __GL_SHADER_DISK_CACHE_PATH different for each game. This allowed me to have separate caches per game. This meant my caches survived longer, and I didn’t have to rebuild them so often. However, lately I’m noticing that for a specific game: Overwatch on Wine, I’m losing my shadercache without the driver changing or game changing. This happens when it gets bigger then ~150mb (Seen it hit 175mb), the next time I log in it is back to 0mb and you have play a couple of hours to fill it again, this means stuttering, lots of stuttering, especially the first game.

So now my questions:

  1. Am I correct that the maximum size is still 256mb? (Is it smaller / bigger?)
  2. At what point does it get garbage collected?
  3. Are there other reasons why the cache might be getting deleted?
  4. If the max is 256mb, why is it so small?
  5. Could we get an increase? Or a way to set a bigger maximum with an ENV variable?

Now, I’m just citing one game, but I’m sure it won’t the only game that has that many shaders. And the purpose of the shader cache gets defeated if you constantly have to rebuild it.

Dox

I think the problem here is wine, with native games I get one directory per game under GlCache.

With wine you get 1 directory / game as well (Prolly / wine version, I have multiple). However if you use the default location for all games, it seems to garbage collect shader caches really fast. So I’m assuming the maximum of the entire directory GLCache can be 256mb.

Separating the games to their own shader cache directories already makes the caches last longer (most games only have 10 to 50 mb). But Overwatch alone is reaching 170+ mb now, and with every hero, every new map they introduce, the bigger the cache will be.

I really wish I could increase the cache size somehow.

yeah i vote for a env variable to change the cache size.

The shader cache size is a (soft) maximum of 128MB. It is a good idea to use a dedicated location for each application so that heavy applications can use the entire shader cache, in fact various games on Steam are doing exactly this.
There is no setting available to increase the size of the cache currently.

Like said, I’m already using separate locations for each application. But that simply isn’t enough it seems. I’m already going past that 128mb for Overwatch. I’m sure others have have issues found similar problems with other games

So questions:

  1. Whats a (soft) maximum? Does it mean that it can go over it, but it gets garbage collected next time I start an application?
  2. Can you please increase that (soft)maximum?
  3. And could we please get that ENV var so we can increase the setting ourselves, so that if we run into the (hopefully new, and much higher) maximum again, we can actually work around it?

I have shader cache of GTA 5 from using DXVK/wine 552MB (that was when DXVK had its own implementation of shader cache) how on earth should we fit that into the cache?

please add env var to increase it

Was wondering why Overwatch with wine and dxvk went from super smooth after cacheing a lot of shaders to awful without any major changes. Makes sense if there is a max cache size that gets garbage collected. I setup a dedicated shader cache folder for Overwatch I’ll start monitoring size on each session. The 128 limit seems very low for what is needed in some games.

After playing for only an hour and not seeing all heroes/abilities it is sitting at over 70MB.
In my second session after playing another hour and a half (or less than 3 hours of play time total) the cache is over 91 MB.
In my third session (hitting about 5 or 6 hours total of play time) the shader cache topped 128MB.
By 7 or 8 hours of playing was at 138MB.

I’m going to make a copy of the Overwatch shadercache when it gets to around 115/120MB or so. If my understanding of this issue is correct, you can cache more than 128MB but if you do then on the next start of the game the cache will be wiped. If that is true I should be able to copy that in before sessions to be sure that I won’t have to rebuild the whole thing from scratch due to this limitation. The real fix for this obviously being to allow users to set an env var for shaderchache max size.

I also vote for having bigger cache dirs if technically possible.

Enhancement request tracked as NVIDIA bug 2109833.

This is a high priority bug. I’d be thankful if you implement it.

SCENARIOS:
Quake Champions: Unplayable until shader cache is full. But since this game shaders are so big, this process is repeated every time a new level loads. In multiplayer matches, this causes desconection and game crashes.
Cemu + Zelda Breath of the Wild: It requires about 500 MB of shader cache to run properly, so currently it runs super slow.

I’d also encorage you to increase the default amount of shader cache to 256MB.

I’d suggest an env variable override and a 1Gb cache by default…
Surely people want to leave buffer margins to avoid cache overflows…
Just more support headaches for Nvidia - after all…

It’s 2018 people - disk space is cheap (for reference I’ve 3Tb of SSD’s in my ROG laptop)!
I’d also choose to redirect a larger disc cache to tmp RAM anyway (since I’ve 32 Gb RAM).

Just my $0.02!

Bob

Yeah, I’m going to agree with BobbyWya here. I have plenty of disc space for the shadercache. And 1 GB cache for a good game is nothing, especially since it gives you a better experience. And if we can set the size bigger ourselves, it would be future proofish.

It is really lovely that the shader compilation is already faster now with 396.XX, thank you for that. Hopefully the cache size will be increased soon aswell.

Dox

Another vote for a larger cache size, an env variable to set a custom size seems to the best solution. Another example - Witcher 3 runs very good in Wine/DXVK however just 5 minutes into exploring the first area already makes up 9.2mb of shader cache, given the size of the game it wouldn’t be long till it reaches the limit.

Hi, I agree with the general sentiment of this thread. I would greatly appreciate the ability to increase the cache size to a user defined amount. While 128MB is fine for some games, some games require more than this. Because of this, a user should be able to set the cache size to any size they want.

I hope that you do increase the max shader disk cache (maybe even let us set it up ourselves using an environment variable), because the current one is really not enough.

I play Overwatch, and after some time the cache gets garbage collected and needs to be refilled, even though it has a dedicated one used only for it, with the current 128mb limit.

__GL_SHADER_DISK_CACHE_SKIP_CLEANUP=1 should remove all limitations to shader cache size, and that should work on most currently released drivers.
Configuration of the size of the cache will be exposed in later drivers, expect it to appear in our changelog.

Thank you for this answer. I was pointed towards this ENV variable this weekend aswell by Plagman (Pierre-Loup Griffais), and was testing it to see if it worked. Seems it was available for a bit already, but just never communicated (or at least nobody was able to find anything about it). Now that we have a green post, we have confirmation.

I think for all practical purposes __GL_SHADER_DISK_CACHE_SKIP_CLEANUP=1 will serve, unless there is some drawback of having it too big. Like loading time, or any other form of bottlenecking. We are going to test it and should we find any issues, we will report back. And I guess we will see the changed size in action in the future,

Thanks again,

Dox

Hello, this is very helpfull on Linux, but is there any way to do the same thing on Windows (10) ? Thank you.