Running out of video memory when running simultaneous OpenGL processes

Hey guys.

I’m having trouble running several OpenGL processes at the same time on Linux. What I want is for one of them to be active and fullscreen, and the other ones to stay in background and not affect the performance of the foreground process.

Right know I am unmapping them in X and even suspending them by killing them with SIGSTOP to enure that they do not get any cpu time. The problem is that they still seem to reside in video memory, as far as i can see when running nvidia-smi tool.

This causes problems for the OpenGL processes starting later, that can not get the vram that they need. The driver does not seem to free any of the video memory used by the suspended processes, which was my impression that it would, to allow several processes to use the gpu.

Is there any way to free gpu memory without terminating the processes?



No, suspending a process or unmapping its window does not cause that process to release its memory allocations. You can force that by doing a mode switch or a VT switch.

Thanks for the reply

I tried the following:
Suspended a few OpenGL apps in one x-session, noted vram usage from nvidia-smi, and then tried each of these:

  1. Tried to change resolution back and forth, but no drop in vram usage.
  2. Switch vt to console, login and look at nvidia-smi. Vram usage was the same.
  3. Switch back from console to x and try to run another OpenGL app. Performance of the new app was bad, because it was not able to allocate sufficient vram. Nvidia-smi values still hadn’t dropped.
  4. Created additional x-session, switched to it, and tried running opengl apps there while the others were suspended in the first x-session. Still bad performance and high vram usage.

I experienced multiple crashes of the xserver during the above tests. The crashes do not occur when switching resolution, vt or x-sessions without any opengl apps open.

Any suggestions?

Can you please provide the application or a representative sample, and an nvidia-bug-report.log.gz file generated after the X server crash has occurred?

The problem can be recreated using the Unity3D game: angrybots, which is a demo project incluced with unity. I have created a linux build, that i have attached.

To make x crash, i just start enough instances of the game up, to fill my vram, which i monitor using nvidia-smi. Then do a vt-switch, and try to switch back which causes the crash.

I have attached nvidia-bug-report.log.gz from before and after the crash, and also Xorg.0.log.old, because im not sure that the script included it. It contains the crash in the last part of the file:

[ 58066.488] (EE) NVIDIA(0): Failed to allocate primary buffer: out of memory.
[ 58066.488] (EE) NVIDIA(0):  *** Aborting ***
[ 58066.565] (EE) 
[ 58066.565] (EE) Backtrace:
[ 58066.566] (EE) 0: /usr/bin/X (xorg_backtrace+0x36) [0x7f4b438a7476]
[ 58066.566] (EE) 1: /usr/bin/X (0x7f4b436f7000+0x1b42b9) [0x7f4b438ab2b9]
[ 58066.566] (EE) 2: /lib/x86_64-linux-gnu/ (0x7f4b427fa000+0xfbd0) [0x7f4b42809bd0]
[ 58066.566] (EE) 3: /usr/lib/xorg/modules/drivers/ (0x7f4b3c995000+0x84286) [0x7f4b3ca19286]
[ 58066.566] (EE) 4: /usr/lib/xorg/modules/drivers/ (0x7f4b3c995000+0x49ad35) [0x7f4b3ce2fd35]
[ 58066.566] (EE) 5: /usr/lib/xorg/modules/drivers/ (0x7f4b3c995000+0x4a0114) [0x7f4b3ce35114]
[ 58066.566] (EE) 6: /usr/lib/xorg/modules/drivers/ (0x7f4b3c995000+0x4acb12) [0x7f4b3ce41b12]
[ 58066.566] (EE) 7: /usr/bin/X (xf86Wakeup+0x43a) [0x7f4b4378e38a]
[ 58066.566] (EE) 8: /usr/bin/X (WakeupHandler+0x6b) [0x7f4b43753d7b]
[ 58066.566] (EE) 9: /usr/bin/X (WaitForSomething+0x1a4) [0x7f4b438a4884]
[ 58066.566] (EE) 10: /usr/bin/X (0x7f4b436f7000+0x58811) [0x7f4b4374f811]
[ 58066.566] (EE) 11: /usr/bin/X (0x7f4b436f7000+0x4757a) [0x7f4b4373e57a]
[ 58066.566] (EE) 12: /lib/x86_64-linux-gnu/ (__libc_start_main+0xf5) [0x7f4b41447ea5]
[ 58066.566] (EE) 13: /usr/bin/X (0x7f4b436f7000+0x478c1) [0x7f4b4373e8c1]
[ 58066.566] (EE) 
[ 58066.566] (EE) Segmentation fault at address 0x25
[ 58066.566] 
Fatal server error:
[ 58066.566] Caught signal 11 (Segmentation fault). Server aborting

Found out that i could attach the game instead of linking to it.

Can’t get attachments to work. They just say [SCANNING] forever.
I have uploaded all relevant files to:

Xorg.0.log.old.txt (42.9 KB)

Can you confirm that you received the information and files, aplattner?