opengl renderer problem

I’m working on an opengl renderer on ubuntu14.04, which receives opengl commands from a guest VM and apply them in host system. Each renderer process provides rendering service for one guest VM.

I manage to get 8 guest VMs run simutaneously and everything seems ok.
As the number goes to 9, it generates a glxMakeCurrent failure, which is weird, glGetError returns 0.

Now I get 4 nVidia GRID K2 installed. When the number reaches 12, all the renderers seem to be abnormal. I made some time count, sometimes the glxMakeCurrent call costs 2-3 seconds. Nvidia-smi runs really slow too, but it shows GPU-Util is less than 10%.

My goal is making at least 50 guest VMs running and the road seems to be really tough…

Any helps or suggestions would be greatly appreciated.

Here’s nvidia-smi output

Tue Aug 16 18:27:22 2016       
+------------------------------------------------------+                       
| NVIDIA-SMI 352.79     Driver Version: 352.79         |                       
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GRID K2             Off  | 0000:04:00.0     Off |                  Off |
| N/A   39C    P0    46W / 117W |    364MiB /  4095MiB |      5%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GRID K2             Off  | 0000:05:00.0     Off |                  Off |
| N/A   40C    P0    44W / 117W |   1322MiB /  4095MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GRID K2             Off  | 0000:83:00.0     Off |                  Off |
| N/A   44C    P0    43W / 117W |    851MiB /  4095MiB |      1%      Default |
+-------------------------------+----------------------+----------------------+
|   3  GRID K2             Off  | 0000:84:00.0     Off |                  Off |
| N/A   42C    P0    41W / 117W |    495MiB /  4095MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  GRID K2             Off  | 0000:89:00.0     Off |                  Off |
| N/A   28C    P8    17W / 117W |    323MiB /  4095MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   5  GRID K2             Off  | 0000:8A:00.0     Off |                  Off |
| N/A   42C    P0    43W / 117W |   1332MiB /  4095MiB |      5%      Default |
+-------------------------------+----------------------+----------------------+
|   6  GRID K2             Off  | 0000:8D:00.0     Off |                  Off |
| N/A   37C    P0    47W / 117W |   1310MiB /  4095MiB |      6%      Default |
+-------------------------------+----------------------+----------------------+
|   7  GRID K2             Off  | 0000:8E:00.0     Off |                  Off |
| N/A   48C    P0    44W / 117W |   1108MiB /  4095MiB |      4%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1722    G   /usr/local/Xorg/bin/Xorg                       215MiB |
|    0     14677    G   ...opengl_render/host/renderer/opengl_render    54MiB |
|    0     29131    G   ...opengl_render/host/renderer/opengl_render    53MiB |
|    0     36103    G   ...opengl_render/host/renderer/opengl_render    26MiB |
|    0     36622    G   ...opengl_render/host/renderer/opengl_render    18MiB |
|    1      1728    G   /usr/local/Xorg/bin/Xorg                       650MiB |
|    1     11851  C+G   ...opengl_render/host/renderer/opengl_render   187MiB |
|    1     16170  C+G   ...opengl_render/host/renderer/opengl_render   143MiB |
|    1     20994  C+G   ...opengl_render/host/renderer/opengl_render   281MiB |
|    2      1735    G   /usr/local/Xorg/bin/Xorg                       583MiB |
|    2      2540  C+G   ...opengl_render/host/renderer/opengl_render   138MiB |
|    2     14782    G   ...opengl_render/host/renderer/opengl_render    36MiB |
|    2     27951    G   ...opengl_render/host/renderer/opengl_render    55MiB |
|    3      1741    G   /usr/local/Xorg/bin/Xorg                       301MiB |
|    3     21122    G   ...opengl_render/host/renderer/opengl_render    57MiB |
|    3     21559    G   ...opengl_render/host/renderer/opengl_render    38MiB |
|    3     39115    G   ...opengl_render/host/renderer/opengl_render    38MiB |
|    3     40164    G   ...opengl_render/host/renderer/opengl_render    36MiB |
|    4      1747    G   /usr/local/Xorg/bin/Xorg                       170MiB |
|    4     17735    G   ...opengl_render/host/renderer/opengl_render    55MiB |
|    4     18751    G   ...opengl_render/host/renderer/opengl_render    38MiB |
|    4     21581    G   ...opengl_render/host/renderer/opengl_render    36MiB |
|    5      1752    G   /usr/local/Xorg/bin/Xorg                       599MiB |
|    5      2128  C+G   ...opengl_render/host/renderer/opengl_render   243MiB |
|    5     19257  C+G   ...opengl_render/host/renderer/opengl_render   201MiB |
|    5     26109  C+G   ...opengl_render/host/renderer/opengl_render   267MiB |
|    6      1757    G   /usr/local/Xorg/bin/Xorg                       647MiB |
|    6     11064  C+G   ...opengl_render/host/renderer/opengl_render   203MiB |
|    6     11355  C+G   ...opengl_render/host/renderer/opengl_render   167MiB |
|    6     12225  C+G   ...opengl_render/host/renderer/opengl_render   262MiB |
|    7      1762    G   /usr/local/Xorg/bin/Xorg                       551MiB |
|    7     12998  C+G   ...opengl_render/host/renderer/opengl_render   216MiB |
|    7     14793    G   ...opengl_render/host/renderer/opengl_render    36MiB |
|    7     35355  C+G   ...opengl_render/host/renderer/opengl_render   282MiB |
+-----------------------------------------------------------------------------+