Hi,
i deployed my model on linux server with two GTX1080ti GPUs, it serves normally in about 50 days. but recently it crashed twice.
syslog info:
kernel: [20562525.414620] python2.7[124140]: segfault at 0 ip (null) sp 00007f7f9bffef10 error 14 in nvidiactl[200000000+200000]
nvidia-smi:
±----------------------------------------------------------------------------+
| NVIDIA-SMI 390.42 Driver Version: 390.42 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 108… Off | 00000000:03:00.0 Off | N/A |
| 23% 39C P2 77W / 250W | 1893MiB / 11178MiB | 96% Default |
±------------------------------±---------------------±---------------------+
| 1 GeForce GTX 108… Off | 00000000:82:00.0 Off | N/A |
| 23% 35C P2 55W / 250W | 5419MiB / 11178MiB | 0% Default |
±------------------------------±---------------------±---------------------+
any idea of this problem??? thx