Random segmentation faults from TensorFlow

I downloaded TensorFlow code that works fine in 2D, and then extended it to 3D. However, I randomly get segmentation faults that seem to point to libcuda. Sometimes the training runs for 5-6 hours and then ends with a segmentation fault, sometimes the segmentation fault comes after 2 minutes. Any ideas what the problem can be?