An update, problem solved.
All I need to do is to add cuda driver’s path (in my case /usr/lib/nvidia-375) to the LD_LIBRARY_PATH.
A side problem is the libGL.* in nvidia driver will take priority over mesa libGL. The fix is to remove or rename/relocate the libGL.so*/libGLX.so*/libGLdispatch.so* to a different folder, or not to install them if you install from the .run file (I prefer to install from apt-get).
In summary, in order to make this to work, you need to
- make sure you have enabled onboard graphics in the BIOS settings (or set it as primary)
- install both xorg intel driver and nvidia/cuda drivers
- start nvidia-settings, and go to the PRIME settings page, set Intel (Power Saving Mode) as default
- modify your .bashrc and set LD_LIBRARY_PATH to at least contain /usr/local/cuda/lib64:/usr/lib/nvidia-XXX where XXX in my case is 375.
- logout to restart X or reboot
- run ldd $( which glxinfo ) to make sure your GL libraries point to mesa, or run glmark2 to confirm GL status
- (update) if the libGL printed from step 5 points to nvidia’s driver folder, you need to remove/rename the libGL.so*/libGLX.so*/libGLdispatch.so* under nvidia driver folder so that your OS can pick up the mesa libGL library.
- run nvidia-smi to list your dedicated NVIDIA GPU, and run your CUDA program, you should not see any errors.