Thank you for reminding me of careful checks.
I ran __NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo | grep vendor and here is the output
server glx vendor string: SGI
client glx vendor string: Mesa Project and SGI
OpenGL vendor string: X.Org
I then ran glxgear in the same bash and checked the output of nvidia-smi. There was no application showing there.
Alternatively, I ran sudo prime-select nvidia and reboot but then I was stuck at the login page (it prompted up again every time after I tried to login).
Finally, I followed this answer to configure the files. But after reboot, nvidia-smi showed No running processes found (even no xorg running).
So just as you said the AMD/Nvidia combo is not fully supported by Ubuntu 18.04. It is ok for me to keep using the Nvidia GPU in render offload mode. But, since seemingly I failed to run selective applications (such as glxgear) on the Nvidia GPU manually, is the render offload mode actually equivalent to the sole-iGPU mode under this circumstance?