NVIDIA GPU usage over SSH

Hi,

I have a server running Ubuntu 23.10, with a nvidia gpu, aswell as an igpu. Now when I login into the server with the GUI directly on the server using a monitor, mouse, keyboard etc, I get:
sr650:~/$ glxinfo | egrep “OpenGL vendor|OpenGL renderer”
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer: NVIDIA GeForce RTX3060/PCIe/SSE2

Now the weird part, and the issue. If I run the same command through an SSH terminal I get:
sr650:~/$ glxinfo | egrep “OpenGL vendor|OpenGL renderer”
OpenGL vendor string: Mesa
OpenGL renderer string: llvmpipe (LLVM 15.0.7, 256 bits)

The consequence is that when I run gstreamer with an opengl command, it doesn’t use the nvidia gpu, but rather the llvmpipe, and I can’t make it not different somehow… the gpu in use is anywhere from a rtx3060 to an RTX A5000. All resulting in the same behavior.
How would I go in attacking this issue?

Many thanks.