The issue was indeed me running it headless.
This helped me: JetPack 4.3: MESA-LOADER: failed to open swrast while in xrdp session - #24 by sorlando961
In summary: I had to start a new X server on :0 and use x11vnc to bind the vncserver to :0 (instead of using (tiger) vncserver to creating a new session on :1. I was then able to render to :0 and run my applications on the GPU:
$ sudo service lightdm stop
then on separate shells:
$ X
$ x11vnc -auth /var/run/lightdm/root/\:0 -geometry 1920x1080