I re-installed the Nano OS today from scratch to remove any uncertainty. Here is what works and don’t so:
I first had to modify my .bashrc file to fix the missing cuda problem (noted here), I was then able to verify that cuda was now working by checking nvcc --version.
I then passed onto the camera module (IMX477) and followed the guide here. Checked and everything was working, camera was available and the controls (PTZ) were all functional.
However I need to be able to use the camera in an environment with other libraries hence the virtualenv, I then had build opencv from source (used this) since gstreamer was not by default using pip, I also enabled GTK with QT. Then whenever I try to use the same demo code from earlier It gives that error code. I used the same opencv version (4.1.1) from when using no env.