I tried cross-compiling those three modules and received this error:
user@tegra-ubuntu:/lib/modules/5.10.104-tegra/extra/opensrc-disp$ sudo insmod nvidia.ko
insmod: ERROR: could not insert module nvidia.ko: No such device
Apparently those modules are not needed for Xavier according to @DaveYYY Insmod NVIDIA-kernel-module-source-TempVersion Failed - #9 by liwang54321
Currently, the last option I can think of is editing gst-nvargustcamerasrc to remove the dependency on EGLStreams and/or passing EGL_NO_DISPLAY. If there is something else you can suggest, please let me know.