Does differing host/network byte ordering on AGX add to network latency?

This is inside the kernel. There is a tegre_defconfig file to define which options are enabled/disabled in our kernel driver.

Download the kernel source code from

https://developer.nvidia.com/embedded/linux-tegra-archive

Follow the document here to build the kernel on the host.

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/kernel_custom.html#wwpID0E0WD0HA

Replace your kernel to /boot/Image on the device and reboot.

And just some precaution that rebuilding kernel has chance to crash the system so using uart serial console might be helpful.