From the user which you are using on your host, and from “sudo” of your host, try to connect manually on command line for both Jetson side accounts “nvidia” and “ubuntu”. Probably in some case it’ll want to know if you accept this unknown connection (and you’d need to answer “yes”):
…there may be combinations of source and destination accounts which are waiting for “yes” answer and you can’t see it. Once you answer “yes” it shouldn’t ask again unless something happens to the network.