True - if you have your board behind a firewall or with an ISP (or even a cell modem carrier) that blocks port 22, you will have the same issues.
In that case, I would change the ssh port in Linux, and create a very strong password as it is likely any product on the internet will be attacked shortly after your Ethernet link light goes green.