It turns out the printed paper come along with the Tegra TX1 box has wrong username and passwd. The print said it should be nvidia:nvidia, I checked the online document, it should be ubuntu:ubuntu. The latter works for me.
But it is such a stupid mistake that shouldn’t occur at all. :(