Finally after many frustrating attempts I found the issue and am posting here. Maybe it will help someone and save them some time.
It seems the 2GB Jetson Nano Development Kit will boot up from the SD card as long as the image is the perfect fit.
4gb image will not work on the 2gb Nano but the download page is not clear.
The 2gb image did not work either. The Nvidia logo showed up but it asked for a password to login.
Turned out version 4.62 was not compatible. It seems some of the older 2gb Development kit hardware will work only with version 4.61 of the SD Card image.