Sounds more like the host PC being 18.04 versus 20.04 which you are interested in. It is the GUI front end, JetPack/SDK Manager, which has this requirement. I think command line flash will work with 4.6.2 on a wider range of hosts versus GUI.
If everything has been unpacked and set up correctly (which is something SDKM would do for you), then flashing might be as simple as something like this (from the Linux_for_Tegra/ directory): sudo ./flash.sh jetson-xavier mmcblk0p1
(there can be some minor variations in the target name)
This would not give you options to install extras like CUDA. You would then have to do this yourself as an extra step.