The ARMv8-A 64-bit has the equivalent of 32-bit ARMv7 hard float plus NEON (simply labeled ARMv8 without the “-A”) in compatibility mode. Consider this similar to a desktop CPU having 64-bit x86_64, plus 32-bit i686…x86_64 is the desktop native architecture, i686 is a foreign architecture. Adding armhf on a 64-bit ARMv8-A is exactly analogous to i686 compatibility on a desktop and is perfectly reasonable foreign architecture which the CPU supports.
Beyond that however, there are a lot o desktops which have tested and refined mixing i686 and x86_64. Under ARMv8-A that practice is untested. Under a desktop gcc actually combines 64-bit and 32-bit, under ARMv8-A you’d have to have separate discrete compilers if you wish to compile both architectures natively. To execute you’d need to also add armhf linker and binutils. How well they work together might be an issue or it might “just work”…if you are entertained by the experiment I’d say “go for it”, it’s interesting new grounds many people have wondered about when they think of building kernels natively on the Jetson. In your case it sounds like you just want to run 32-bit armhf and may not need the 32-bit compiler.