No, this wasnt the issue.
The solution for my problem was provided here:
particularly this:
sudo update-binfmts --import qemu-aarch64
sudo update-binfmts --import qemu-arm
sudo update-binfmts --import qemu-armeb
seemed to resolve the issue.
Maybe someone should update the instructions for running this with WSL.
This made it work for me on WSL2