Hi anmelle,
Are you using the devkit or custom board for Orin Nano?
I would suggest you also applying the patch from 40hdr - SPI1 gpio padctl register bit[10] effect by gpiod tools in JP6 - #20 by KevinFFF since you want to control GPIO and don’t want it becoming SFIO.
You can configure the pin as GPIO/Bi-directional in pinmux spreadsheet for this use case.
Please refer to Changing the GPIO Pins for details.