Since you mentioned my lib I can tell you about my own experience playing with the Nano. Sure you can mmap() CPU registers as I did, not everything though, depending of what you are trying to map you are going to get a seg fault because IOMMU and such, the kernel is going to block you.
Then other challenge would be, how to capture interruptions from user space e.g. how to know when you finish an spi transfer or the like, that’s going to be a problem and actually for the i2c and spi ports, I changed my approach and I just ioctl() the character devices (used the driver that comes with the OS) but changing pinmux/cntrl registers programmatically instead of fighting the device tree.
Anyways, if what you want is just manipulate the pins, use the spi and i2c buses you can use my lib, just check the examples on github and take it from there, you can use c++. Feedback from other people has been good so far, of course you can also modify it or whatever.
If what you want is do your own thing from scratch, keep in mind all of the above, this is not an MCU, there is an OS/kernel between you and the metal and so to achieve certain things e.g. improve the spi behaviour like allowing continuous transmission or whatever, you will have to modify/create your own driver or character device and replace what is already there.
One thing that might happen is that you do your own thing in user space and you are fighting with the driver in the background, that is something to keep in mind.
Also essential to know the hardware, register addresses and so forth, get the Technical Reference Manual, you can find it in the download area.
This is kind of a brief all over the place, but there is a lot to it if you never programmed hardware on a linux environment.