I tried running sway-git and wlroots-git on the new 510.39.01 Beta and it seems to run fine. No asserts as far as I can tell.
Not sure if the driver update fixed it or an sway/wlroots update.
Worth checking out for you if updating sway/wlroots helps or the new driver.
Just for the record, sway 1.7 and wlroot 0.15 with nvidia 510 beta drivers still fails on argb8888 and xrgb8888. This commit on wlroots fixed the problem, that is why sway-git and wlroots-git works. The guy explains in the commit that the reason it fails on wlroots 0.15 is because the nvidia driver doesn’t support EXT_read_format_bgra.
So, next version of wlroots should work fine with the nvidia driver.