Note that the kernel image itself loads its beginning physical address at 0x81000000. Below 0x81000000 physical address is the initrd, and below this is module space. The total space below the kernel which comprises the initrd plus module space must not exceed 32MB. Although I had assumed modules were being pushed beyond 32MB via the initrd size, it looks like the initrd is trying to decompress and overwriting the lower kernel physical address.
I think my first suggestion actually went the wrong direction, as I was looking for module boundaries, but the issue is the initrd being too large. Modules too far out can’t branch to the kernel, initrd too large overwrites the kernel in the lower address space. Changing allowed initrd from 24MB to 16MB would actually give it less room when it was already overwriting with 24MB.
I do not know of any kernel configuration to give more than 24MB to initrd. Something may exist, but it seems you have no choice but to cut down the initrd size, and probably go back to the …LESS_24M configuration. I’m not sure but it may be possible to squeeze a bit more space out for initrd if you completely remove kernel module support (that’s a pretty big loss).