My /lib/modules/$(uname -r)/build is mismatch

Modules have to be built against a given kernel or they won’t load. Changing configuration of non-module aspects can alter module load, especially the CONFIG_LOCALVERSION changing. You might find these of interest: