Sliding a procedural block on an heightfield terrain type (uniform surface)?

I created a height-field terrain with 0 height everywhere – I’m assuming this is a uniform surface – and I try to slide a the procedural asset box over it (applying forces to the center of the box). The motion here is dissimilar to that of sliding the box on default ground plane. The motion is staggered, much like it’s interacting with something on the surface. I tested this with 0 static/dynamic frictions as well, similar observations. Am I missing something here? or has anyone else also come across this?

Is this something to do with the amount of horizontal scaling of the surface?

Really appreciate the help!

PS. this not a triangle mesh. I use the add_heightfield api.