I did some google search and updated my egl-wayland to the latest on ubuntu20.04(1.1.3). I am using nvidia driver version 515.65.01, which should have the problem fixed in theory.
My surface is created with vulkan::ext::vkCreateWaylandSurfaceKHR(instance, &surfaceCreateInfo, NULL, &surface);
I noticed the fix for egl-wayland is in for 1.1.10, which is only available on ubuntu22.04? Does that mean vulkan will not be available for wayland on 20.04?