Image load/store format affects precision?

I current voxelize a scene and mix it with a previous frame. Currently ColorOut and ColorIn have the GL_R11F_G11F_B10F format. Before the compute shader, colors are normal, and performing a simple copy (eg. imageStore(ColorOut, texel, imageLoad(ColorIn, texel))) retains correct color. Using mix() skews colors.

vec3 previousColor = imageLoad(ColorOut, texel).rgb;
vec3 currentColor = imageLoad(ColorIn, texel).rgb;

vec3 color = mix(previousColor, currentColor, VOXEL_TAA_WEIGHT);

imageStore(ColorOut, texel, vec4(color, 1.0));

The lower VOXEL_TAA_WEIGHT is, the more grossly shifted colors become. Changing the format of the textures to GL_RGBA16F or GL_RGBA32F produces expected results.

The upper image shows mix() with GL_R11F_G11F_B10F, the lower RGBA16F.

Any insight or workarounds are well appreciated.