Apply rendering jitter for TAA without projection matrix

Gaming and Visualization Technologies APIs Vulkan
1

I’m trying to implement temporal anti-aliasing for an application. It’s an odd application where I can intercept and modify “fixed function” properties of a pipeline like the viewport, but I have no way to discern or modify the projection matrix. So I’m trying to figure out if there’s a way to apply rendering jitter for TAA without the projection matrix.
For my first attempt I applied the offset to the viewport, which did cause the screen to jitter, but the offset is different at different depths, which doesn’t help.
Now I’m trying to apply the jitter to the sample location, which I think should work exactly the way that TAA expects, however I don’t actually see any jitter being applied to the rendering.
This is what I’m doing at pipeline creation:

VkPipelineSampleLocationsStateCreateInfoEXT multisample_locations_ci{
    .sType = VK_STRUCTURE_TYPE_PIPELINE_SAMPLE_LOCATIONS_STATE_CREATE_INFO_EXT,
    .sampleLocationsEnable = true,
    .sampleLocationsInfo{
        .sType = VK_STRUCTURE_TYPE_SAMPLE_LOCATIONS_INFO_EXT,
        .sampleLocationsPerPixel = VK_SAMPLE_COUNT_1_BIT,
        .sampleLocationGridSize{1, 1},
        .sampleLocationsCount = 1,
        .pSampleLocations = &jitter,
    },
};
multisample_ci.pNext = &multisample_locations_ci;
dynamic_states.push_back(VK_DYNAMIC_STATE_SAMPLE_LOCATIONS_EXT);

The multisample and dynamic state infos are properly chained to the pipeline creation info.
Then before drawing I do:

cmdbuf.SetSampleLocationsEXT(VkSampleLocationsInfoEXT{
    .sType = VK_STRUCTURE_TYPE_SAMPLE_LOCATIONS_INFO_EXT,
    .sampleLocationsPerPixel = VK_SAMPLE_COUNT_1_BIT,
    .sampleLocationGridSize{1, 1},
    .sampleLocationsCount = 1,
    .pSampleLocations = &jitter,
});

And I know from debugging that the value of jitter is updating from frame to frame the way I expect it to with this pattern:

std::vector<VkSampleLocationEXT> jitter_phases;
...
jitter_phases = {
    {1, -1},  {1, 3},   {-3, -1}, {3, -3},
    {1, 1},   {-3, 1},  {1, -3},  {3, 3},
    {-1, 1},  {-1, -3}, {3, 1},   {-3, 3},
    {-1, -1}, {3, -1},  {-1, 3},  {-3, -3},
};
for (auto& [x, y] : jitter_phases) {
    x /= 8.0f;
    y /= 8.0f;
    x += 0.5f;
    y += 0.5f;
}

I also fixed any errors related to using custom sample sample locations with depth buffers.
So is there something I’m missing that would cause the pipeline to ignore sample locations? Or am I completely misunderstanding the way “MSAA” sample locations work, and this won’t apply an offset to the rendered image?

2

I’ve figured out that the custom sample locations are working; however, not in the way I need them to work. I’ve modified the SaschaWillems multisampling example and I can see that the edges of the object do shimmer, but the textures appear to be static. Is it normal that MSAA sample locations don’t affect the fragment inputs, like texture UVs?

3

Aha, the fragment inputs need the sample decorator.