Hello Forum,
I have a simple test program that renders one alpha-tested quad using vullkan RT, it should like this without applying OMM:
I have an any-hit shader that looks as follows.
#version 460
#extension GL_EXT_ray_tracing : require
#extension GL_GOOGLE_include_directive : enable
#include "raycommon.glsli"
layout(location=0) rayPayloadInEXT hitPayload prd;
hitAttributeEXT vec2 attribs;
layout(binding=3) uniform sampler2D texSampler1; // alpha map
struct Vertex {
vec2 pos;
vec3 color;
vec2 uv;
};
layout(std430, binding=4) readonly buffer VertexBuf {
Vertex vertices[];
};
layout(std430, binding=5) readonly buffer IndexBuf {
uint indices[];
};
void main() {
// read UV
uint idx0 = indices[gl_PrimitiveID * 3];
uint idx1 = indices[gl_PrimitiveID * 3 + 1];
uint idx2 = indices[gl_PrimitiveID * 3 + 2];
vec2 uv0 = vertices[idx0].uv;
vec2 uv1 = vertices[idx1].uv;
vec2 uv2 = vertices[idx2].uv;
const vec3 bary = vec3(1.0 - attribs.x - attribs.y, attribs.x, attribs.y);
const vec2 uv = uv0 * bary.x + uv1 * bary.y + uv2 * bary.z;
vec3 t = texture(texSampler1, uv).xyz;
if (t.x < 0.5) {
ignoreIntersectionEXT;
} else {
}
}
However, when I apply OMM to it, the render results become blank with all miss color. I took a closer look:
- I found the
UV
showing up in the any-hit shader are all(0, 0)
instead of their normal values before applying the OMM. This causes the any-hit shader to think all rays fall on to the transparent part of the alpha map. - The reason the
UV
s are all(0, 0)
is thegl_PrimitiveId
becomes a very large, invalid value, rather than 0 or 1 as it should before applying the OMM. - If I remove
ignoreIntersectionEXT
in the any-hit shader and assign the payload withprd.hitValue = vec3(uv, 0);
, I see an outline of the OMM: - The OMM shows up in NSight Graphics correctly:
- The barycentrics (
attribs
) are correct. - I checked the OMM integration guide, it suggests using
attribs
alone is enough for obtaining the UV:const float texCoords = GetTexCoords(barycentrics);
. How is this achieved? - I’m using the CPU OMM baker with a very low subdivision level of 2, the bake results match the appearance of the OMM in NSight.
- Making each BLAS containing only 1 triangle and using
gl_InstanceID
to index into thevertex
array works, so it seems that onlygl_PrimitiveID
is broken:
*
I’m using an RTX3060.
What could I have done wrong?
Thanks for your viewing this post and input!