In our renderer, for visibility rays, we have been tagging rays with OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT and reporting positive visibility if the miss program gets called, occlusion if not. We are disabling closest and any hit programs.

This has been working well until we added (Catmull-Rom) curves. We are now seeing inconsistent results, where some shadow rays which should not report self-intersection because of backface culling, actually do, as if the internal curve intersector reports temp backface intersections, and is relying on the scheduler to ignore them.

Should we be using OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT with cubic curves? Should this actually work?

This is what it looks like (with the flag):

This is what it should look like (without the flag):


Hi @omaury, welcome!

Great question. So what you are seeing in the first image is not really related to backface culling, it is because we automatically reduce the precision of the curve intersector when using the terminate-on-first-hit flag. We expect that when using this flag, that the user cares about querying visibility accurately, but does not intend to render anything with the u and t return values.

We expect the clipping planes should be accurate, and the silhouettes should be accurate, but we assume that the hit point itself does not need to be accurate. This is because when using terminate-on-first-hit, you always have the possibility of getting hits that are out of depth order, so the hit point position might be arbitrary and unpredictable. For example, you have a couple of curves that visibly overlap, and if they got a little closer together, or if you had more curves, you’d probably start to see these out-of-order hits in your renders. Whether a hit is out of order normally doesn’t matter for a shadow ray, since you’re normally only testing visibility between a primary hit point and a sample on the light source. It usually makes no difference if there’s 1 occluder, or there are 2 or more occluders, and it normally doesn’t matter which occluder the ray returns, since that information will be discarded if we’re only checking whether the point is in shadow or not.

So, if you are using the terminate-on-first-hit flag only for shadow rays, and just got a little nervous when debug-visualizing the results, but you won’t be rendering the results outside of debugging, then you should be able to continue using the flag with cubic curves, and it should work fine. (If you do see any problems with the visibility query, please let us know.)

If you have a workflow where you need an accurate hit point for secondary / shadow rays, then in that case I would advise against using the terminate-on-first-hit flag, and that applies to all primitive types including triangles; the hit point is just never predictable when using terminate-on-first-hit, by design.

And generally speaking, I think you probably never want to use the terminate-on-first-hit flag for primary / camera rays, other than for testing / debugging / visualization.


Hi @dhart ,

Thanks for your welcome and response :) In the image showcasing the artefacts, only the shadow rays use the OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT flag. What you’re seeing is a hair scattering bsdf with shadow rays cast toward a sphere light on the right. All shadow rays originate at camera ray intersections plus a small offset in the direction of the light.

In other words, I’m not sure what we’re seeing fits your description: The image is not a debug image, we were expecting to get a valid rendered frame (Unless I’m not understanding your explanation?). That being said, if this flag degrades the accuracy of the curve intersections, this could very well be a precision issue creating false self-intersections at the origin of the shadow rays?


Oh I see, you understood my explanation, and I misunderstood the image. So in that case, yes my guess is the same as yours, that you’re seeing self-intersection artifacts, and they will indeed get worse based on this lower precision. Try using a small t_min value and/or pushing the rays forward by an epsilon and see if it goes away.

Assuming this is the issue, I don’t know if we have any guidance on what to use for the epsilon value outside of increasing it until the artifacts go away. We do have some code to use for this problem when rendering with triangles, but it won’t work for curves. I will ask around, including the author of the cubic curve intersector, and see if there are any better suggestions.


Great, thanks David!

Okay so a few thoughts here, but no super easy answer.

Speaking generally, there should always be an offset from a hit point to the ray origin of a shadow ray, or other secondary ray. Out of curiosity, did you use an offset or a non-zero t_min value, or were you starting the shadow ray exactly at the hit point and using t_min=0 for the pictures above? I just looked, and in one of our internal curve samples, we used a hard-coded t_min value of 2e-4. This is a hacky and less-than-ideal solution, but the code has survived this way for a couple of years without needing to change, so YMMV.

There are two ideas for how to create a better offset in the case of cubic curves that should probably be employed. First, the offset should probably move in the direction of the surface normal at the hit point, i.e., to place the new origin close to the hit point, while guaranteeing the new point is some epsilon outside the curve. (I’m sure you know that the calculated hit point can end up inside the curve, or slightly under the surface, due to floating point error and precision of the intersector and precision of the hit point calculation.) Second, the offset should probably be proportional to the curve radius, since errors in the intersector that can cause self-intersection problems are likely proportional to the radius.

With those two things, a small hard-coded factor should be somewhat reasonable. Mainly you will just want to make sure the factor is large enough to eliminate the self-intersection problems while being small enough to avoid any visible light leaks anywhere. It really depends on your scene, but in the case of curves, sometimes light leaks are less of a problem than with triangle meshes, since curves are less likely to overlap and form corners where leaks will be visible.

Now, if you did these things, your new image will end up looking different from your 2nd image above (the reference image without the early terminate flag.) This is because adding the offset will introduce (physically correct) self-shadowing of the curves; it will put the larger specular highlights that are on the back side of the curves relative to the light into shadow. You probably know this already, but just for completeness and for the benefit of others reading, I’ll mention that this does bring up a tiny point about casting shadow rays you might want to consider, if you didn’t already - most people will call a surface shadowed if the surface normal points away from the light source sample. In other words, you wouldn’t even cast a shadow ray in the first place, if the dot product between the vector towards the light and the surface normal at your hit point is negative. If that trick were employed in the above images, you might not have noticed any issues with the early terminate flag, since it looks like most or all the artifacts are on the side that should be in shadow.

I hope that helps. Please let me know if adding an offset actually solves the problem, or whether I’m speculating and ranting about the wrong issue. If the offset does work, we would be interested to hear back if you have trouble using an offset, and whether you would prefer that the precision of the shadow-ray curve intersector matched the camera-ray curve intersector exactly. We believe this will not get rid of the need for an offset between the camera ray hit point and the shadow ray origin, but it would probably reduce the magnitude of the offset needed. The tradeoff for greater precision is just slightly higher run time for the intersector - we can improve the precision of the curve intersector’s shadow ray calculations at the cost of a little bit of performance. We are open to tuning the balance here based on user priorities like yours.


Thanks for thinking more about this David, and sharing your thoughts. A couple comments in response to your questions:

  • “do we use a shadow bias?”: Yes we do, and although its value has not been a problem until now (before curves), I think it’s actually a bit small, it’s something like 1e-6. We can play with this value, this might actually be a way to keep the flag.
  • “should we be calling a shadow ray if N.L < 0?”: For hair scattering yes. As you know, hair fibers are transmissive (TT lobe) so it makes sense to probe the scene for light visibility from any azimuthal direction. Ideally, in this case, we would move the shadow ray origin to the other side of the curve, but we are counting on the back face culling of cubic curves to treat shadow ray offsets uniformly for meshes and curves.

Thanks again!

1 Like

Gotcha, thanks for the addendum! So if this is for hair, depending on your accuracy & precision needs, another hacky but simple idea is you could potentially consider casting the secondary rays from the curve axis evaluated at parameter u, instead of the hit point, and continue to rely on the rays to usually escape based on the curves being backface-culled by default. This hacky approximation might work well if your production hair is always thin and sub-pixel on screen, but obviously not so well if you’re doing near-field close-ups similar to the images above, or need higher accuracy for other reasons.


Are you able to share the test scene, or the curve data from the test scene in your images? (Just so we can reproduce your exact issue, and study it a little.)


Sure, this is the curve data (where the 4th number is the radius):

curves_plane_debug.txt (20.5 KB)

The renderer uses these vertices as CVs for Catmull-Rom curves. They are standing on a 2x2 x-y plane located at the origin.

Hope this helps,

1 Like

Just as another example to look at, my MDL_renderer example supports cubic B-splines for hair rendering with MDL’s hair_bsdf() (Chiang model).

That example is using anyhit programs for shadow ray testing though because the renderer needs to support cutout opacity.
The rtigo10 example in the same repository is using faster shadow rays with OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT but that doesn’t have curve primitives.

But I’m also relying on the fact that OptiX curves are not hitting back faces and only use the scene epsilon to offset the continuation or shadow rays and that works fine:

Hi Olivier -

Thanks for the curve data! Could you share the light source info as well, and anything you can about how you calculate your shadow ray offsets? A couple of people on our team tried reproducing the shadowing artifacts you saw, and we haven’t seen them just yet. We have a potential improvement in the works for the curve intersector, and we’d like to verify it would actually make a difference for you in this specific case.


Hi David,

In mitsuba, the light is defined as:

<shape type="sphere">
	<point name="center" x="1.4055345058441162" y="0" z="1.1138004064559937"/>
	<float name="radius" value="0.1"/>
	<emitter type="area">
		<rgb name="radiance" value="50"/>

And the camera:

<sensor type="perspective">
	<string name="fov_axis" value="x"/>
	<float name="fov" value="39.597755"/>
	<float name="principal_point_offset_x" value="0.0"/>
	<float name="principal_point_offset_y" value="0.0"/>
	<float name="near_clip" value="0.100000"/>
	<float name="far_clip" value="1000.000000"/>
	<transform name="to_world">
		<rotate x="1" angle="99.66544418009823"/>
		<rotate y="1" angle="-0.5811123345021262"/>
		<rotate z="1" angle="176.09562299256692"/>
		<translate value="-0.330339 -4.028209 1.537609"/>
	<sampler type="independent">
		<integer name="sample_count" value="512"/>
	<film type="hdrfilm">
		<integer name="width" value="1024"/>
		<integer name="height" value="1024"/>
		<rfilter type="box"/>

For the offset, you can see here:

Hope this helps,

Thank you! I’ll let you know what we see.


Hi Olivier,

Several of us have tried, and so far we’ve been unable to reproduce the issue in your first image. A couple of minor changes that improve the accuracy of OptiX cubic & quadratic curves when using OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT have been made, which might help you if/when we release them. It’s not necessarily critical that we reproduce the issue exactly, but it would still be nice to repro and fully understand the issue you saw, and validate that these small changes will help with certainty.

I’m wondering if the problem might lie somewhere else. I should have asked earlier, but since you’re changing the curve type, are you also updating the surface normal calculation to match the new curve type? Are you using the surface normal evaluation code we provide in the optixHair sample? Just asking since that might be an easy thing to overlook, and the shadow samples could be pretty sensitive to surface normal.

Any chance you could render a closeup of the artifacts? Like maybe zoomed in on the section with the large specular highlight on the rightmost curve in the top picture?


Hi David,

A couple of minor changes that improve the accuracy of OptiX cubic & quadratic curves when using OPTIX_RAY_FLAG_TERMINATE_ON_FIRST_HIT have been made, which might help you if/when we release them.

Great, we’ll wait for next release and test again with the flag.

Are you also updating the surface normal calculation to match the new curve type?

Yes, as far we know: As you can see from image, the normal looks correct, at least what we expect it to be for a thick Catmull Rom curve.

Any chance you could render a closeup of the artifacts? Like maybe zoomed in on the section with the large specular highlight on the rightmost curve in the top picture?

I’ll try to get a close up when I revisit this, but it might not be soon.