Interactive instancing

I’m trying to interactively paint instances of a mesh onto to some geometry but performance is too slow
https://youtu.be/wcsva9HcIeA

Frames are normally rendering in 10-15 ms but when a new Transform node for the instanced geometry is added then the rendering time jumps to over 1 second.

ScopedStopWatch::~ScopedStopWatch() t7660]	"OptixPaint::renderScene took 0.010392 seconds"
ScopedStopWatch::~ScopedStopWatch() t7660]	"OptixPaint::renderScene took 0.01001 seconds"
[ActiveOptix.cpp:201 ActiveOptix::processItem() t7660]	"----- New item is arriving  Bunny::4"
ScopedStopWatch::~ScopedStopWatch() t7660]	"OptixPaint::renderScene took 1.41146 seconds"
ScopedStopWatch::~ScopedStopWatch() t7660]	"OptixPaint::renderScene took 0.014292 seconds"

Here’s the code I’m using to add the instances to the scene.

void OptixPaint::addDynamicInstance(ItemHandle item)
{
	ItemHandle source = item->from();
	
	ItemID renderID = source->getRenderID();
	auto it = dynamicMap.find(renderID);
	if( it != dynamicMap.end())
	{
		Transform sourceTransform = scene->getChild<Transform>(renderID);

		// make a new geometry instance
		GeometryInstance inst = context->createGeometryInstance();

		// reuse the geometry from item being instanced
		GeometryGroup sourceGroup = sourceTransform->getChild<GeometryGroup>();
		GeometryInstance sourceInst = sourceGroup->getChild(0);
		Geometry g = sourceInst->getGeometry();
		inst->setGeometry(g);

		// use the source material
		MaterialHandle m =  materialHandler.getItemMaterial(source->getID());
		inst->setMaterialCount(1);
		inst->setMaterial(0, m->getMaterial());

		// create a new geometry group and add the new instance
		GeometryGroup group = context->createGeometryGroup();
		group->setAcceleration(context->createAcceleration("Trbvh"));
		group->addChild(inst);

		// create a transform node for the item's world transform
		Transform t = context->createTransform();
		t->setMatrix(true, item->spaceTime().worldTransform.matrix().data(), nullptr);
		t->setChild(group);

		bool motionBlur = props.getVal<bool>(OptixKeys::motionblur);
		if (motionBlur)
			computeMotionBlur(item, t);

                 // add to root node
		item->setRenderID(scene->addChild(t));
		
		dynamicMap.insert(std::make_pair(item->getRenderID(), item));
	}
}

Am I doing something stupid again or is this the best Optix can do?

Also at the end of the video, I’ve added some motion blur. Can someone tell what the best strategy would be for smoothing out the motion blur for each frame?

Thanks!

The 1 second delay looks like an OptiX recompile of the scene. You can confirm this using the usage report feature (see the -r flag on optixMeshViewer).

Basically on the first frame OptiX doesn’t know you’re going to add a bunch of instances in the future, so it puts the scene on the device in such a way that it’s faster to trace on the current frame, but harder to change for the next frame without recompiling. Let me think about how to best get around that here. Can you put a tight upper bound up front on the max number of instances to be created?

One side note: all instances of the same Geometry/Material can point to the same acceleration structure on line 27, since this part of the scene graph is below the Transform, and thus identical. This won’t solve your main problem with the recompiles though. And you’re already correctly sharing at the GeometryInstance level, that’s good.

For your motion blur question: motion blur can be smoothed out by shooting more subpixel samples if you have the time budget (film renderers shoot hundreds of rays per pixel for final frames), or by being really clever about picking the subpixel samples, e.g. using methods like stratified sampling or QMC.

Looking at the app in the video, could you maybe have a mode where you freeze the animation, enable motion blur, and let it refine as in the optixMotionBlur SDK sample? It will converge.

“The 1 second delay looks like an OptiX recompile of the scene. You can confirm this using the usage report feature (see the -r flag on optixMeshViewer).”

Yes, that’s exactly what’s happening.

“Can you put a tight upper bound up front on the max number of instances to be created?”

I’d prefer keep it open ended if possible. But I’ll take whatever you can give me. :)

“One side note: all instances of the same Geometry/Material can point to the same acceleration structure on line 27”

I thought that might be the case but the Instancing sample creates a new acceleration structure for each instance so that’s what I did too.

for( unsigned int row = 0; row < num_rows; ++row ) {
        for( unsigned int col = 0; col < num_cols; ++col ) {
            GeometryGroup gg = context->createGeometryGroup();
            gg->setAcceleration( context->createAcceleration( no_accel ? "NoAccel" : "Trbvh" ) );
            gg->setChildCount( 2 );

Thanks for the help!

Good point on optixInstancing not sharing accels, I’ll change that.

As a workaround for the unpredictable compile, you can create a pool of instances up front and then attach them to the scene on demand. I’ll email you an example of that later today.

Thanks for the example scene! I have the pool working in my project but Optix always pauses to recompile after 1st instance is added and after the 20th or so instance is added. Same thing happens in your example scene. You can see it if you activate the user_report_level and set the pool_size to 30 and keep adding instances with the space bar. Is there any way to get around that?

“Looking at the app in the video, could you maybe have a mode where you freeze the animation, enable motion blur, and let it refine as in the optixMotionBlur SDK sample? It will converge”

I ended up added a user variable for AA passes that loops that many passes in the ray generation program and it works pretty well for my needs.

Here’s several hundred instances and 16 AA passes per frame
https://www.youtube.com/watch?v=E_rzhhaJftM&feature=youtu.be

The pause after adding the first instance is because you are adding a new program to optix. Optix needs to re-JIT code to add in this new program. You can usually avoid this pause by making sure that all necessary RTprograms are present at startup, even if they are not traced against (eg, a Geometry can be created with zero primitives).

I’ll look into the pause after the 20th instance. Keith is probably right about the first pause.

Thanks!

I’m not clear about what Keith means though.

In my project, all the Optix Routines that I will use are created and added to the Context during an initialization step before rendering starts. Then a ground plane is created and I see the expected Optix recompile. Then the user interactively adds an item to be instanced. I create a pool of instances using Routines that are already in the system then but don’t add the instances to the Top Group as in Dylan’s example. Optix recompiles after this as expected.

Then when the user starts paint instances, the first and the 20th instance cause a recompile delay … but all the other instances appear instantly as expected.

Thanks again for the help!

For the recompile as the first instance is added:

Do you have a different program (closest hit, intersection, anything really) on the instances vs. on the ground plane? If so, then that new program will become visible to the graph only at the time you attach the instance to the scene. It is not visible in the instance pool because there is no way to hit the instance with a ray, so OptiX ignores the unused programs.

For the recompile on instance 20 in my example:

Nice observation, I didn’t catch that one. I see why it’s happening, but don’t want to get too deep into details – it’s related to how OptiX chooses to store scenes of different sizes in memory. I’m less optimistic about finding a workaround for that. I’ll email you if I find one.

“Do you have a different program (closest hit, intersection, anything really) on the instances vs. on the ground plane?”

Yes, the instances and ground plane use different programs but by the time I pop the first instance off the pool and into the scene, the object being instanced has already been added to the scene. The instances all use the same intersection programs, material and acceleration structure as the object being instanced.

Too bad about the recompile on instance 20 … that’s a show stopper for me. I hope that’s a workaround down the road.

Thanks for taking the time to look into this!

Everything is working great now when doing simple ray tracing but when I switch to PathTracing, using the basic code from the Denoiser sample scene, the instances are rendering black. Any ideas why that is happening.

https://youtu.be/rAnl7GFhs5M

Wild guesses:

  • You exceeded the reserved stack space and the new instances throw exceptions. (The per ray data in that path tracer is bigger than the original one.)
  • The material assignments for the new instances is broken or your material parameters are black.
  • The per ray type closesthit, potential anyhit, or the miss program aren’t properly assigned.
  • Your maximum path length is too low to hit a light. (Well, with primary rays only, i.e. path length 1, diffuse materials should get direct lighting nonetheless. Maybe the material is specular.)
  • The closest hit program does not transform the vertex attributes (esp. normals) into world coordinates and the lighting breaks.
  • The normals are unnormalized after transforming them into world space.
  • Your Transforms have a mirror operation and the transformed sufaces are inside out and your closest hit program doesn’t turn the normals to the face the ray is looking at.

Thanks for the help!

I pretty sure it’s none of your guesses though.

I’m reusing the same geometry and material that the item being instanced is using and that item renders fine in the PathTracer. The instances render black even when there’s just 1 item and 1 instance in the scene.

The simple raytracing renderer works fine and shares the exact same code for creating the instance pool that the PathTracer uses:

Before I try to modify a sample scene to try to reproduce it there, can you think of anything else to check?

ItemID index = sourceItem->getRenderID();
	Transform sourceTransform = scene->getChild<Transform>(index);
	try
	{
		sourceTransform->validate();
	}
	catch (const optix::Exception & e)
	{
		LOG(CRITICAL) << e.getErrorString();
		continue;
	}
	
	// reuse the geometry and material from item being instanced
	GeometryGroup sourceGroup = sourceTransform->getChild<GeometryGroup>();
	if (!sourceGroup->getChildCount()) continue;

	GeometryInstance sourceInst = sourceGroup->getChild(0);
	if (!sourceInst) continue;

	Geometry sourceGeometry = sourceInst->getGeometry();
	if (!sourceGeometry) continue;

	try 
	{
		sourceGeometry->validate();
	}
	catch (const optix::Exception & e)
	{
		LOG(CRITICAL) << e.getErrorString();
		continue;
	}

	// make a new geometry instance
	GeometryInstance newInstance = context->createGeometryInstance();
	if (!newInstance) continue;
	newInstance->setGeometry(sourceGeometry);

	// reuse source material
	newInstance->setMaterialCount(1);
	optix::Material sourceMaterial = sourceInst->getMaterial(0);
	if (!sourceMaterial) continue;

	try
	{
		sourceMaterial->validate();
	}
	catch (const optix::Exception & e)
	{
		LOG(CRITICAL) << e.getErrorString();
		newInstance->destroy();
		continue;
	}
	newInstance->setMaterial(0, sourceMaterial);

	// create a new geometry group and add the new instance
	GeometryGroup group = context->createGeometryGroup();

	// reuse source acceleration structure
	group->setAcceleration(sourceGroup->getAcceleration());
	group->addChild(newInstance);

	// create a transform node for the source items's world transform
	Transform t = context->createTransform();
	t->setMatrix(true, sourceItem->spaceTime().worldTransform.matrix().data(), nullptr);
	t->setChild(group);

pool.push_back(t);

Did you verify that? E.g. by enabling exceptions and exception programs.

The code looks OK on first sight.

Is any of the continue states reached?
Mind that the C++ wrappers do not do any ref-counting on the underlying OptiX objects!
If you have destroyed, for example, a material node on any of the objects, it’s gone, even if you assigned it to hundreds of GeometryInstances.

I’m not using OptiX itself to track my scene graph, but a small shadow graph around the OptiX object.

Do you have any variables declared at GeometryInstance level? (There aren’t any set on the newInstance.)
For example I’m only using two Material nodes in my whole path tracer for opaque and cutout opacity materials and a single index variable at the GeometryInstance decides which material parameters from a global buffer to use.

You could try OptiX 5.0.1.

(I would recommend to name GeometryGroup variables geometryGroup for clarity because there are also Group nodes.)

Did you verify that? E.g. by enabling exceptions and exception programs.

Yes and I’ve stepped through the code to make sure it’s all ok. If any of the “continues” were hit then the instance wouldn’t be added but they’re all there … just black

Do you have any variables declared at GeometryInstance level? (There aren’t any set on the newInstance.)

Yes. There’s newInstance[“diffuse_color”]->setFloat(color) that wasn’t being set. Dang, I was so sure that was the problem but I’m still getting black instances after setting it.

You could try OptiX 5.0.1.
I’ve updated but it does not fix

I would recommend to name GeometryGroup variables geometryGroup for clarity because there are also Group nodes.

Good idea. Thanks for all the help!

Ok, then I don’t know. ;-)

Assuming this is on application side, one thing you could do locally would be to generate an OAC trace of a minimal case with maybe one additional instance and trace that once with the working shading and then with the one which doesn’t. https://devtalk.nvidia.com/default/topic/803116/?comment=4436953

Keep the interactions as identical as possible to get comparable API traces. Then look at or diff the OptiX API calls in the trace.oac text file done in the two cases and try to find what’s different in the failing case.

When setting the diffuse_color on the GeometryInstance level, make sure the same variable isn’t declared in the Material or Context scope as well. Otherwise the closest hit and any hit programs would pick it up from there due to the variable scoping order in OptiX.

Other than that, you could add some printf for a failing launch index into the device code to determine if the diffuse_color is actually black or if it’s something else.

Yes. There’s newInstance[“diffuse_color”]->setFloat(color) that wasn’t being set. Dang, I was so sure that was the problem but I’m still getting black instances after setting it.

I just fixed it! Setting “diffuse_color” for the instances was the problem after all. I’m not 100% sure what was wrong but the first time I tried, I iterated through the instance pool after it was created to set the “diffuse_color” for each instance. This time I set the “diffuse_color” while creating the instance and it works.

Thanks again for the help!

newInstance->setMaterial(0, sourceMaterial);

if (renderer == RendererType::Pathtrace)
{
	newInstance["diffuse_color"]->setFloat(sourceInst["diffuse_color"]->getFloat3());
}

// create a new geometry group and add the new instance
GeometryGroup group = context->createGeometryGroup();

// reuse source acceleration structure
group->setAcceleration(sourceGroup->getAcceleration());
group->addChild(newInstance);

// create a transform node for the source items's world transform
Transform t = context->createTransform();
t->setMatrix(true, sourceItem->spaceTime().worldTransform.matrix().data(), nullptr);
t->setChild(group);