In my OptiX7-based app (using MDL SDK 2019.2) I currently get this error: “[2][COMPILE FEEDBACK]: COMPILE ERROR: failed to create pipeline”
when I tried to use several MDL materials which have textures.
If some MDL materials (up to 4) are used, anything runs fine, but when more additional ones are added, the pipeline creation fails.
It seems only to affect MDL materials, which have textures. (The first 4 also have textures, but they succeed)
Pipeline creation is done before SBT and so even if there would be an error in my arg_block + Texture_handler code, that is not even executed when the pipeline is built. Of course all mdl callables creation succeeds. The texture handling is present in the .cu file.
Some MDL materials also had geometry: material_geometry(…)
eliminating some helped a bit, but not for all materials.
What could be wrong?
Thank you!
My current system: OptiX 7.0.0 SDK CUDA 10.1.243 GTX 1050 2GB Win10PRO 64bit (version 1809; build 17763.107) device driver: 442.50 VS2019 v16.4.5 (toolkit v140 of VS2015)
Sadly, OptiX 7 is still not very helpful regarding pipeline creation errors in official builds.
Try looking for possible linking errors, multiply defined symbols or missing symbols like non-existing symbols mentioned in “entryFunctionNameDC” fields, or missing semantic prefix in functions names (like “direct_callable”).
Does this also happen if you replace all textures in the MDL materials by colors to get rid of the texture runtime calls? Maybe check the generated PTX code for unknown functions.
It’s really weird, that this seems to only happen in a certain combination of materials, the single materials all work fine, right? Maybe the combination results in a duplicated symbol in one OptiX module?
If this doesn’t help you find the problem, you could send me your project (or a stripped down version of it) and I could compile and try it with an internal build of OptiX to get useful error messages. I’ll send you a PM, in case you want to go this route.
I first tried to replace all the textures with smaller texture files. Same problem.
Then I also removed all the ones using my custom cutout handling; Still same problem.
When using each material alone anything works fine.
I replaced all the textures in the MDL materials by colors; this works for all tested MDL materials.
All “entryFunctionNameDC” settings I use as you told me:
All MDL materials work together in one scene when I use the OptiX6.5-based version of my app (using the same MDL SDK 2019.2);
It also works on OptiX7 if I only use the first 4 MDL materials of them only.
Or if I use one of them only.
All of those materials are referencing textures.
for pipeline_link_options.maxTraceDepth I tried 3, 4 and 5 which should be more than enough; I found no difference with the pipeline creation problem on all of these.
There is still only one mdl_ibl.ptx file. This PTX code then is used for all MDL materials I build there.
The same PTX module is always used for “mat.module” in the code aboce.
Or do I miss something? I send the PTX and the MDL files as private message;
Honestly I don’t really know which of the functions in there are “unknown”.
Maybe you can find something on them.
obivously it seems to be something with the texture functions
Thank you. I hope we find some way with the texture functions first.
Actually I meant the PTX generated by the MDL SDK, which you can get from the ITarget_code object.
But with your set of materials, I was already able to reproduce the pipeline creation error and the internal error messages report multiply defined symbols.
Comparing your materials, one can see, that POTGROUND, WALL and PLANT are identical except for their parameters. As you probably use class-compilation, this will result in the exact same generated PTX code.
OptiX uses the function name + some hash over the code and maybe some options to generate the internal function names. So here, multiple modules then contained the same internal function name leading to the pipeline creation error.
So there are two ways to fix that:
You don’t need to generate new target code, when the material is the same except for the target argument block.
Create a map from the hash returned by mi::neuraylib::ICompiled_material::get_hash() to generated mi::neuraylib::ITarget_code and OptiX modules containing the generated code and check it before actually translating the code. You can create a new target argument block for a different compiled material (with a matching hash) using mi::neuraylib::ITarget_code::create_argument_block().
The sub-optimal but simple way would be to just use different function names instead of always the same “__direct_callable__mdlcode*”.
Moritz, thank you very much!
Now its working fine!
A question remains: How to use mi::neuraylib::ITarget_code::create_argument_block() properly?
I simply build the arg block in any case this way from the current compiled material:
arg_block = code_ptx->get_argument_block(0);
I use the same method also if the compiled material hash is re-used. But then I simply don’t build the module and the 4 DC’s. Instead I use a second map which contains the related material id (for a hash) for which the DC’s were build. From that (hash, material_id) pair I use the base DC offset. This works fine.
arg_block = reused_code->create_argument_block(0, // index
compiled_material.get(), // material The class-compiled MDL material
NULL // Callback for retrieving resource indices for resource values.
);
// "reused_code" is the target code from the map (found based on the hash)
This seems to be still valid for MDK SDK 2019.2, right ? So I would need to make another map with all textures; based on upcased texture file path; retrieved by target_code->get_texture(i); and then only store those, which are not identical;
EDIT: I scored this out. (See my next post after this one) Adapting the Texture_handler field “textures” with a custom list (per material) indexing into an array of texture pointers on the GPU, right?
The texture ids of a material is related (starting with 0 for the first) to that list right?
What about the underlying CUDA TextureObjects “cudaTextureObject_t”? Can that be safely removed on duplicates?
So technically I couild simply replace the “Texture” device pointer in the “state.mdl_textures” array with the duplicate and release the “Texture” object, which is the one overwritten by the replacement, right?
The reused_code approach looks fine, but you cannot provide a NULL resource callback to “create_argument_block”, you need to implement mi::neuraylib::ITarget_resource_callback.
It’s not just the texture file path, but also the shape (2d/3d/cube) and the gamma value.
The texture IDs are always per link unit. I don’t know, if your version already uses link units. You could add multiple materials to one link unit and then only have one list of texture IDs.
Or have a map which maps the texture ids per material to your own texture list.
But generally, yes. Not sure about the rest, I will read that again tomorrow ^_^
I wrote a small example implementation of the callback, expecting, that you can register new textures to the MDL helper and retrieve the resource index, which you would have to use with the texture runtime to get data for this texture.
/// Callback that notifies the application about new resources when generating an
/// argument block for an existing target code.
class Resource_callback
: public mi::base::Interface_implement<mi::neuraylib::ITarget_resource_callback>
{
public:
/// Constructor.
Resource_callback(
mi::neuraylib::ITransaction *transaction,
mi::neuraylib::ITarget_code const *target_code,
Mdl_helper *mdl_helper)
: m_transaction(mi::base::make_handle_dup(transaction))
, m_target_code(mi::base::make_handle_dup(target_code))
, m_mdl_helper(mdl_helper)
{
}
/// Destructor.
virtual ~Resource_callback() = default;
/// Returns a resource index for the given resource value usable by the target code
/// resource handler for the corresponding resource type.
///
/// \param resource the resource value
///
/// \returns a resource index or 0 if no resource index can be returned
mi::Uint32 get_resource_index(mi::neuraylib::IValue_resource const *resource) override
{
// handle resources already known by the target code
mi::Uint32 res_idx = m_target_code->get_known_resource_index(m_transaction.get(), resource);
if (res_idx != 0)
return res_idx;
// check whether we already prepared the resource
auto it = m_resource_cache.find(resource);
if (it != m_resource_cache.end())
return it->second;
mi::base::Handle<mi::neuraylib::IValue_texture const> val_texture(
resource->get_interface<mi::neuraylib::IValue_texture const>());
if (!val_texture)
return 0u; // not a texture, unknown resource
mi::base::Handle<const mi::neuraylib::IType_texture> texture_type(
val_texture->get_type());
mi::neuraylib::ITarget_code::Texture_shape shape =
mi::neuraylib::ITarget_code::Texture_shape(texture_type->get_shape());
mi::base::Handle<mi::neuraylib::ITexture const> texture(
m_transaction->access<mi::neuraylib::ITexture>(resource->get_value()));
// prepare texture and get index
res_idx = m_mdl_helper->prepare_texture(
m_transaction.get(),
texture.get(),
shape);
m_resource_cache[resource] = res_idx;
return res_idx;
}
/// Returns a string identifier for the given string value usable by the target code.
///
/// The value 0 is always the "not known string".
///
/// \param s the string value
mi::Uint32 get_string_index(mi::neuraylib::IValue_string const *s) override
{
char const *str_val = s->get_value();
if (str_val == nullptr)
return 0u;
for (mi::Size i = 0, n = m_target_code->get_string_constant_count(); i < n; ++i) {
if (strcmp(m_target_code->get_string_constant(i), str_val) == 0)
return mi::Uint32(i);
}
// string not known by code
return 0u;
}
private:
mi::base::Handle<mi::neuraylib::ITransaction> m_transaction;
mi::base::Handle<const mi::neuraylib::ITarget_code> m_target_code;
Mdl_helper *m_mdl_helper;
std::map<mi::neuraylib::IValue_resource const *, mi::Uint32> m_resource_cache;
};
Usage ("this" is the Mdl_helper):
mi::base::Handle<Resource_callback> res_callback(
new Resource_callback(transaction, code_ptx.get(), this));
mi::base::Handle<mi::neuraylib::ITarget_argument_block const> arg_block(
code_ptx->create_argument_block(0, compiled_material.get(), res_callback.get());
During going through all of the target_code->get_texture(i) entries in my app, each file name is checked for being a duplicate. I simply re-use the “Texture” object from that original texture again in the global mdl_textures vector instead of calling prepare_texture at all.
So I don’t need to build them twice (and so I also do not need to deallocate any “cudaTextureObject_t” objects, cause only each texture is build one time). I think I stick with what its in my implementation without using “create_argument_block”.
That makes some of my questions in my previous post unnecessary. (I scored them out)
I tried to use your code, but I have no overloaded function you call in there:
from here: MDL SDK 2019.2 (binary) : mdl-sdk-325000.1814.zip\mdl-sdk-325000.1814\examples\mdl_sdk\shared\example_cuda_shared.h
I looked into it, but its unclear to me what exactly to change to make it fit the overloaded header.
And I only need to build the texture if its acutally no duplicate. But that checking I do after “compile_mdl_material”.
In my code this “prepare_texture” version is executed only for non-duplicates.
Sorry, I was in a hurry yesterday and I thought I had mentioned adapting “prepare_texture” to make it usable for more cases.
Basically I just replaced the code_ptx and texture_index parameters by ITexture and Texture_shape.
The textures also have to be stored somewhere else then, though.