Question about MDL wrapper

As told in this post
i ask my issue again here in the MDL SDK forum:

I analyzed the MDL Expressions sample and I found some simpler MDL examples in the MDL SDK 296300.4444
Although I completely built the MDL Expressions sample from original OptiX SDK 5.0.0 installation it crashes with some of the MDL examples of the MDL SDK. First they load properly all the texture files and they prompt on error correctly if there’s a file missing. But on all cases where “df::fresnel_layer” is used and on many cases where “df::weighted_layer” is used, a crash occurs (but no error message). The ones I got working are: chrome.mdl, colored_wax.mdl, copper.mdl, aluminium_anodized.mdl and colored_flint_glass.mdl;
the copper.mdl sample technically works when replacing “fresnel_bsdf” with pure “diffuse_reflection_bsdf” or “simple_glossy_bsdf” all other samples in the MDL SDK crash on my system with the exception shown in the attachment. I always use “surface.scattering.tint” as expression path.

to reproduce this exception: in file “optixMDLExpressions.mdl” you simply can replace the material “M_checker” with this version:

export material M_checker() = let{
    bsdf fresnel_bsdf = df::fresnel_layer(
            ior: color(1.5, 1.5, 1.5),
            weight: 0.5,
            layer: df::simple_glossy_bsdf(tint: checker(50), roughness_u : 0.5),
            base: df::diffuse_reflection_bsdf(tint: checker(100)) 
} in 
    surface: material_surface(
    //   scattering: df::diffuse_reflection_bsdf(tint: checker(100))  // OK  (=ORG)
    //   scattering: df::simple_glossy_bsdf(tint: checker(50), roughness_u : 0.5) // OK
        scattering: fresnel_bsdf  // CRASH

System: OptiX 5.0.0 SDK CUDA 9.0 GTX 1050 Win10PRO 64bit (version 1607) device driver: 388.59 TDR re-enabled:with longer delays VS2017 v 15.5.6 (toolkit v140 of VS2015); mdl_wrapper.dll version: MDL SDK 2017.1, build 296300.2288, 27 Sep 2017, nt-x86-64-vc11

original post here:

Of course the sample should not crash and that needs fixing.

Could it be that there is a misunderstanding what that sample is doing?

The sample is not able to render arbitrary MDL materials. It shows how to use an MDL expression within Optix. In this case it uses the expression attached to the “tint” parameter of the surface bsdf. So you should be able to exchange the “checker” function with any other MDL function or expression and use that. The material is merely there to provide a container for the expression and requires a surface bsdf with a “tint” parameter

thank you for your answer.

yes, I’m aware, that the MDLExpressions sample only renders all as phong. In the sample I get .tint and .shading_normal from the expression. Maybe I understood it wrong, but in “MDL_spec_1.3.4_13Jan2017.pdf” under “13.8 Expression sharing in material definitions with let-expressions” there is an example with a let-expression which also contains a fresnel_layer. In my tests these let-expressions generally worked with the MDLExpressions sample of OptiX 5.0.0 if no fresnel_layer is used or if the fresnel bsdf is only defined in the let-expression, but nowhere used. The fresnel_layer returns type “bsdf”; the same type as “diffuse_reflection_bsdf” returns, so it should work, or did I miss something?
For example “chrome.mdl” (from the MDL SDK) is compiled without problems in the MDLExpressions sample; But when compiling original “asphalt.mdl” a crash occurs; both use a let-expression, but “asphalt.mdl” uses a fresnel_layer. If I use a diffuse_bsdf there (instead of fresnel_layer) the “asphalt.mdl” compiles without problem.

All these I tested and they work in the MDLExpressions sample: diffuse_reflection_bsdf, simple_glossy_bsdf, microfacet_beckmann_smith_bsdf, ward_geisler_moroder_bsdf, microfacet_ggx_smith_bsdf, microfacet_beckmann_vcavities_bsdf, microfacet_ggx_vcavities_bsdf, backscattering_glossy_reflection_bsdf
(I used a derivate of the “copper.mdl” where I replaced the fresnel_layer with one of the pure bsdf for scattering instead. using as mdl 1.3)

I searched a lot online for a documentation for the “expression path” of function “mdl_wrapper_compile_expression”, but I did not find any. in the MDLExpressions sample: “surface.scattering.tint” obviously for color and “geometry.normal” for normal vector is given. So is there a syntax documentation for this? Does the fresnel_layer need some additional handling there? is there a syntax for something like “geometry.reflection vector” and “geometry.refraction vector”?

You said I could change the “checker” function and that is clear. But I changed the “M_checker” material. In my tests it only failed on fresnel_layer and in some cases on weighted_layer, but when a surface bsdf is required, then that one should also be able to contain a fresnel_layer, shouldn’t it?

I read your paper “Sharing Physically Based Materials Between Renderers with MDL” and I successfully compiled the both plaster examples (pages 44+45) and I tried to use the “plastic” from page 46; but compiling also failed in my test: there the fresnel_layer is part of the scattering expression.

yes, the fresnel layering needs more complex handling.
surface.scattering contains the bsdf for the surface interaction. In MDL the BSDF can also be a graph of BSDF. Below the surface.scattering, the parameters depend on the actual bsdf used and are not fixed. In the fresnel case, surface.scattering will be fresnel_layer and the fresnel layer has the following parameters:
ior, weight, layer, base

accordingly you can access in the api the parameters of the surface bsdf, in this case:
surface.scattering.ior, surface.scattering.weight, surface.scattering.layer.

surface.scattering.ior and surface.scattering.weight would possibly be expressions to evaluate.
layer and base in turn are bsdf too, layer would be of type simple_glossy_bsdf

so keeping the path notion you would in this case have a
surface.scattering.layer.tint and a surface.scattering.base.tint that you can evaluate, but no surface.scattering.tint.

The documentation for the possible structures under surface.scattering really is the MDL spec shipped with the SDK. To correctly handle this you need to query the actual type of the bsdf and react accordingly.

ok, thank you very much for that information. I very appreciate that.

I built the mdl_wrapper.cpp + mdl_helper.cpp from source code provided in the OptiX 5.0.0 sample. I used SDK MDL 296300.4444 include folder “mi”; and so eliminated mdl_wrapper.dll ( MDL SDK v 296300.2288) shipped with OptiX.
I used libmdl_sdk.dll and nv_freimage.dll from the newer MDL SDK, but unfortunately dds.dll (using image v13) from OptiX 5.0.0 is not compatible with the newer MDL SDK (using image v14) and there is no DDS.dll in the MDL SDK. console Output: error: Image plugin of name “dds” from library “dds.dll” has unsupported plugin type “image v13”. Please use a version of the plugin that has been compiled for the currently supported plugin type “image v14”. I found in mdl_helper.cpp : “Consider the dds plugin as optional” So I removed all environment materials in optixMDLExpressions.mdl which use .dds files and call the “addMDLEnvironment” function only with “perez_sun_and_sky()”. On running the MDLexpressions sample (original mdl expressions without any fresnel_layers) now no output is present. However all mdl compilings succeed without error or exception, but window remains black while fps output is running. Is there some setting which changed from 296300.2288 to 296300.4444 ?

I then was able to find the exception reason for fresnel_layer issue when trying to use it again falsly with surface.scattering.tint: the call in mdl_helper.cpp in Mdl_helper::Mdl_compile_result::compile_expression() to function “translate_material_expression()” returns NULL (failure) error code: -2: Invalid path (non-existing). but the return value is then used in Mdl_helper::create_program. So Mdl_helper::create_program would need to check for NULL PTR or for valid result. So the exception occurs only in the MDL wrapper, not in the MDL SDK itself.

An additional call to “translate_material_expression()” with “.layer.” inbetween would then succeed.
Or I could query the expression through ICompiled_material::lookup_sub_expression() in icompiled_material.h, right? It returns NULL if failed, then I would query again until found.

But changing mdl_helper.cpp or querying for the bsdf type would require to successfully use the source code of mdl_wrapper.cpp + mdl_helper.cpp and the MDL SDK instead of the pre-compiled mdl_wrapper.dll of OptiX 5.0.0
Please tell me, where I can download MDL SDK version 296300.2288 (I did not find any page; only the newer one)
Or what to do to use the MDLExpressions sample with the MDL SDK 296300.4444 ?

I re-installed CUDA 9.0 to avoid any dependency problems on that side, but still the same problem as described above (no output when building mdl_helper.cpp + mdl_wrapper.cpp using MDL SDK 2963000.4444 for use in the OptiX 5.0.0 MDLExpressions sample). The problem can be simply reproduced by installing OptiX 5.0.0 SDK and removing DDS.dll + handling for loading all environment maps in (.mdl and .cpp files) which use them; and including “mi” from MDL SDK. The original MDLExpression scene loads, but the window remains black.
It works fine with the mdl_wrapper.dll shipped with OptiX 5.0.0.

When I want to use the MDL material in a Path Tracer I would like to have a reflective/refractive direction from the material (based on the BSDF). But “geometry.displacement” as expression path, only sets a displacement, but it does not return a reflective/refractive direction, right?
How to obtain the reflective/refractive direction from the MDL material?

carpet_measured.mdl and concrete_measured.mdl use a .mbsdf and “surface.scattering.layer.tint” and "“surface.scattering.tint” fails on them.
Please tell me, how to build an expression path for them?

But all the other .mdl examples from the MDL SDK generally successfully compile now (when “.layer” is used accordingly) with the mdl_wrapper.dll of version 296300.2288

The SDK doe not give you that, you will have to implement matching BSDF in your pathtracer and evaluate them. Saying that, the upcoming MDL SDK 2018 (in the process of beeing uploaded to the Designworks repository, i will post in the forum once its up) will have a sample implementation of the BSDF and a more complex sample that illustrates how to use it in Optix/CUDA. (it will also come with dds.dll) But for example for emission or sss, you will still need to implement the relevant DF yourself!

hmm… yes and no. Tint is just one of many things that could have an expression attached and the tree of BSDF could be deep with many BSDF that actually have tint parameters. The key here is that your renderer must know and implement all the possible bsdf specified in the MDL spec.

Then, you need to traverse the tree of bsdf, i.e. get surface.scattering, check its semantics (doc/api/html/classmi_1_1neuraylib_1_1IFunction__definition.html ->semantics ) get all the parameters , configure your bsdf implementation according to the value of the parameters, if the parameter is a bsdf-> traverse down and do the same.

Thank you very much.
So I will wait for the MDL SDK 2018, so that I know how to use it with OptiX.

There is a example_df_cuda in MDL SDK 2018,but Using the opengl rendering pipeline
please how to us ray tracing in optix?

An optix sample with the same functionality as the CUDA sample is delivered as part of optix 5.1

Thank you for your reply.

May I ask when the optix 5.1 version is about to be released?

“beginning of may”. It should come soon.

I also recommend to wait for the OptiX 5.1 release containing the new “optixMDLSphere” example which corresponds to the “example_df_cuda” MDL SDK 2018 example.
But to get you started, here are some hints what you would have to do.

To use the new generated BSDF code with OptiX 5.0.1, you first have to generate the programs for the BSDF init, sample, evaluate and PDF functions (see for some details).
Instead of calling IMdl_backend::translate_material_expression() (as done in Mdl_helper::Mdl_compile_result::compile_expression()),
you call IMdl_backend::translate_material_df() with the compiled material, the path to the BSDF you want to compile and a prefix for the function names of the resulting functions (which doesn’t really matter for OptiX).
The resulting ITarget_code object will contain the PTX code of the four functions in the above order, accessible via ITarget_code::get_callable_function().

For each function, an optix::Program object has to be generated and the texture functions have to be assigned to them as previously done in Mdl_helper::create_program().
The texture samplers only have to be generated once, of course.

Note, that a new texture access function “tex_resolution_2d” is required and has to be added to the tex_prog_names array in mdl_helper.cpp.
Here is an example implementation to be put in

// Implementation of resolution_2d function needed by generated code.
// Note: uvtile textures are not supported in this example implementation
RT_CALLABLE_PROGRAM void tex_resolution_2d(
    int                    result[2],
    Core_tex_handler const *self,
    unsigned               texture_idx,
    int const              /*uv_tile*/[2])
    if ( texture_idx == 0 ) {
        // invalid texture returns zero
        result[0] = 0;
        result[1] = 0;

    uint3 size = rtTexSize(texture_sampler_ids[texture_idx - 1]);

    result[0] = size.x;
    result[1] = size.y;

The new OptiX 5.1 version of Mdl_helper contains a compile_df() function which does all this for you.

In the CUDA code for the OptiX 5.1 example optixMDLSphere, the usage is basically identically to the CUDA code of MDL SDK example example_df_cuda, so please refer to the MDL SDK example on how to use the functions.

Instead of the function pointer arrays for the BSDF functions, you use callable program variables:

// MDL BSDF programs
rtDeclareVariable( rtCallableProgramId<mi::neuraylib::Bsdf_init_function>,     mdl_bsdf_init, , );
rtDeclareVariable( rtCallableProgramId<mi::neuraylib::Bsdf_sample_function>,   mdl_bsdf_sample, , );
rtDeclareVariable( rtCallableProgramId<mi::neuraylib::Bsdf_evaluate_function>, mdl_bsdf_evaluate, , );
rtDeclareVariable( rtCallableProgramId<mi::neuraylib::Bsdf_pdf_function>,      mdl_bsdf_pdf, , );

You have to assign the IDs of the generated OptiX programs to those variables in your host code.

As in the optixMDLExpressions example, we use an “empty” mi::neuraylib::Resource_data object, as the texture functions and the samplers were already assigned to the callable programs.

To get access to the mi::neuraylib types, you should include mi/neuraylib/target_code_types.h in the CUDA code.
You will have to adapt the CMakeLists.txt file to get access to the include file, by adding the mdl-sdk folder in some places:

# NVRTC absolute include paths to the headers used to build the samples
  \"${OptiX_INCLUDE}\", \\
  \"${OptiX_INCLUDE}/optixu\", \\
  \"${CMAKE_CURRENT_SOURCE_DIR}/support/mdl-sdk/include\", \\
  \"${CUDA_INCLUDE_DIRS}\", ")


# Path to sutil.h that all the samples need
include_directories( ${CMAKE_CURRENT_SOURCE_DIR}/sutil
                     ${CUDA_INCLUDE_DIRS} )


if(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/support/mdl-sdk/include/mi/mdl_sdk.h)
  # The MDL_SDK wrapper library can only be built when the MDL SDK is available

The FreeImage plugin of MDL SDK 2018 is not compatible with the one shipped with OptiX 5.0.1, so you need to copy the one from the MDL SDK to nvmdl_freeimage.dll/ next to the libmdl_sdk binary in your output folder and use this filename with IMdl_compiler::load_plugin_library() in the Mdl_helper constructor.

Thank you for this great new MDL SDK 2018 !
I built the “example_df_cuda” as VS2017 project (had to do it from an empty project, cause it was not available in the solution created; but it works now) and I tested all the MDL SDK .mdl examples with it and added a traversal (using mdl_compiler->load_module and Compiled_material_traverser_print from MDL SDK “example_traversal”). Directly before the Material_compiler “mc” is created. mdl_compiler->load_module is called then. I hope its not loaded twice, is it?

in “visit_begin” I then check for the semantics. So all unupported semantics (“measured” BSDFs, EDFs) are found now and processing quits. DS_INTRINSIC_DF_DIFFUSE_EDF causes no exception, so it seems to be implemented, right?
I tried all microfacet ggx, microfacet beckmann, ward and all run great! Also the modifiers “thin_film”, “tint”, “directional_factor” and “measured_curve_factor” and the mixers “normalized_mix” and “clamped_mix” work fine
Generally all the layers “weighted_layer” and “fresnel_layer” also work fine and also their semantics are now detected fine.
The sample with a fresnel and weighted layer “::carpaint_orange_peel::carpaint_orange_peel_2_layer” loads successfully and the sphere has a great material. So layers seem to work ok.

My test app fails yet only in a very complex case, when a fresnel layer uses as layer a “normalized_mix” and if its base is also a “normalized_mix” or if one of them is a “weighted_layer” which also contains a “normalized_mix”.
How to detect such complex cases? (I could count all found semantics but is there a better way?)

However, I will wait for OptiX 5.1 to use it directly in my main app. For testing I use “example_df_cuda” for now.

You are right, IMdl_compiler::load_module() uses a module cache, so it will not be loaded twice.

EDFs as well as measured BSDFs are not implemented, yet, so DS_INTRINSIC_DF_DIFFUSE_EDF also isn’t. It just doesn’t crash…
In the next release, a proper error code will be returned for unsupported DFs and two missing MDL 1.4 BSDFs (color_custom_curve_layer and color_measured_curve_layer) will be added.

Can you please elaborate a bit more about the failure?
What are you trying to detect and why?
The code generation should work for all combinations of (supported) BSDFs.

I tried to construct a material from your description, but it works fine with example_df_cuda:

mdl 1.4;

import df::*;
import math::*;
import state::*;

export color checker(float factor) {
    float3 pos = state::position();
    float3 f = math::abs(pos) * factor;
    int t = (int(f.x) ^ int(f.y) ^ int(f.z)) & 0xFF;
    return color(t / 255.0);

export material test_layers() = material(
    surface: material_surface(
        scattering: df::fresnel_layer(
            ior: 1.5,
            base: df::normalized_mix(
                        weight: 0.3,
                        component: df::diffuse_reflection_bsdf(tint: checker(100))),
                        weight: 0.9,
                        component: df::simple_glossy_bsdf(
                            roughness_u: 0.2, roughness_v: 0.4, tint: color(1, 0.3, 0.3)))
            layer: df::normalized_mix(
                        weight: 0.8,
                        component: df::diffuse_reflection_bsdf(tint: checker(20))),
                        weight: 0.4,
                        component: df::simple_glossy_bsdf(
                            roughness_u: 0.6, roughness_v: 0.2, tint: color(0.3, 1, 0.3)))

Can you please post your test case, if the code generation fails for it?

sorry, that I’m answering so late; but I did not get a notify email for this post, although I follow.

I checked out the code you provided. That also works fine and is correctly executed.

I want to detect any unsupported (not implemented) parts, so that I either can set my own implemenation to them (if I have one) or that the entire MDL material file is rejected without crashing, when it demands any (yet) unsupported features. In my main app I use an own 3D object format, which should optionally use MDL materials. It should fallback to an other (default) material handling (or simply skip) when something goes wrong or if unsupported features are demanded.

Here the code which crashes:

// modified; parts taken from a MDL SDK sample
mdl 1.4;
import df::*;
import state::*;
import base::*;
import tex::*;

export material test28(
    color base_color = color(0.805, 0.395, 0.305),
    float roughness = 0.1
) = let{

    bsdf microfacet_ggx_smith = df::microfacet_ggx_smith_bsdf(
        mode: df::scatter_reflect,
        tint: base_color,
        roughness_u: roughness,        
        roughness_v: roughness        
    bsdf microfacet_beckmann_smith = df::microfacet_beckmann_smith_bsdf(
        mode: df::scatter_reflect,
        tint: base_color,
        roughness_u: roughness,        
        roughness_v: roughness        

   df::bsdf_component[2]   bsdf_com  = 
                                                    df::bsdf_component ( 0.4, microfacet_ggx_smith), 
                                                    df::bsdf_component ( 0.6, microfacet_beckmann_smith)

   bsdf mix1 =  df::normalized_mix(
     components : bsdf_com

df::bsdf_component[2]   bsdf_com2  = 
                                                    df::bsdf_component ( 0.4, microfacet_ggx_smith),                // crash
                                                    df::bsdf_component ( 0.6, microfacet_beckmann_smith)      // crash
                                                    //df::bsdf_component ( 0.3, microfacet_ggx_smith),                  // OK, no  crash
                                                    //df::bsdf_component ( 0.7, microfacet_beckmann_smith)

bsdf mix2 =  df::normalized_mix(
     components : bsdf_com2

   bsdf mix3 =  df::clamped_mix(    // same crash if this is used instead of mix2  (and if same weights)
     components : bsdf_com2

bsdf fresnel1 = df::fresnel_layer(
       ior : 1.6,
       weight : 0.9,
       layer : mix1, 
       base : mix2 // mix3

} in material(
    surface: material_surface(
      scattering:  fresnel1

in the attachment is a screenshot of the exception message. No error message was thrown.
the “semantics=” log output is generated in Compiled_material_traverser_print::visit_begin at case mi::neuraylib::IExpression::EK_DIRECT_CALL
the “2 semantics=” log output is generated in Compiled_material_traverser_base::traverse also at case mi::neuraylib::IExpression::EK_DIRECT_CALL

I tried this out also with the first original build of df_CUDA.exe which also crashes.

It does NOT crash if bsdf_com and bsdf_com2 use different weights.

But it also crashs if ONE of the weights is identical in bsdf_com2 to the ones in bsdf_com (on same BSDF):

df::bsdf_component[2]   bsdf_com2  = 
                                                    df::bsdf_component ( 0.4, microfacet_ggx_smith),                // crash   0.4 same weight
                                                    df::bsdf_component ( 0.3, microfacet_beckmann_smith)      // crash

// OR:
                                                   // df::bsdf_component ( 0.5, microfacet_ggx_smith),                // crash
                                                   // df::bsdf_component ( 0.6, microfacet_beckmann_smith)      // crash  0.6 same weight

I updated the source code and message text in the previous post. the text in the notify email of the initial post did not point out the difference between using same or different weights in the bsdf_component
When using different weights, no crash occurs.

Thanks a lot for the test case! The bug has actually already been fixed in the current development version. The fix will be included in the next release.

The problem appears, when non-BSDF values containing BSDFs are used multiple times (in your example mix1.components and mix2.components are the same and thus mapped to the same expression).
In this case, the generator tries to write code for calculating these values and storing them in the provided or local texture result buffers. For values containing BSDFs this obviously doesn’t work.
In the next version, such values will be skipped.

Btw, OptiX 5.1 has been released containing the optixMDLSphere example corresponding to the example_df_cuda example.

Thanks for the information about the optixMDLSphere (in OptiX 5.1).

in the file metal_examples.mdl (MDL SDK 2018) in “metal_brushed” the same texture (brushed_full_contrast.jpg) is loaded for roughness_texture_u (line 576) and again for roughness_texture_v (line 581).
Is there an internal pooling in MDL, which detects such duplicates? I did not find anything about that in the documentation.

And in “metal_generic” in line 173 the weighted_layer there has no “base” specified. After some trys I found, that df::diffuse_reflection_bsdf(tint: color(0)) seems to be the default BSDF there. Is that true?
One would assume that df::diffuse_reflection_bsdf() would be the default there. The documentation says “bsdf base = bsdf(),” on the definition of a weighted_layer. In the release notes I did not find anything about it.

You said, that all the edf’s are yet not supported in this use case. What about the vdf’s? In colored_wax.mdl a df::anisotropic_vdf() is present and the file runs when use in df_CUDA.

bsdf() is essentially a black bsdf not scattering at all. The reasoning here is that the defaults should never add unwanted scattering, if you want to have scattering you need to specify it explicitely. A construct like in metal allows you to add a dedicated normal map to the “layer” and/or easily have separate color/weight controls for a bsdf.

The sample has no volume handling yet, after all for volumes the df is simple, its the intergrator that’s involved.