Distillation of MDL over to a generic PBR shading model

Gaming and Visualization Technologies Visualization MDL SDK
1

I have been procrastinating to pen this posting for months as I had otherwise wanted to make it much more technical and specific. Rather, let me generalize my questions which allude to the bigger questions on my mind.

I am familiar with the MDL SDK, its programming, all of its samples and in the usage of the baking + distillation code to create PNG textures on disk + PBR-centric material parameters. No main questions in that regard.

  1. Let me pose my basic question: does anyone, say internally to the MDL development team, have a “better” example of their own about how to distill down any generic MDL material to a PBR-centric uber-shader? I know that this question could be interpreted in many different ways and hence please excuse the generality of my request. The example code is pretty good but, to me, it has gaping holes in it in terms of what I subconsciously wish it to do. Without getting into absolutes and specifics, it would be nice to see additional examples for AO mappings, geometry displacement (which I’ve been dabbling with lately), clearcoat tint color, how to map environment maps, etc. Ideally I would prefer to give you real world examples of what I feel is missing but, again, I am going to generalize in this porting.

1b) As a side note, there is an excellent glTF PBR material → MDL mapping in the example code, which of course maps over to a custom glTF-centric MDL material.

  1. Here’s an internal issue on my mind right now which I could benefit from some hand holding or basic education. Again, let me generalize. “Given any MDL material, for which I could read its source code and learn how it works, what are the basic rules to translate that shader information over to the queries needed to extend the distillation rules for such shaders?”. As an example I was looking at 15mins ago, a DomeSky shader makes reference to ‘Emission’ + associated texture map from which I’d like to make a generalized all-encompassing distillation rule to map over those to a PBR-centric environment map. I do understand that the distillation process is approximate, at best.

  2. And let me ask this simple question while I am making this posting. I’m sure I could find this info myself. Are the Normal Maps generated by the distiller in DirectX or OpenGL format for its RGB channels?

  3. And let me pose this question just while I am doing a brain dump. Is there a good and definitive example for querying, displaying/editing and sending back MDL parameters to the SDK? I had queued up the following references which touch on this subject. My hesitation in using or cloning this code is that it would be difficult to maintain in the future if and when NVIDIA decides to revise the system, as been happening over the last few years with the evolution of MDL and its related SDK. VRED has recently added in such parameter querying + editing so I was wondering what example code other vendors had used as the basis of their implementations?

traversal\compiled_material_traverser_print.cpp
shared\gui\gui_material_properties.cpp
shared\compiled_material_traverser_base.cpp
df_cuda\example_df_cuda.cpp
mdl-sdk\examples\mdl_sdk\optix7\optix7_mdl.cpp

2

First we need to distinguish between “distilling” (to a simplified target material) and “baking” (of material parameters to a texture)

Re 1:

AO, displacement, env maps are all pretty specific to your implementation. Providing examples for all possible kind of renderers is out of scope for what we can do with our examples.
What we can do is help with specific problems. For example displacement: what you get from baking in the SDK is basically a tangent space vector with a certain length, If you want a simple height map, there is no 1:1 transformation, you could approximate the result by using the z component of that vector.

Clearcoat tint should actually be trivial. (with the release of the 2023 version of the SDK, the rules used for distilling are also public https://github.com/NVIDIA/MDL-SDK/blob/master/src/shaders/plugin/mdl_distiller/dist_rules_ue.mdltl)

Distilling is an MDL-> MDL translation.so the result of “distilling to UE4” is an MDL material, not a UE4 material and has more parameters then a UE4 material , the example code then just extracts the parameters needed for UE4.
But the distiller leaves other parameters untouched, the topmost ggx lobe does have a tint that you should just be able to also bake to a texture.

1b: an example for distilling to GLTF is in the works.

Re2.
We will need to write more documentation on how to write distilling rules and its not something i can cover in a short forum post

In general the point of distilling is to transform a material with an unknown structure to something that has a known structure that we can easily read a parameter from.

So for clearcoat (if the distilling found a component that works as clearcoat) after UE4 distilling, the result MDL material will have the following partial structure:

in material(
	surface: material_surface(
		scattering: df::custom_curve_layer( 
                         weight: <somevalue>,
                         layer: df::microfacet_ggx_smith_bsdf(
                                  tint: <somecolor>,
...
               )
	)
);

and that that corresponds to the path used in example_distilling_glsl.cpp:

        std::string path_prefix = "surface.scattering.";

        // Check for a clearcoat layer, first. If present, it is the outermost layer
        if (semantic == mi::neuraylib::IFunction_definition::DS_INTRINSIC_DF_CUSTOM_CURVE_LAYER)
        {
            // Add clearcoat expression paths
            add_material_expression("clearcoat_weight", path_prefix + "weight");
            add_material_expression("clearcoat_color", path_prefix + "layer.tint");

the rules in https://github.com/NVIDIA/MDL-SDK/blob/master/src/shaders/plugin/mdl_distiller/dist_rules_ue.mdltl from line 710 are the sauce that makes this work. they will make sure you can rely on that path to contain the clearcoat

now ue 4 distiller does not touch emission, you would need to write custom distilling rules that transform an arbitrary edf tree into something well defined for inspection.

re 3: the image origin is “lower left” like GLSL and thats what determines the green channel color. so “opengl” style normal maps are more straightforward exports

re4:
basically the samples are the definitive resource :-D its the same sample code other vendors use as the basis for their implementation.
cheers
Jan

3

Hello Jan

I just wanted to drop a quick response to acknowledge your detailed response and to thank you for taking so much time to consider my questions + respond to them in such detail.

It will take a bit of time for me to digest your suggestions and to increase my understanding of the back-end distillation process. This was a good “boost” to help me re-focus my efforts.