Issues: Link units (mdl-sdk-296300.4444)

Gaming and Visualization Technologies Visualization MDL SDK
1

Since you are apparently already aware of some or all of them, I’ll just provide a list for synchronization - details on request.

  1. Code translated from a link unit is randomly missing code for added entities
  2. The LLVM backend refuses to generate bitcode from a link unit and always generates IR
  3. Trying to create a target argument block for code generated from a link unit results in a crash
  4. Missing API for enumerating and specifying the source material of function when creating a target argument block
  5. There is no way to set the compiled-in uniform state (SDK docs indicate that compiling in that state might be temporary solution to begin with)
2

Thanks for reporting,

Code translated from a link unit is randomly missing code for added entities
Fixed in the next release

Trying to create a target argument block for code generated from a link unit results in a crash
For link units, no target argument blocks / layouts were generated. Fixed in the next release

The LLVM backend always translates link units to IR, even when bitcode is requested.
Fixed in the next release

Missing API for enumerating and specifying the source material of function when creating a target argument block
Maybe this is a misunderstanding of the API. To get the offset of an element, one has to use get_layout() after choosing the according subelement with get_nested_state(). The m_data_offset field in the layout state is only for internal use.
Example:
MDL:

export struct sub_struct
{
    bool   a;
    bool   b;
    color  c;
    float  d;
};

export struct top_struct
{
    float a;
  double x;
    sub_struct b;
    int c;
    sub_struct d;
    float e;
};

export material test_mat(float p = 0.3,
    top_struct s = top_struct(
        0.1,
        0.2,
        sub_struct(false, true, color(0.3, 0.5, 0.7), 0.6),
        17,
        sub_struct(true, false, color(0.8, 0.3, 0.1), 0.3),
        1.3))
= let {
    float3 uvw = state::texture_coordinate(0);

    // Evaluate bools
    float a = s.b.a ? 0.5 : 0;
    float b = s.b.b ? 0.25 : 0;
    float c = s.d.a ? 0.125 : 0;
    float d = s.d.b ? 0.0625 : 0;
    float sum = a + b + c + d;

    // Evaluate numbers
    float e = s.a * 0.1 + s.c / 100.0 * 0.2 + s.b.d * 0.3 + s.x * 0.5;
    float f = s.e * 0.5 + s.c / 100.0 * 0.3 + s.d.d * 0.2 + s.x * 0.1;

    // Evaluate colors
    float3 col_b = float3(s.b.c);
    float3 col_d = float3(s.d.c);
    float g = col_b.x * 2 + col_b.y * 0.3 + col_b.z * 0.2;
    float h = col_d.x * 0.2 + col_d.y * 0.3 + col_d.z * 0.5;

    color params_res = color(
        sum + p * 0.7,
        math::lerp(e, f, uvw.x),
        math::lerp(g, h, uvw.y));

    color color_expr = params_res * math::sin(uvw.x * math::PI * 8);
    float float_expr = float3( color_expr).x;
} in material(
    surface: material_surface(scattering: df::diffuse_reflection_bsdf(
        tint: color_expr))
);

C++ code:

// find the material parameter "s"
mi::Size s_param_index, num_params = compiled_material->get_parameter_count();
for (s_param_index = 0; s_param_index < num_params; ++s_param_index) {
    if (!strcmp(compiled_material->get_parameter_name(s_param_index), "s"))
        break;
}
if (s_param_index >= num_params) return;

mi::base::Handle<const mi::neuraylib::ITarget_value_layout> layout(
    code_cuda_ptx->get_argument_block_layout());

// access the "top_struct s" parameter
mi::neuraylib::Target_value_layout_state state(layout->get_nested_state(s_param_index));
mi::neuraylib::IValue::Kind kind;
mi::Size size;
mi::Size offset = layout->get_layout(kind, size, state);
std::cout << "s:     offset = " << offset << ", size = " << size << std::endl;

// access the forth element of top_struct (int c)
mi::neuraylib::Target_value_layout_state state_x(layout->get_nested_state(3, state));
offset = layout->get_layout(kind, size, state_x);
std::cout << "s.c:   offset = " << offset << ", size = " << size << std::endl;

// access the third element of top_struct (sub_struct b)
mi::neuraylib::Target_value_layout_state state_b(layout->get_nested_state(2, state));
offset = layout->get_layout(kind, size, state_b);
std::cout << "s.b:    offset = " << offset << ", size = " << size << std::endl;

// access the third element of sub_struct (color c)
mi::neuraylib::Target_value_layout_state state_b_c(layout->get_nested_state(2, state_b));
offset = layout->get_layout(kind, size, state_b_c);
std::cout << "s.b.c:  offset = " << offset << ", size = " << size << std::endl;

// access the green element of color
mi::neuraylib::Target_value_layout_state state_b_c_g(layout->get_nested_state(1, state_b_c));
offset = layout->get_layout(kind, size, state_b_c_g);
std::cout << "s.b.c.g: offset = " << offset << ", size = " << size << std::endl;

// access the non-existing first element of the green element
mi::neuraylib::Target_value_layout_state state_b_c_g_0(layout->get_nested_state(0, state_b_c_g));
if (state_b_c_g_0.m_state_offs == ~mi::Uint32(0)) {
    std::cout << "No subelement available for s.b.c.g!" << std::endl;
}

Output:

s: offset = 0, size = 144
s.c: offset = 64, size = 4
s.b: offset = 16, size = 48
s.b.c: offset = 32, size = 12
No subelement available for s.b.c.g!

3

I don’t think it is, here. I might have been unclear:

The API allows you to add expressions from different materials plus extra functions to a single link unit. If your final target (after linking with external code) is some kind of “uber shader” (containing all materials) and you want to use all processor threads for compiling it, using one link unit per thread seems a reasonable recipe to keep the memory use and final link time down (see discussion about multi-threading). In practice, if you want to support preview renders , you may probably want to be less radical and go for groups of 16 materials or so, to allow for caching / fast recompilation.

It doesn’t seem to make sense to lump together all lose parameters of all materials in the link unit - you wouldn’t be able to effectively manage individual instances of the material classes anymore.

Another way is to promote one link unit per material, but in that case you’d a) have to properly document that restriction somewhere (or make the API more restrictive), and b) provide better means to avoid duplication of dependent data (it appears you can compile to a data segment, but are those data blocks shared across multiple link-units? From what I read so far, most probably not).