Issues: Target argument blocks (mdl-sdk-296300.4444)

Gaming and Visualization Technologies Visualization MDL SDK
1

Table of contents

  1. Code: Wrong data offsets in structure traversal
  2. Code: Layout out of order in respect to material parameters
  3. Code: Textures in default expression result in multiple entries
  4. Docs: Difference between 'get_target_argument_block' and 'create_target_argument_block' is not sufficiently clear
  5. Docs: State offset never equals '~mi::State(0)' or '~0u' in traversal
  6. Docs: Confusing wording, "known" vs. "unknown" resource indices

EDIT: There are some more when using link units, that I’ll group with the link unit issues.

[s]1. Wrong data offsets in structure traversal

Using code to prints the argument block

// ...

called like

Handle< ITarget_argument_block > block_data( /* ... */ );
Handle< ITarget_value_layout > block_layout( /* ... */ );

dump_argument_block( block_layout.get(), block_data->get_data() );

we can observe that structure members always have a zero offset. It could be possible to work around this limitation by adding up the size of previous struct members IFF that size includes padding for alignment. Can you confirm whether this is the case?

Summing up the entires further reveals that we don’t end up at the reported total size of the argument block after traversal.[/s]

[s]2. Layout out of order in respect to material parameters

I could observe a material with ‘color, float3, color’ parameters (in that order) yield a traversal order of ‘color, color, float3’. Since there is no way to acquire the parameter name, this seems to be a show stopper for using target argument blocks right now.[/s]

[s]3. Textures in default expression result in multiple entries

Things get really wild when we have a default expression that contains a texture fetch. Then we get many entries for that texture. The obvious expectation would be that we just get an input of type ‘texture_return’, however, that might be too naive since we might want to let the integrating code know how to wire up that input by default somehow (but the default structurally differs from the input in that case)…[/s]

4. Difference between ‘get_target_argument_block’ and ‘create_target_argument_block’ is not sufficiently clear

In fact, I have yet to figure that one out myself. The current breakage doesn’t exactly make it easy to do that experimentally. What’s the justification for both variants’ existence? Looking at 3, I suspect we might have something conceptually half-baked here.

5. State offset never equals ‘~mi::State(0)’ or ‘~0u’ in traversal

The docs for ‘ITarget_value_layout::get_nested_state’ are lying:

The returned state’s ‘m_state_offs’, being a 32-bit unsigned integer, can never be equal to the 64-bit unsigned integer '~mi::Size(0)` with all bits set and no sign extension happening.

In practice, it also never happens to equal ‘~0u’ (a 32-bit integer with all bits set, which would be an obvious guess under “error correction by common sense”), which can easily be verified with a conditional breakpoint in line 38 of above code.

Just removing that sentence from the docs will fix it.

6. Confusing wording, “known” vs. “unknown” resource indices

Let’s see:

We have a “resource id” by which you identify the resource in your handler code. Then there’s an “index of the resource reference” (with separate index spaces for different types of resources) that describes the use of a resource in MDL target code. The former defaults to the latter, but can be changed in the target argument block if you use class compilation. A callback is provided to map resource paths to ids in this context.

With a wording like that, instead of talking about “known” vs “unknown” resource indices and what is known when, the whole story becomes rather obvious and approximately thousand times easier/faster to grok for the reader.

2
  1. Wrong data offsets in structure traversal
    Maybe this is a misunderstanding of the API. To get the offset of an element, one has to use get_layout() after choosing the according subelement with get_nested_state(). The m_data_offset field in the layout state is only for internal use.
    Example:
    MDL:
export struct sub_struct
{
    bool   a;
    bool   b;
    color  c;
    float  d;
};

export struct top_struct
{
    float a;
  double x;
    sub_struct b;
    int c;
    sub_struct d;
    float e;
};

export material test_mat(float p = 0.3,
    top_struct s = top_struct(
        0.1,
        0.2,
        sub_struct(false, true, color(0.3, 0.5, 0.7), 0.6),
        17,
        sub_struct(true, false, color(0.8, 0.3, 0.1), 0.3),
        1.3))
= let {
    float3 uvw = state::texture_coordinate(0);

    // Evaluate bools
    float a = s.b.a ? 0.5 : 0;
    float b = s.b.b ? 0.25 : 0;
    float c = s.d.a ? 0.125 : 0;
    float d = s.d.b ? 0.0625 : 0;
    float sum = a + b + c + d;

    // Evaluate numbers
    float e = s.a * 0.1 + s.c / 100.0 * 0.2 + s.b.d * 0.3 + s.x * 0.5;
    float f = s.e * 0.5 + s.c / 100.0 * 0.3 + s.d.d * 0.2 + s.x * 0.1;

    // Evaluate colors
    float3 col_b = float3(s.b.c);
    float3 col_d = float3(s.d.c);
    float g = col_b.x * 2 + col_b.y * 0.3 + col_b.z * 0.2;
    float h = col_d.x * 0.2 + col_d.y * 0.3 + col_d.z * 0.5;

    color params_res = color(
        sum + p * 0.7,
        math::lerp(e, f, uvw.x),
        math::lerp(g, h, uvw.y));

    color color_expr = params_res * math::sin(uvw.x * math::PI * 8);
    float float_expr = float3( color_expr).x;
} in material(
    surface: material_surface(scattering: df::diffuse_reflection_bsdf(
        tint: color_expr))
);

C++ code:

// find the material parameter "s"
mi::Size s_param_index, num_params = compiled_material->get_parameter_count();
for (s_param_index = 0; s_param_index < num_params; ++s_param_index) {
    if (!strcmp(compiled_material->get_parameter_name(s_param_index), "s"))
        break;
}
if (s_param_index >= num_params) return;

mi::base::Handle<const mi::neuraylib::ITarget_value_layout> layout(
    code_cuda_ptx->get_argument_block_layout());

// access the "top_struct s" parameter
mi::neuraylib::Target_value_layout_state state(layout->get_nested_state(s_param_index));
mi::neuraylib::IValue::Kind kind;
mi::Size size;
mi::Size offset = layout->get_layout(kind, size, state);
std::cout << "s:     offset = " << offset << ", size = " << size << std::endl;

// access the forth element of top_struct (int c)
mi::neuraylib::Target_value_layout_state state_x(layout->get_nested_state(3, state));
offset = layout->get_layout(kind, size, state_x);
std::cout << "s.c:   offset = " << offset << ", size = " << size << std::endl;

// access the third element of top_struct (sub_struct b)
mi::neuraylib::Target_value_layout_state state_b(layout->get_nested_state(2, state));
offset = layout->get_layout(kind, size, state_b);
std::cout << "s.b:    offset = " << offset << ", size = " << size << std::endl;

// access the third element of sub_struct (color c)
mi::neuraylib::Target_value_layout_state state_b_c(layout->get_nested_state(2, state_b));
offset = layout->get_layout(kind, size, state_b_c);
std::cout << "s.b.c:  offset = " << offset << ", size = " << size << std::endl;

// access the green element of color
mi::neuraylib::Target_value_layout_state state_b_c_g(layout->get_nested_state(1, state_b_c));
offset = layout->get_layout(kind, size, state_b_c_g);
std::cout << "s.b.c.g: offset = " << offset << ", size = " << size << std::endl;

// access the non-existing first element of the green element
mi::neuraylib::Target_value_layout_state state_b_c_g_0(layout->get_nested_state(0, state_b_c_g));
if (state_b_c_g_0.m_state_offs == ~mi::Uint32(0)) {
    std::cout << "No subelement available for s.b.c.g!" << std::endl;
}

Output:

s: offset = 0, size = 144
s.c: offset = 64, size = 4
s.b: offset = 16, size = 48
s.b.c: offset = 32, size = 12
No subelement available for s.b.c.g!

  1. Difference between ‘get_target_argument_block’ and ‘create_target_argument_block’ is not sufficiently clear
    For each translated class-compiled entity generated from a compiled material, a target argument block and a layout is created and stored inside the ITarget_code object. As the ITarget_code is const, so is the Target_argument_block object.
    To use a block with changed parameters, you can either
    • get the existing one via get_target_argument_block(), clone it and modify the content of the non-const clone with help of the layout information, or
    • create a new target argument block via create_target_argument_block() using a compiled material with different parameters provided to the same material.

So the get_target_argument_block() variant is more for as-fast-as-possible modification of parameters, while create_target_argument_block() variant is for people who want to only change the parameters in the database and rerun most of the code except the final target code generation (if the hash of the compiled material didn’t change by changing the parameters).

  1. State offset never equals ‘~mi::State(0)’ or ‘~0u’ in traversal
    you are right
    6 We have some internal discussion about this
3

Thanks, Jan.

Ah, ‘get_layout’ returns the offsets! I can confirm that those seem indeed correct (wondering why ‘m_data_offs’ is not a befriended private, though).

Sadly, this still applies (wish it was a misunderstanding too, but seems rather unlikely). Since you didn’t comment on it, and it’s the most important one, I’m making sure it doesn’t get lost (probably should have made it the first in the list).

I think I figured out the semantics here: It appears like all inputs of the default expression are captured. This is at least questionable: Let’s say we have a material with a parameter like this:

color tint = base::file_texture(texture_2d("my_texture_image.png")).tint

Wouldn’t we rather want a color pin that can be wired up to something other/more than an image in the shader graph, instead of all inputs of ‘base::file_texture’ and ‘texture_2d’ here? May make some sense if all shading graph was MDL-based, I guess…

OK, I get it. So it exposes the default argument block which is used when none is explicitly specified? To describe the use case: It effectively provides access to a single instance of the compiled material class that can be modified in-place. Typically that might be the case when using the native backend, right?

4

For the protocol, here is the corrected traversal routine:

char const* value_kind_str( mi::neuraylib::IValue::Kind value )
{
	using mi::neuraylib::IValue;

	switch ( value )
	{
		case IValue::VK_ARRAY: return "array";
		case IValue::VK_BOOL: return "bool";
		case IValue::VK_BSDF_MEASUREMENT: return "bsdf measurement";
		case IValue::VK_COLOR: return "color";
		case IValue::VK_DOUBLE: return "double";
		case IValue::VK_ENUM: return "enum";
		case IValue::VK_FLOAT: return "float";
		case IValue::VK_INT: return "int";
		case IValue::VK_INVALID_DF: return "invalid df";
		case IValue::VK_LIGHT_PROFILE: return "light profile";
		case IValue::VK_MATRIX: return "matrix";
		case IValue::VK_STRING: return "string";
		case IValue::VK_STRUCT: return "struct";
		case IValue::VK_TEXTURE: return "texture";
		case IValue::VK_VECTOR: return "vector";
		default: return "<unknown>";
	}
}

mi::Size dump_argument_block(
		mi::neuraylib::ITarget_value_layout const* meta,
		char const* data,
		mi::neuraylib::Target_value_layout_state state = mi::neuraylib::Target_value_layout_state(),
		std::string const& indent = std::string())
{ 
	using namespace std;
	using namespace mi::neuraylib;

	IValue::Kind kind;
	mi::Size size;
	mi::Size offset = meta->get_layout( kind, size, state );
	printf( "%s%s (%lld bytes @ offset %d) = ", 
			indent.c_str(), value_kind_str( kind ), size, offset );

	bool broken_offsets = false;
	switch ( kind )
	{
		case IValue::VK_DOUBLE:
			printf( "%lf\n", *(double const*) (data + offset) );
			return size;
		case IValue::VK_FLOAT:
			printf( "%f\n", *(float const*) (data + offset) );
			return size;
		case IValue::VK_BOOL:
		case IValue::VK_ENUM:
		case IValue::VK_INT:
		case IValue::VK_TEXTURE:
		case IValue::VK_LIGHT_PROFILE:
		case IValue::VK_BSDF_MEASUREMENT:
			printf( "%d\n", *(int const*) (data + offset) );
			return size;
		case IValue::VK_STRING:
			printf( "%s\n", *(char const* const*) (data + offset) );
			return size;
		case IValue::VK_STRUCT:
			broken_offsets = true;
		default: 
			break;
	}

	printf( "{\n" );
	std::string indent_next = indent + "\t";

	mi::Size n_elems = meta->get_num_elements( state );
	for ( int i = 0; i != n_elems; ++ i )
	{
		Target_value_layout_state state_next = meta->get_nested_state( i, state );

		mi::Uint32 data_offset_next = state_next.m_data_offs;

		dump_argument_block( meta, data, state_next, indent_next );
	}

	printf( "%s}\n", indent.c_str() );
	return size;
}
5
  1. Layout out of order in respect to material parameters

The parameters of the target argument block correspond to the parameters of the compiled material. So ICompiler_material::get_parameter_name() and ICompiled_material::get_parameter_count() can be used to identify the parameters.

Note, that the parameters of a compiled material do not in general correspond to the parameters of a material:
• Unused parameters won’t make it into the compiled material.
• The order is arbitrary.
• Function calls will become part of the material. Hence, if you instantiate a material parameter “int x” with a call to “math::max(1, 3)” (prototype “int math::max(int a, int b)”), there will be no parameter with name “x” but the two constants used in the call with be turned into the parameters “x.a” and “x.b”.

In our example, the second parameter of the material “s” actually became the first parameter of the compiled material.

  1. Textures in default expression result in multiple entries

First, this behavior does not depend on any default parameters. It depends on the actual arguments during the material instantiation.
As explained in point 2, function calls provided as material arguments become part of the compiled material.

So, if you modify the material in our example to add a third parameter like this:

export material test_mat(float p = 0.3,
top_struct s = top_struct(
0.1,
0.2,
sub_struct(false, true, color(0.3, 0.5, 0.7), 0.6),
17,
sub_struct(true, false, color(0.8, 0.3, 0.1), 0.3),
1.3),
color texcol = base::file_texture(texture_2d(“test_cube.png”)).tint)

and add texcol to color_expr (so it is used), you will get these parameter names:

  • s
  • p
  • texcol.s.texture
  • texcol.s.color_offset
  • texcol.s.color_scale
  • texcol.s.mono_source
  • texcol.s.uvw.position.index
  • texcol.s.uvw.tangent_u.index
  • texcol.s.uvw.tangent_v.index
  • texcol.s.crop_u
  • texcol.s.crop_v
  • texcol.s.wrap_u
  • texcol.s.wrap_v
  • texcol.s.clip

If you check the documentation of base::file_texture() in doc/base_module/index.html you will see, that each parameter of base::file_texture() became a parameter of the compiled material and each function call used as an argument to base::file_texture() (including default arguments like base::texture_coordinate_info()) was further split into its subparts. “texcol.s” comes from the automatically generated struct member accessor function “texture_return.tint(texture_return s)”.

Every argument of a compiled material parameter must be a constant. Hence, it does not allow further attachments of functions. Especially if you use a constant texture_return as an argument, you will get an input of type texture_return, but it will only accept constants of this type (no texture lookups).

If you replace the texcol parameter by

base::texture_return texcol = base::texture_return(color(0), 0))

and replace the usage by texcol.tint, you will get these parameter names:

  • s
  • p
  • texcol

Here, for texcol, only a constant tint and the mono value of the texture_return structure can be set.

6

Great! Thanks for clarifying. So we can cross another one off the list (and doh - could’ve figured that one out).

So you can change parameters directly through the argument block and in this case you are bounded by what can’t change the hashes…

With point 2 invalidated, that behavior is actually sound.

In essence, this means:

  • If you add function calls to the inputs, you do the wiring within MDL.
  • If you want to wire-up varyings from an external shader graph, use constants for the defaults.