Announcement: MDL SDK 2024.0.2 released

Gaming and Visualization Technologies Visualization MDL SDK
1

Get the update on https://developer.nvidia.com/mdl-sdk and https://github.com/NVIDIA/MDL-SDK.

MDL SDK 2024.0.2 (377400.2626): 07 Aug 2024

Added and Changed Features

  • General
    • MDL Arnold: Added support for Arnold SDK version 7.3.2.1.

Fixed Bugs

  • General

    • Fixed missing leading “::” for qualified names reported by IMdl_discovery_api,
      if search paths had a trailing OS separator.
    • Improved error message that can be triggered when calling non-const methods
      on const objects. This can e.g. be triggered via the Python binding by accessing
      (not editing) database elements.
    • Improved coverage tests for the Python binding.

  • MDL Compiler and Backends

    • Fixed crash in some cases when derivatives were calculated for a struct also containing
      non-floating point fields.
    • Fixed wrong digit that can be added to tex::texture_isvalid() calls for
      HLSL and GLSL backends, if more than one texture type (for instance texture_2d and
      texture_3d) are used together in one module.

MDL SDK 2024.0.1 (377400.2109): 18 Jul 2024

Added and Changed Features

  • MDL 1.9 Language Specification

    • Updated version to 1.9.
    • declarative and struct_category as new reserved words.
    • Added the concept of declarative structure definitions with
      and without a structure category to the conventional
      structure definitions.
    • Added the definition for the assignment operator (=) for
      declarative structs with and without a structure category.
    • Restricted the type cast operator on structure types to
      conventional structure types.
    • Added declarative functions.
    • Restricted overload resolution on material definitions.
    • Extended overload resolution to define functions whose signature
      is a prefix of others as more specific.
    • Specified that function overload sets can contain both
      declarative and non-declarative functions.
    • Clarified that the auto return type for functions is allowed
      on function definitions and not on function declarations.
    • Defined material definitions as functions returning a declarative
      structure definition of the material_category structure category.
    • Defined the builtin material type as a declarative structure type
      with the material_category structure category.
    • Changed the ior field of the builtin material type to
      be of varying type and not uniform color anymore.
    • Defined the builtin distribution function types bsdf,
      edf, vdf, and hair_bsdf as a declarative structure type
      without structure category and without fields.
    • Redefined the compound types in the material type to
      declarative structure types without structure category.
    • Added rounding function round_away_from_zero() and clarified rounding
      behavior of round().
    • Removed the statement for microfacet BSDF models that they become
      black in transmission mode if the ior values indicate total
      interior reflection.

  • General

    • The material.ior field is now varying for all versions of MDL (not only for MDL 1.9).
      This avoids to break the ABI to the renderer, since there is still only one material type.
      The new method IMdl_configuration::set_material_ior_frequency() allows to revert
      that change as a workaround for rare compatibility problems with MDL modules authored
      for MDL versions before 1.9. Please note that this will switch the field to uniform
      for all version (including MDL 1.9). Note further that this option cannot be set dynamically,
      it is only possible to set it at SDK start time.

    • Added CMake option MDL_BUILD_SDK which is ON by default. If set OFF
      only the MDL Core and mdlc will be built.

    • The alpha channel is now always treated as linear, independent of the gamma value
      specified for the remaining channels.

    • Added new AOV(Arbitrary Output Variables) support. It allows rendering applications
      to produce customized data in addition to the usual image output buffers:

      • The interface IStruct_category has been added to represent the new
        concept of struct categories. The new interface IStruct_category_list
        represents an ordered collection of struct categories identified by name or index.
      • The new methods IType_struct::get_struct_category() and
        IModule::get_struct_categories() allow to query the struct categories
        of a struct type and those defined in a module, respectively.
      • The new methods IType::is_declarative(),
        IFunction_definition::is_declarative(), and
        Definition_wrapper::is_declarative() indicate whether a type or function is
        declarative.
      • The interface IType_factory has been extended with methods to create,
        clone, compare, and dump struct categories and/or struct category lists.
      • A sixth template-like function, the so-called decl_cast operator, has
        been added. The new method IExpression_factory::create_decl_cast()
        provides a convenient way to create calls of this function definition.
      • The method IMaterial_instance::create_compiled_material() accepts the
        new option “target_type” on the execution context. This option behaves as if
        the entire material is wrapped into a decl_cast operator with the given
        target type.
      • The new method ICompiled_material::get_sub_expression_hash() allows to
        compute hash values of arbitrary sub-expressions and is not limited to
        predefined material slots as ICompiled_material::get_slot_hash().
      • The module builder has been extended to support struct categories: The new
        method IModule_builder::add_struct_category() allows to create them.
        The signature of the method IModule_builder::add_struct_type() has been
        extended with a struct_category parameter. The signature of several
        methods on IModule_builder has been extended with a
        is_declarative parameter. The old signatures are still available if
        MI_NEURAYLIB_DEPRECATED_15_0 is defined.

    • The methods for texture export (IMdl_impexp_api::export_canvas() and
      IImage_api::create_buffer_from_canvas()) and the image plugin API have
      been changed to use a generic options map instead of two hard-coded options.
      Supported options are documented at IImage_api. The old signatures are
      still available if MI_NEURAYLIB_DEPRECATED_15_0 is defined.

    • The new export option “exr:data_type” allows to control the data type of EXR channels.

    • The database supports now multiple scopes. For details, see the
      documentation for the class IScope and the methods create_scope(),
      get_scope(), and remove_scope() on IDatabase.

    • The new method IExpression_factory::create_temporary() allows to create
      temporary references.

    • The signature of IMdl_module_builder::add_function() has been extended
      to support temporaries. The old signature is still available if
      MI_NEURAYLIB_DEPRECATED_15_0 is defined.

    • The free functions set_value() and get_value() on IData
      support now arrays (similar to what existed already for IValue).
      Arrays can be specified as pointer/length pair, or as std::vector.

    • Exit codes in case of array size mismatches for the the free functions
      set_value() and get_value() on IValue have been changed
      from -3 (implementation) and -4 (documentation) to -5 for consistency with
      IData.

    • The methods IModule::get_function_overloads() have been updated to
      implement the modified overload resolution rules for MDL >= 1.9.

    • Additional performance improvements for the creation of compiled materials.

    • The recommended vcpkg version is now 5d675c7e5. This
      changes the recommended versions for Boost to 1.84, OpenImageIO to 2.5.8.0,
      and GLFW to 3.4.

    • The license texts have been split into three separate files
      LICENSE.md, LICENSE_IMAGES.md, and LICENSE_THIRD_PARTY.md.

    • The API reference documentation for the MDL SDK and MDL
      Core is now installed as part of the “install” target.

    • Added the compiler flag -flax-vector-conversions when
      using OpenImageIO on Linux and ARM.

    • Python Bindings:

      • Changed the recommended Python version to 3.10.
      • Improved type hints.
      • Added stub functions to keep the bindings backwards compatible.
      • Prepared for an update to Swig 4.2.1.
      • Added more tests to improve coverage.

  • MDL Compiler and Backends

    • Increased default MDL version to 1.9.
    • Added support for MDL 1.9:
      • Implemented math::round_away_from_zero().
      • Implemented structure categories, declarative structures and declarative functions.
      • Adapted overload resolution to new rules.
    • Added backend option "libbsdf_flags_in_bsdf_data" to enable use of the new
      "flags" field in the BSDF data structures for libbsdf. The flags can be used
      to restrict generated sample, evaluate, pdf and auxiliary functions to only
      calculate reflections, transmissions or both.
    • MDL Core API: Support for declarative structs and functions (AOV support).
      • Extend the abstract syntax of MDL
        • New declaration kind mi::mdl::IDeclaration::Kind::DK_STRUCT_CATEGORY and corresponding
          class mi::mdl::IDeclaration_struct_category for struct category declarations.
        • New methods on mi::mdl::IDeclaration_type_struct for modifying a struct’s category:
          • IQualified_name const *get_struct_category_name().
          • IDefinition const *get_struct_category_definition().
          • void set_struct_category_definition(IDefinition const *category_definition).
        • New methods on mi::mdl::IDeclaration_type_struct and mi::mdl::IDeclaration_function
          for the declarative flags:
          • bool is_declarative().
          • void set_declarative(bool flag).
        • New and modified methods on mi::mdl::IDeclaration_factory for handling struct categories
          and changes to structs and functions:
          • IDeclaration_struct_category *create_struct_category(...).
          • IDeclaration_type_struct *create_struct(bool declarative = false, ...,
            IQualified_name const *category_name).
          • IDeclaration_function *create_function(bool is_declarative, ...).
        • Changes to mi::mdl::IDefinition to support declarative features:
          • New definition kind DK_STRUCT_CATEGORY.
          • New definition flag DP_IS_DECLARATIVE.
          • New intrinsic DS_INTRINSIC_DAG_DECL_CAST.
          • New function IStruct_category const *get_category() to get the category of
            a category definition.
        • Changes to mi::mdl::IGenerated_code_dag to support declarative features:
          • New function property FP_IS_DECLARATIVE.
          • The accessor functions for functions and materials have been aligned, so each
            function exists. In one variant for functions and one for materials, for example:
            get_material_return_type() vs. get_function_return_type().
          • New accessor functions for struct categories: get_struct_category_count(),
            get_struct_category(), etc.
        • mi::mdl::IMaterial_instance::initialize() now has a parameter to set the target
          type of the material.
          When non-null, it specifies how the application how to view a material’s fields.
        • New functions on mi::mdl::IMaterial_instance:
          • lookup_sub_expression(...) Look up a materials sub-expression by path.
          • get_sub_expression_hash(...) get the hash of the sub-expression rooted at
            a given path.
        • mi::mdl::IType has changed: New function mi::mdl::IType::is_declarative() to return
          whether a type is declarative or not.
        • New type mi::mdl::IStruct_category to reference struct categories in struct types.
        • New method IStruct_category const *mi::mdl::IType_struct::get_struct_category().
        • New method mi::mdl::IType_factory::create_struct_category() to create struct
          category objects.
    • MDL Core API: Others
      • Change representation of enums and structs to immutable types:
        • mi::mdl::IType_enum::get_value() and lookup() methods have changed to return
          a Value object instead of using out parameters.
        • mi::mdl::IType_struct::get_field() has changed to return a Field object instead of
          using out parameters.
        • mi::mdl::IType_factory methods create_enum() and create_struct() are
          changed to take all relevant information and returning immutable types.
          get_predefined_struct() and get_predefined_enum() also return const pointers now.
      • Moved the mi::mdl::IMaterial_instance interface out of mi::mdl::IGenerated_code_dag.
      • Use size_t for MDL AST statement, expression and definition counts/indexes.
      • Make distilling available to users of ibmdl_core.

  • MDL Distiller and Baker

    • Extend distiller node types to include 4-way mixers.
    • The class mi::neuraylib::IBaker has been extended to support setting UV ranges.

  • MDL Core API examples

    • Added new code generation example.

  • MDL SDK examples

    • The documentation for the distilling examples is now
      reachable from the list of examples and topics.
    • Added a "--allowed_scatter_mode" parameter to the df_cuda, df_native, df_vulkan
      and dxr examples to demonstrate the BSDF flags feature.
    • MDL Example Distilling
      • Added option --uv_range.
      • Updated the example to detect whether the material being distilled and baked
        is using UV tile materials and if this is the case, run distilling and baking
        for each of the tiles.
    • MDL Example df_vulkan:
      • Added support for class compilation and implemented GLSL runtime functions
        for argument block reads.
      • Added command line option for outputting the generated GLSL target code to a file.
      • Changed command line option names to match other examples.
    • MDL Example DXR:
      • Renamed --gpu-debug to --gpu_debug to make it conform with other options.
      • Added colored console output for errors and warnings.
      • Changed D3D resource handling to make use of dynamic resources and add Microsoft Agility
        SDK as optional dependency.
      • Added the ‘-distill_debug’ option to improve debugging of distiller plugins.
      • Added support for MDL 1.9 custom materials using the ‘--material_type’ option.
      • Allowed to select and display AOV by specifying expressions using the ‘--aov’ option.
      • Improved the selection of MDL expressions to generate shader code for.
      • Switched MaterialX SDK to 1.38.9.
      • Added CAMERA_POSITION scene data for a prototype implementation of MaterialX NPR nodes.

Fixed Bugs

  • General

    • Python Bindings: Fixed various function binding issues.
    • Fixed get_option_count() and get_option_name() functions on IMdl_execution_context.
    • Fixed IImage::set_from_canvas() and IImage::reset_reader() to
      properly recognize uvtile sequences with a single tile.
    • Fixed support for search paths from “MDL_SYSTEM_PATH” and “MDL_USER_PATH”
      environment variables containing Unicode characters on Windows.

  • MDL Compiler and Backends

    • The MDL core compiler did not compute correctly the uniform/varying property of
      single expression body functions, causing these to be always treated as uniform.
      This is fixed now. Additionally, Now the result of presets is always the same as
      for the underlying function.
    • Improved performance in HLSL/GLSL code generator by reducing the amount of
      generated code.
    • Fixed ICompiled_material opacity methods always returning OPACITY_UNKNOWN
      for MDL 1.5 - 1.8 materials.
    • Fixed invalid code generation for scene data function calls in environment functions.
      Default values are returned now.
    • Fixed failing argument block creation with MDL Core, when INVALID_REF values
      are provided (like texture_2d()).
    • Fixed performance regression for materials using base::transform_coordinate() or
      base::lookup_volume_coefficients() in some cases.
    • Fixed assignments to wrong struct members for GLSL/HLSL in rare cases.
    • Fixed 3-argument df::tint() function using shading normal instead of geometry normal
      to differentiate between reflection and transmission.
    • Fixed derivatives not being calculated when they were “hidden” in a call
      in the arguments of a user-defined function which does not use derivatives.
    • Fixed a null pointer reference when parsing crafted LLVM bitcode metadata.
    • Disabled MI_MDL_HLSL_LOAD_MODULE environment variable
      to avoid loading unwanted LLVM modules.
    • Fixed use of unsupported comdats for the native backend on MacOSX.
    • Fixed crash when translating color mixers with bsdf()/edf() components.
    • Fixed lost derivatives when assigning structs to each other.
    • Do not erroneously generate GLSL variable names that are GLSL keywords or
      restricted identifiers.
    • Fixed handling of resource sets that contain a frame number first followed by an UDIM.
    • Fixed a potential crash due to wrong reference count management in handling of module imports.
    • Fixed GLSL/HLSL code generators that sometimes generate variables in the wrong scope.
    • Do not try to resolve empty resource urls (which are invalid in MDL), and do not rewrite
      them into pseudo-absolute.
    • “<package>/” form, they stay unchanged (and are still not valid).
      This is more a cosmetic improvement.
    • Do not create the same error message more then once in some conditions.
    • Fixed some crashes in the core compiler when heavily invalid code is compiled.
    • Fixed a case where error messages were generated without file/line number.
    • Fixed compilation of constant functions that use the uniform state.
    • Improved error message ‘xxx’ is not a package or module name; Now it shows the
      wrong part only instead of the whole package.
    • Fixed GLSL/HLSL backends to generate valid code for function returning void
      (in MDL: empty structs or arrays of size zero).
    • Warn, if a function returns a “void-like” value (and hence might be optimized away).

  • MDL Distiller and Baker

    • Transmissive_pbr target: Prevent division by zero in case of both diffuse and
      glossy contributions are colored black.
    • Improved support for vMaterials2 composites in the transmissive_pbr target.
    • Improved support for color_custom_curve_layer as it is used in some vMaterials (all distilling targets).
    • Reduced overestimation of diffuse contribution in treansmissive_pbr/ue4 targets for some cases.

  • MDL SDK examples

    • Fixed the OptiX 7 example such that it now also works on Linux.
    • Fixed linker errors in Vulkan examples when using Vulkan SDK version 1.3.275 or newer.
    • MDL Example Traversal:
      • Fixed usage information. The example expects a module as input, not a material.
    • MDL Example DXR:
      Fixed glTF interaction between volume attenuation and single sided materials.