UTF-8 char in file name for texture_2d() causes compilation arror


While the MDL SDK supports generally UTF-8 files, I encountered a situation, where an unicode char in an UTF-8 encoded .mdl file (with a texture file name for texture_2d()) causes a compilation error “)” expected. For example the unicode “lion” (♌) char (UTF-8 0xE2 0x99 0x8C) works well when used in a file name.
But the “unicorn” char (🦄) (UTF-8 0xF0 0x9F 0xA6 0x84) is shown correctly in Windows Explorer and in the Notepad Editor; and even here in the forum, but when compling a file with such a char in its file name in the UTF-8 encoded .mdl file, MDL SDK, reports compilation error “)” expected .

F0       11110000
9F       10011111
A6       10100110
84       10000100

I’ve found nothing indicating any UTF-8 changes in the relase notes / documentation, so
the MDL SDK 2020.1.2 version of the file compilercore_wchar_support.cpp seems to be still unchanged like in the latest version: github compilercore_wchar_support.cpp #L135

reproduce by renaming a jpg file to “🦄P♌M.jpg” and load this MDL material:

mdl 1.6;
import ::df::*;
import ::state::*;
import ::math::*;
import ::base::*;
import ::tex::*;
export material NodeMaterial(
  uniform texture_2d Texture = texture_2d("🦄P♌M.jpg", ::tex::gamma_srgb  ),
  float specular = 0.5,
  float roughness = 0.5,
  float ior = 1.5
) = let{
  base::texture_coordinate_info coordinate = base::coordinate_source(coordinate_system: base::texture_coordinate_object);
  float3 TexCoord3 = state::texture_coordinate(0);
  float2 TexCoord = float2(TexCoord3.x, TexCoord3.y);
  float4 TexImageReturn = tex::lookup_float4(Texture, TexCoord);
  bsdf DiffuseLayer = df::diffuse_reflection_bsdf(
    tint: color(TexImageReturn.x,TexImageReturn.y,TexImageReturn.z),  roughness: 0.0
  float roughnessU = roughness * 0.1;
  float roughnessV_2 = roughness * 0.1;
  bsdf GlossyShader_2 = df::microfacet_ggx_smith_bsdf(
    roughness_u: roughnessU,
    roughness_v: roughnessV_2,
    tint: color(1.0),
    mode: df::scatter_reflect
  float spec_weight_3 = specular * 0.08;
  bsdf BsdfPrincipledGGX_Return = df::weighted_layer(
    weight: spec_weight_3,
    layer: GlossyShader_2,
    base: DiffuseLayer
  bsdf scatteringBSDF = BsdfPrincipledGGX_Return;

} in material(
    surface: material_surface(
      scattering: scatteringBSDF

If you rename another jpg file to “P♌M.jpg” and also remove the unicorn char in the texture_2d () file name within the .mdl file, compilation succeeds.

my system:

  • Win10PRO 64bit (version 21H1; build 19043.1237)
  • device driver: 526.47
  • CUDA 11.8
  • MDL SDK 2020.1.2 ABI
  • (pre-compiled binaries MDL build 334300.5582, 12 Nov 2020)
  • VS2019 v16.11.17


thanks for the report. We can reproduce the issue and track this as MDL-1040.

Right now only Unicode characters up to U+FFFF are supported (the BMP). The unicorn is outside of that range. Of course, a proper error message is missing. Support for the full Unicode range is planned for later.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.