Change the texture with 2d array (image) in python extension

[AlanHuang]

Hi, I’m implementing a texture editer in Omniverse Code 2022.3.1,
and found that I cannot subsitute the texture in the shader setting with a 2d array (np.ndarray) via python script.
It works by saving the array as local files, and editing the AssetPath, but it’s too slow and not intuitive :(
Is there any method to change the texture without editing the AssetPath?

Thanks!

[mati-nvidia]

Hi @AlanHuang! Sound cool! This should be possible with the DynamicTextureProvider API: ByteImageProvider — omni.ui 2.14.11 documentation
I show an example here: NVIDIA Omniverse Developer Office Hour - 11/4/2022 | NVIDIA On-Demand

Let me know how it goes!

[AlanHuang]

Thanks for the info!
It works now :)
Actually I’ve found DynamicTextureProvider but had no idea how to use it due to insufficient instruction.
Another problem I found is that the time consumption of set_byta_array seems large while editing AssetPath takes less. My texture contains diffuse, normal and roughness of 2K or 4K.
Is there any advice to optimize the performance?

Thanks again :)

[mati-nvidia]

HI @AlanHuang. The dev team has some more profiling work that they would like to do there. One alternative that I can think of is to try Variant Sets to do your texture swapping.

[AlanHuang]

Thanks for the concern!
I just found the way to improve the performance after I look into your example of video texture (GitHub - jshrake-nvidia/kit-cv-video-example: Example Omniverse Kit extension that demonstrates how to stream video to a dynamic texture using OpenCV VideoCapture and omni.ui.DynamicTextureProvider.).
I found that set_data_array is far faster than set_bytes_data, which I used due to the sRGB texture type.
Therefore I apply set_data_array with manual sRGB conversion. and the time cost is reduced to 0.02 from > 0.5.
I think it’s an acceptable time cost now.
Thanks again :)