Is there the ability to use Python Script to load image and Shader files in both Local and Nucleus?

[ft-lab]

Hi,
I have recently been doing some in-depth research on Hydra and have found two useful features.

The ability to load image and Shader files (text files), whether local or Nucleus, is available in C++.
I was curious if this could be implemented in Python Script, so I was looking into it.
However, it seemed that it could not be implemented because the corresponding class was not defined.

Load image file (both Local and Nucleus)

The following is an implementation in C++.

#include "pxr/pxr.h"
#include "pxr/imaging/glf/image.h"

#include <string>
#include <iostream>
#include <memory>
#include <vector>

using namespace PXR_INTERNAL_NS;

int m_width, m_height;

int m_bytesPerPixel;	// Bytes per pixel.
GLenum m_format;		// GL_RGB/GL_RGBA etc.
GLenum m_type;			// GL_UNSIGNED_BYTE、GL_FLOAT etc.
std::string m_fileName;	// File name.

std::vector<unsigned char> m_data;	// Stores data.

bool LoadTextureImage (const std::string& fileName)
{
	const GlfImageSharedPtr& imgR = GlfImage::OpenForReading(fileName);
	if (!imgR) return false;
	
	m_width         = imgR->GetWidth();
	m_height        = imgR->GetHeight();
	m_format        = imgR->GetFormat();
	m_bytesPerPixel = imgR->GetBytesPerPixel();
	m_type          = imgR->GetType();
	m_fileName      = imgR->GetFilename();

	// Buffer to be allocated.
	const size_t buffSize = m_width * m_height * m_bytesPerPixel;
	m_data.resize(buffSize);

	// Read image buffer.
	GlfImage::StorageSpec storage;
    storage.width   = m_width;
    storage.height  = m_height;
    storage.format  = m_format;
    storage.type    = m_type;
	storage.flipped = true;			// Upside down for OpenGL.
	storage.data    = &(m_data[0]);
	const bool ret = imgR->Read(storage);

	return ret;
}

Python Script on Omniverse does not define Glf.Image and this could not be implemented.
Local files can be read with PIL’s Image.open.
It would be wonderful if image files on Nucleus could also be easily brought to PIL.Image.

Currently, is it a shortcut to copy the file locally with omni.client.copy and then pass it to the PIL.Image ?
However, this is a bit of a roundabout way of doing things.

Load text file (both Local and Nucleus)

The following is an implementation in C++.

#include "pxr/base/arch/fileSystem.h"
#include "pxr/usd/sdf/fileFormat.h"

#include "pxr/usd/ar/resolver.h"
#include "pxr/usd/ar/asset.h"

#include <istream>
#include <fstream>
#include <iostream>
#include <memory>

using namespace PXR_INTERNAL_NS;

std::string LoadShaderSource (const std::string& fileName)
{
	ArResolver& resolver = ArGetResolver();
	const std::string resolvedPath = resolver.Resolve(fileName);
	if (resolvedPath.empty()) return "";

	std::shared_ptr<ArAsset> asset = resolver.OpenAsset(resolvedPath);
	if (!asset) return "";
	const size_t bSize = asset->GetSize();

	std::vector<unsigned char> buffer(bSize + 16);
	asset->Read(&(buffer[0]), bSize, 0);
	buffer[bSize] = '\0';

	return std::string((char *)&(buffer[0]));
}

Load the file directly using ArResolver.OpenAsset.
This could also be done from C++ for both local and Nucleus.
In the case of Hydra, I needed to pass the MDL directly to the renderer, so this feature was very helpful.
In Python Script, I gave up because OpenAsset of Ar.Resolver is not defined.
Would it be better to use omni.client.copy to copy the text locally and then load the text, as I did with the image?

Research Environment

OS : Windows 10
Omniverse Create 2022.1.5
Visual Studio 2019 (Hydra Check)

Python Script was checked with Omniverse Create 2022.1.5 and 2022.2.0.

[mati-nvidia]

Hi @ft-lab. Good question! There is omni.client.read_file and omni.client.read_file_async so that you can read the binary contents of any file and then avoid the copy workaround you’re using. Here’s an example covering the PIL case you mentioned:

import io

import omni.client
from PIL import Image

result, vers, content = omni.client.read_file(r"omniverse://localhost/Users/mcodesal/AwesomeUSD_2by1_sm.png")
image_data = memoryview(content).tobytes()
image = Image.open(io.BytesIO(image_data))
image.show()
image.save(r"C:\temp\usd.png")

Let me know if you have any more questions.

[ft-lab]

Thank you for your prompt reply.
Wow,This is a great answer!
It worked perfectly!!