You must generate two 32-bit float EXR textures, one for NDC (\rightarrow) ray direction and one for view direction (\rightarrow) NDC, then assign both texture assets plus the nominal sensor size and optical center on the camera prim. forums.developer.nvidia
What the API expects
OmniLensDistortionLutAPI uses a generalized projection model driven by two user-provided textures rather than a fixed polynomial lens model. The required schema attributes are omni:lensdistortion:lut:rayEnterDirectionTexture, omni:lensdistortion:lut:rayExitPositionTexture, omni:lensdistortion:lut:nominalWidth, omni:lensdistortion:lut:nominalHeight, omni:lensdistortion:lut:opticalCenter, and the model token omni:lensdistortion:model = "lut". forums.developer.nvidia
The two textures
The rayEnterDirectionTexture is an RGB, 32-bit float EXR where each texel stores an XYZ normalized view direction in camera-local space, and it represents the unproject operation from normalized screen coordinates to a ray direction. At runtime, Omniverse uses a pixel’s NDC as UVs into this texture and reads back the direction to trace the primary ray. forums.developer.nvidia
The rayExitPositionTexture is a 32-bit float EXR with two channels in practice stored in RG, and it represents the project operation from a view direction or view-space position back to normalized screen coordinates in ([0,1]). At runtime, the renderer samples this texture using an octahedrally encoded view direction to recover the corresponding NDC on the image plane. forums.developer.nvidia
How to generate them
The document says to write a Python script that fills image arrays and then packs them into EXR files, and its example uses OpenEXR, Imath, and numpy. For the direction texture, each texel’s normalized UV is treated as NDC, passed into your custom unproject function, and the returned normalized direction is written into the R, G, and B channels. forums.developer.nvidia
For the exit-position texture, each texel UV is treated as an octahedral-encoded direction, decoded to a 3D unit vector with the provided helper function, then passed into your custom project function, which must return normalized screen coordinates in ([0,1]); those are written into the R and G channels. NVIDIA’s sample script provides the octahedral encode/decode helpers, EXR save function, and a concrete fisheye example based on a Kannala-Brandt-style polynomial model. forums.developer.nvidia
Your Python workflow
Here is the exact workflow the page implies:
- Install Python dependencies:
OpenEXR, Imath, numpy; the sample also imports scipy.optimize.root_scalar for one inverse solve in the fisheye example. forums.developer.nvidia
- Implement
save_img() to write float EXR files with 2 or 3 channels. forums.developer.nvidia
- Reuse or adapt the sample’s octahedral helpers:
oct_to_unit_vector, unit_vector_to_oct, and get_octahedral_directions. forums.developer.nvidia
- Write your own unproject function: NDC/UV (\rightarrow) normalized camera-space ray. forums.developer.nvidia
- Write your own project function: camera-space ray (\rightarrow) normalized screen coordinate in ([0,1]). forums.developer.nvidia
- Populate the two EXR textures at a resolution at least as high as your target render resolution, because the page warns that lower-resolution LUTs can introduce projection artifacts. forums.developer.nvidia
Minimal code shape
This is the pattern you should follow, based directly on the document’s requirements and sample structure. forums.developer.nvidia
import OpenEXR
import Imath
import numpy as np
g_compression = Imath.Compression.ZIP_COMPRESSION
def save_img(img_path: str, img_data):
height, width, channel_count = img_data.shape
channel_names = ['R', 'G', 'B', 'A']
channel_map = {}
channel_types = {}
for i in range(channel_count):
channel_map[channel_names[i]] = img_data[:, :, i].astype(np.float32).tobytes()
channel_types[channel_names[i]] = Imath.Channel(
Imath.PixelType(Imath.PixelType.FLOAT)
)
header = OpenEXR.Header(width, height)
header['Compression'] = g_compression
header['channels'] = channel_types
exr = OpenEXR.OutputFile(img_path, header)
exr.writePixels(channel_map)
exr.close()
def oct_to_unit_vector(x, y):
dirX, dirY = np.meshgrid(x, y)
dirZ = 1 - np.abs(dirX) - np.abs(dirY)
sx = 2 * np.heaviside(dirX, 1) - 1
sy = 2 * np.heaviside(dirY, 1) - 1
tmpX = dirX
dirX = np.where(dirZ <= 0, (1 - np.abs(dirY)) * sx, dirX)
dirY = np.where(dirZ <= 0, (1 - np.abs(tmpX)) * sy, dirY)
n = 1.0 / np.sqrt(dirX**2 + dirY**2 + dirZ**2)
return dirX * n, dirY * n, dirZ * n
def get_octahedral_directions(width, height):
x = np.linspace(-1, 1, width)
y = np.linspace(-1, 1, height)
return oct_to_unit_vector(x, y)
def custom_unproject(u, v):
# u, v are normalized [0,1]
# Replace this with your real lens calibration model
x = 2.0 * u - 1.0
y = 2.0 * v - 1.0
z = -np.ones_like(x)
n = 1.0 / np.sqrt(x**2 + y**2 + z**2)
return x * n, y * n, z * n
def custom_project(dirX, dirY, dirZ):
# Replace this with the true inverse of custom_unproject
t = -1.0 / np.where(dirZ == 0, -1e-8, dirZ)
x = dirX * t
y = dirY * t
u = (x + 1.0) * 0.5
v = (y + 1.0) * 0.5
return u, v
def generate_ray_enter_direction_texture(texture_width, texture_height, out_path):
u = np.linspace(0.0, 1.0, texture_width)
v = np.linspace(0.0, 1.0, texture_height)
U, V = np.meshgrid(u, v)
dirX, dirY, dirZ = custom_unproject(U, V)
img = np.zeros((texture_height, texture_width, 3), dtype=np.float32)
img[:, :, 0] = dirX
img[:, :, 1] = dirY
img[:, :, 2] = dirZ
save_img(out_path, img)
def generate_ray_exit_position_texture(texture_width, texture_height, out_path):
dirX, dirY, dirZ = get_octahedral_directions(texture_width, texture_height)
u, v = custom_project(dirX, dirY, dirZ)
img = np.zeros((texture_height, texture_width, 2), dtype=np.float32)
img[:, :, 0] = u
img[:, :, 1] = v
save_img(out_path, img)
generate_ray_enter_direction_texture(3840, 2560, "ray_enter_direction.exr")
generate_ray_exit_position_texture(3840, 2560, "ray_exit_position.exr")
That code skeleton matches the document’s required conventions: float EXR output, RGB for direction, RG for NDC, UV-driven unproject texture generation, and octahedral-direction-driven project texture generation. forums.developer.nvidia
What your custom functions must do
Your unproject function should answer: “for this normalized pixel location on the sensor, what 3D ray enters the lens system?” Your project function should answer the inverse: “for this 3D incoming view direction, where does it land on the normalized image plane?” forums.developer.nvidia
If you already have a calibration model from OpenCV, a fisheye polynomial, a measured lens rig, or a table of rays from a calibration process, those become the math inside custom_unproject() and custom_project(). The sample fisheye script is specifically meant to be replaced in the marked spots with your own lens model if you are not using that exact polynomial example. forums.developer.nvidia
How to assign the textures in Python
The page shows that schemata are applied with camera_prim.ApplyAPI(...), and the OmniLensDistortionLutAPI section defines the exact attribute names you must set on the camera. So the Python usage pattern in Omniverse/Kit should look like this: forums.developer.nvidia
from pxr import Usd, Sdf
camera_prim = stage.GetPrimAtPath("/World/Camera")
camera_prim.ApplyAPI("OmniLensDistortionLutAPI")
camera_prim.CreateAttribute(
"omni:lensdistortion:model",
Sdf.ValueTypeNames.Token
).Set("lut")
camera_prim.CreateAttribute(
"omni:lensdistortion:lut:nominalWidth",
Sdf.ValueTypeNames.Float
).Set(1936.0)
camera_prim.CreateAttribute(
"omni:lensdistortion:lut:nominalHeight",
Sdf.ValueTypeNames.Float
).Set(1216.0)
camera_prim.CreateAttribute(
"omni:lensdistortion:lut:opticalCenter",
Sdf.ValueTypeNames.Float2
).Set((968.0, 608.0))
camera_prim.CreateAttribute(
"omni:lensdistortion:lut:rayEnterDirectionTexture",
Sdf.ValueTypeNames.Asset
).Set(Sdf.AssetPath("ray_enter_direction.exr"))
camera_prim.CreateAttribute(
"omni:lensdistortion:lut:rayExitPositionTexture",
Sdf.ValueTypeNames.Asset
).Set(Sdf.AssetPath("ray_exit_position.exr"))
Those attribute names and meanings come directly from the OmniLensDistortionLutAPI table on the page. forums.developer.nvidia
Optical center rule
The page has an important warning about optical center: either encode it into the textures or set it in the camera parameters, but do not effectively apply it twice. If your LUTs already include the optical center offset, set the camera optical center to half the nominal width and height; if your textures are centered at ((0.5, 0.5)), then set the true optical center on the camera attributes instead. forums.developer.nvidia
Resolution rule
The document explicitly warns that projection artifacts can appear if the LUT resolution is lower than the final target view resolution. In practice, that means you should usually generate the EXRs at the render resolution you care about, or somewhat higher if you need extra precision for aggressive distortion. forums.developer.nvidia
