Getting mesh ID and using Warp's mesh query in Omniverse

Omniverse General Discussion
, ,
1

Hello. I’m not experienced with programming, but I’ll try to keep it simple.

Given a mesh with point datasets and an array of coordinates (I’ll call it InputCoordinates). I need to extract the values of the mesh points closest to each InputCoordinate.

So far the closest function I can find to get the closest point is Warp’s mesh query point no sign.
I have the function written down in a script node in an action graph.

Here are the problems I’ve encountered so far.

  1. The query function requires a unique uint64 value called identifier, which there is a action graph node called get graph target id but I couldn’t find its equivalent python function. And so I created a new action graph, assigned the mesh to the visual script, so that the graph target id node will write the identifier uint64 into “_WarpIdentifier” attribute of the mesh.

After that the attribute is inputted into the script node where the warp function is run. But I couldn’t test if this is actually working, or if the ID is referring to the correct mesh.

Would be great if there is a python equivalent of getting the ID of a given bundle or prim path. Or an option to use prim bundle or path as input instead.

  1. Following the reference at Warp 0.13.0, the number of overloads for the function seems to be three: the mesh ID, the coordinate “Point” that I referred as InputCoordinates, and the maximum distance. An error about overloads occurred, and after looking to the .py files at extscache\omni.warp.core-1.0.0-beta.2\warp, specifically “builtins.py”, it turns out there are 6 overloads required, the other 3 being face index, and the UV coordinates of the points.

Regardless, the error stopped showing after entering the 6 parameters.

  1. The final one is CUDA illegal memory access, which causes USD composer to crash, and one I never find which is causing the issue. I’ll just show you the code written in the script node so far.
import warp as wp
import numpy as np
import logging

wp.init()

@wp.kernel
def PlayerFunction(	InputIdentifier: wp.uint64,
					InputCoordinates: wp.array(dtype=wp.vec3),
					OutputColors: wp.array(dtype=wp.vec3),
					OutputU: wp.array(dtype=float),
					OutputV: wp.array(dtype=float)):
	tid = wp.tid()
	
	QIIdentifier = InputIdentifier
	QICoordinate = InputCoordinates[tid]
	QIDistance = float32(1000)
	QOIndex = int32(0)
	QOUCoor = float32(0)
	QOVCoor = float32(0)
	
	Result = wp.mesh_query_point_no_sign(QIIdentifier, QICoordinate, QIDistance, QOIndex, QOUCoor, QOVCoor)
	# Will continue if the warp function is fixed
	
def compute(db):
	logger = logging.getLogger(__name__)
	
	uid = wp.uint64(db.inputs.InputIdentifier)
	coords = wp.array(db.inputs.InputCoordinates, dtype=wp.vec3)
	colors = wp.zeros(len(coords), dtype=wp.vec3)
	UValue = wp.zeros(len(coords), dtype=float)
	VValue = wp.zeros(len(coords), dtype=float)
	
	logger.info("Running Warp kernel")
	
	# launch kernel
	wp.launch(kernel=PlayerFunction,
		dim=1024,
		inputs=[uid , coords, colors, UValue, VValue])
	
	logger.info("Warp kernel completed")

image
image1920×1080 154 KB

Any insights would be appreciated, thanks.

2

I will inquire and get some advice back to you, thanks

3

Hi @luijoevin, the id for wp.mesh_query_point() needs to be from a wp.Mesh object. You can see how to construct one here:

You can then access mesh.id and pass that to a kernel, or down through the OmniGraph (just make sure that the original mesh object will continue to exist somewhere).

For some more basic examples of how to use Warp mesh objects see the following standalone example script: warp/examples/example_mesh.py at main · NVIDIA/warp · GitHub

Hope that helps.

Cheers,
Miles

4

Hello again. I have made some progress and a new kind of problem.

I managed to create a warp mesh and get the query function working. Once the function returns the result face indices, I use warp.mesh_get_index to convert them into point indices. But judging from the position of the points, I think they gave me the wrong indices.

As seen in the image, look for the coordinates of
InputPos: xformOp:translate of the input coordinate
InputPos Warp: InputPos converted to warp arrays
Output Face Pos: Position of face using warp.mesh_eval_position and the face indices from the query
Output Point Pos (mesh_get_point): Position of points using warp.mesh_get_point and the indices of points converted from mesh_get_index
Output Point Pos (point[WarpPointIndex]): Position of points using “points” attribute of the mesh and the indices of the points converted from mesh_get_index

image
image917×838 43.8 KB

Here are the codes used:

@wp.kernel
def PlayerFunction(	InputIdentifier: wp.uint64,
					InputCoordinates: wp.array(dtype=wp.vec3),
					OutputFaceIndex: wp.array(dtype=int),
					OutputPointIndex: wp.array(dtype=int),
					OutputFacePos: wp.array(dtype=wp.vec3),
					OutputPointPos: wp.array(dtype=wp.vec3)):
	
	tid = wp.tid()
	
	QCoord = InputCoordinates[tid]
	QFaceIndex = wp.int(0)
	QUCoord = wp.float(0)
	QVCoord = wp.float(0)
	QPointIndex = wp.int(0)
	
	Result = wp.mesh_query_point_no_sign(InputIdentifier, QCoord, wp.float32(1), QFaceIndex, QUCoord, QVCoord)
	
	if(Result):
		OutputFaceIndex[tid] = QFaceIndex
		FUCoord = wp.float(0)
		FVCoord = wp.float(0)
		OutputFacePos[tid] = wp.mesh_eval_position(InputIdentifier, OutputFaceIndex[tid], FUCoord, FVCoord)

		OutputPointIndex[tid] = wp.mesh_get_index(InputIdentifier, QFaceIndex)
		OutputPointPos[tid] = wp.mesh_get_point(InputIdentifier, OutputPointIndex[tid])

	else:
		OutputFacePos[tid] = wp.vec3(-4.04, -4.04, -4.04)
		OutputPointPos[tid] = wp.vec3(-4.04, -4.04, -4.04)

def GenerateWarpMesh(func_InputPoints, func_InputIndices):
	WarpPoints = wp.array(func_InputPoints, dtype=wp.vec3f)
	WarpIndices = wp.array(func_InputIndices, dtype=wp.int32).flatten()
	
	return wp.Mesh(WarpPoints, WarpIndices)

# some other functions
# ...
# some other functions

def Debug_GetMeshPointAtIndex(func_InputMeshPoint, func_InputIndex):
	OutputPos = []
	for IndexSingle in func_InputIndex:
		OutputPos.append(func_InputMeshPoint[IndexSingle])
	
	return OutputPos

def compute(db: og.Database):
	logger = logging.getLogger(__name__)

	InputGeomPoints = db.inputs.InputMeshPoints
	InputArrowGroup = GetArrowGroup(db.inputs.InputArrowGroupParent)
	ArrowCoords = GetArrowCoord(InputArrowGroup)
	
	logger.info("Debug: " + str(ArrowCoords))
	
	WarpMesh = GenerateWarpMesh(db.inputs.InputMeshPoints, db.inputs.InputMeshIndices)
	
	logger.info("compute: Warp Mesh generated")
	
	WarpMeshID = WarpMesh.id
	WarpCoords = wp.array(ArrowCoords, dtype=wp.vec3)
	WarpFaceIndex = wp.zeros(len(WarpCoords), dtype=int)
	WarpPointIndex = wp.zeros(len(WarpCoords), dtype=int)
	WarpFacePos = wp.zeros(len(WarpCoords), dtype=wp.vec3)
	WarpPointPos = wp.zeros(len(WarpCoords), dtype=wp.vec3)
	
	# launch kernel
	wp.launch(kernel=PlayerFunction,
		dim=len(WarpCoords),
		inputs=[WarpMeshID , WarpCoords, WarpFaceIndex, WarpPointIndex, WarpFacePos, WarpPointPos])
	
	logger.info("compute, warp result.")
	logger.info("InputPos: " + str(ArrowCoords))
	logger.info("InputPos Warp: " + str(WarpCoords))
	logger.info("Output FaceIndex: " + str(WarpFaceIndex.numpy()))
	logger.info("Output Face Pos: " + str(WarpFacePos))
	logger.info("Output Point Index: " + str(WarpPointIndex.numpy()))
	logger.info("Output Point Pos (mesh_get_point): " + str(WarpPointPos))
	DebugPointPos = Debug_GetMeshPointAtIndex(db.inputs.InputMeshPoints, WarpPointIndex.numpy())
	logger.info("Output Point Pos (point[WarpPointIndex]): " + str(DebugPointPos))

#some other functions
# ...
# some other functions

And also, with GenerateWarpMesh function being in compute(db) function, it’s probably going to run every single frame, which explains the reduced framerate.
Since the geometry of the mesh I’m running query on isn’t going to change shape, this process could run only once (which “setup(db)” function can help do the job, I just only need the id of the mesh).
However as of now I don’t have a way to pass the mesh reference from setup(db) to compute(db).

Any insight would be appreciated. Thanks.

5

Hi @luijoevin,

First, regarding the recreation of the mesh each update - I recommend you follow the pattern in the other Warp nodes which use the OmniGraph ‘internal state’ concept to store a state object which holds the mesh, etc. You can see an example here:

Regarding your code snippet, I think the problem is that wp.mesh_get_index() expects a value between [0, num_faces*3]. So to get the vertex indices of the closest triangle you would do:

vertex_i = wp.mesh_get_index(InputIdentifier, QFaceIndex*3+0)
vertex_j = wp.mesh_get_index(InputIdentifier, QFaceIndex*3+1)
vertex_k = wp.mesh_get_index(InputIdentifier, QFaceIndex*3+2)

You can then use those indices directly in wp.mesh_get_point() to retrieve the individual vertex positions.

Hope that helps.

Cheers,
Miles

6

Hello again. I have made some progress with the conversion from face-vertex-indices to point-indices.

I multiplied the QFaceIndex by 3 in wp.mesh_get_index() and took the new result value OutputPointIndex[tid] to wp.mesh_get_point(). However, the result outputted from wp.mesh_get_point() still points to a wrong location.

As a test, I created a linear comparison kernel LinearCompVert to compare InputCoordinate to each mesh point and find the closest mesh point’s index.

That’s when I noticed that the result OutputPointIndex[tid] outputted from wp.mesh_get_index() contains point-indices that matches the result from LinearCompVert.

Turns out I got the correct point-indices after multiplying QFaceIndex by 3 in wp.mesh_get_index(), but then wp.mesh_get_point() gave me a wrong location even with the correct point-indices. (Reason why remains unknown to me, but at this point the wp.mesh_get_point() function is no longer of use.)

By putting the new point-indices into InputMeshPoint, I managed to get 3 positions convincingly close to each InputCoordinate.
The 3 positions in each InputCoordinate contains one that is verified to be the closest point possible, using LinearCompVert.

image
image1350×472 61.8 KB

At this point, the number of points to compare is small enough, I decided to use numpy.argmin() to find the closest point’s index out of the 3 positions.

Since the point dataset is constructed to correspond with the mesh point, I can extract the dataset value of the closest points using the same indices I just found.

With that, the program achieved its purpose. I can continue writing the script to extract the point dataset value using the stored assigned index in each InputCoordinates prim for each tick, and run the Query Kernel again whenever there’s a change to the position of InputCoordinates.

I’ll post the cleaned-up code here as reference for anyone that comes across the same problem:

import warp as wp
import numpy as np
import math
import logging
from pxr import Usd, Sdf, Gf
import omni.usd
from typing import Union

# Main Query Kernel

@wp.kernel
def PlayerFunction(	InputIdentifier: wp.uint64,
					InputCoordinates: wp.array(dtype=wp.vec3),
					InputMeshPoints: wp.array(dtype=wp.vec3),
					QueryPointIndex: wp.array(dtype=int, ndim=2),
					QueryPointPos: wp.array(dtype=wp.vec3, ndim=2),
					QueryPointDist: wp.array(dtype=wp.float32, ndim=2)):
	
	tid = wp.tid()
	
	QCoord = InputCoordinates[tid]
	QFaceIndex = wp.int(0)
	QUCoord = wp.float(0)
	QVCoord = wp.float(0)
	
	Result = wp.mesh_query_point_no_sign(InputIdentifier, QCoord, wp.float32(50), QFaceIndex, QUCoord, QVCoord)
	
	if(Result):
		QueryPointIndex[tid][0] = wp.mesh_get_index(InputIdentifier, QFaceIndex*3)
		QueryPointIndex[tid][1] = wp.mesh_get_index(InputIdentifier, QFaceIndex*3 + 1)
		QueryPointIndex[tid][2] = wp.mesh_get_index(InputIdentifier, QFaceIndex*3 + 2)
		QueryPointPos[tid][0] = InputMeshPoints[QueryPointIndex[tid][0]]
		QueryPointPos[tid][1] = InputMeshPoints[QueryPointIndex[tid][1]]
		QueryPointPos[tid][2] = InputMeshPoints[QueryPointIndex[tid][2]]
		QueryPointDist[tid][0] = wp.length(QueryPointPos[tid][0] - QCoord)
		QueryPointDist[tid][1] = wp.length(QueryPointPos[tid][1] - QCoord)
		QueryPointDist[tid][2] = wp.length(QueryPointPos[tid][2] - QCoord)
		
	else:
		QueryPointIndex[tid][0] = -1
		QueryPointIndex[tid][1] = -1
		QueryPointIndex[tid][2] = -1

# Initialization, warp mesh generation and getting arrow prim

def GenerateWarpMesh(func_InputPoints, func_InputIndices):
	WarpPoints = wp.array(func_InputPoints, dtype=wp.vec3f)
	WarpIndices = wp.array(func_InputIndices, dtype=wp.int32).flatten()
	
	return wp.Mesh(WarpPoints, WarpIndices)

def GetArrowGroup(func_InputArrowGroupParent):
	CurrentStage = omni.usd.get_context().get_stage()
	InputArrowGroupPrim = CurrentStage.GetPrimAtPath(func_InputArrowGroupParent)
	return InputArrowGroupPrim.GetAllChildren()

def GetArrowCoord(func_InputArrowGroup):
	func_ArrowCoords = []
	for i, InputArrowGroupIndex in enumerate(func_InputArrowGroup):
		func_ArrowCoords.append(InputArrowGroupIndex.GetAttribute("xformOp:translate").Get())
	return func_ArrowCoords

# ----------
# Some other functions...
# ----------

def compute(db: og.Database):
	logger = logging.getLogger(__name__)

	InputGeomPoints = db.inputs.InputMeshPoints
	InputArrowGroup = GetArrowGroup(db.inputs.InputArrowGroupParent)
	ArrowCoords = GetArrowCoord(InputArrowGroup)
	
	WarpMesh = GenerateWarpMesh(db.inputs.InputMeshPoints, db.inputs.InputMeshIndices)
	
	logger.info("compute: Warp Mesh generated")
	
	WarpMeshID = WarpMesh.id
	WarpCoords = wp.array(ArrowCoords, dtype=wp.vec3)
	WarpMeshPoint = wp.array(db.inputs.InputMeshPoints, dtype=wp.vec3)
	WarpPointIndex = wp.zeros(shape=(len(WarpCoords), 3), dtype=int)
	WarpPointPos = wp.array(shape=(len(WarpCoords), 3), dtype=wp.vec3)
	WarpPointDist = wp.array(shape=(len(WarpCoords), 3), dtype=wp.float32)
	
	# launch kernel
	wp.launch(kernel=PlayerFunction,
		dim=len(WarpCoords),
		inputs=[WarpMeshID, WarpCoords, WarpMeshPoint, WarpPointIndex, WarpPointPos, WarpPointDist])
	
	logger.info("compute, warp result.")
	for i in range(len(ArrowCoords)):
		logger.info("Arrow " + str(i) + " at " + str(ArrowCoords[i]))
		for j in range(3):
			logger.info("Point Index: " + str(WarpPointIndex.numpy()[i][j]) + ". Distance " + str(WarpPointDist.numpy()[i][j]) +  " at " + str(db.inputs.InputMeshPoints[WarpPointIndex.numpy()[i][j]]))
	logger.info("----------")
	
	ClosestPointIndex = []
	ClosestPointPos = []
	
	logger.info("ClosestIndex: ")
	for i in range(len(WarpPointDist)):
		func_DistClosestIndex = np.argmin(WarpPointDist.numpy()[i])
		func_AssignedIndex = WarpPointIndex.numpy()[i][func_DistClosestIndex]
		ClosestPointIndex.append(func_AssignedIndex)
		ClosestPointPos.append(db.inputs.InputMeshPoints[func_AssignedIndex])
		logger.info("Arrow " + str(i) + " at position " + str(ArrowCoords[i]))
		logger.info(str(ClosestPointIndex[i]) + " at position" + str(ClosestPointPos[i]))
	logger.info("----------")

And the Linear Comparison test kernel, in case you’re also looking for fool-proof method of finding the closest mesh point:

@wp.kernel
def LinearCompVert(	InputIdentifier: wp.uint64,
					InputCoordinates: wp.array(dtype=wp.vec3),
					InputPointCoordinates: wp.array(dtype=wp.vec3),
					OutputDistance: wp.array(dtype=wp.float32, ndim=2)):
	
	tidArrow, tidPoint = wp.tid()
	
	OutputDistance[tidArrow, tidPoint] = wp.length(InputPointCoordinates[tidPoint] - InputCoordinates[tidArrow])

# Paste the code below in compute

	LCV_WarpInputMeshPoint = wp.array(db.inputs.InputMeshPoints, dtype=wp.vec3)
	LCV_WarpOutputDistance = wp.zeros(shape=(len(ArrowCoords), len(LCV_WarpInputMeshPoint)), dtype=wp.float32)
	LCV_WarpOutputDistance.fill_(9999.9)
	
	wp.launch(kernel=LinearCompVert,
		dim=(len(ArrowCoords), len(LCV_WarpInputMeshPoint)),
		inputs=[WarpMeshID, WarpCoords, LCV_WarpInputMeshPoint, LCV_WarpOutputDistance],
		device="cuda")
	
	LCV_ClosestDistance = []
	LCV_ClosestIndex = []
	LCV_ClosestPointPosition = []
	
	logger.info("compute, LinearCompVert")
	for Index in LCV_WarpOutputDistance.numpy():
		LCV_ClosestDistance.append(min(Index))
		func_Index = np.argmin(Index)
		LCV_ClosestIndex.append(func_Index)
		LCV_ClosestPointPosition.append(db.inputs.InputMeshPoints[func_Index])
	for i, ArrowIndex in enumerate(ArrowCoords):
		logger.info("Arrow " + str(i) + " at " + str(np.round(WarpCoords.numpy()[i], 2)))
		logger.info("Index: " + str(LCV_ClosestIndex[i]) + ", Distance " + str(LCV_ClosestDistance[i]) + " at position " + str(np.round(LCV_ClosestPointPosition[i], 2)))
	logger.info("----------")

There are many improvements that can be made, such as interpolation between mesh points and their corresponding point dataset value, or that I haven’t yet implemented passing WarpMesh from setup() to compute(), or for some reason wp.argmin(QueryPointDist[tid]) in the kernel shows an error saying "array has no attribute ‘name’ ". But I can do that later and update here; or create a new topic in the forum. Of course I’m still open for any new insights.

Much appreciated for the help.

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.