Maybe a silly question if there is a simple fix or method I’m missing :)
Is there any way to drive skeleton joints via another transform? Just basic translate and rotate transforms?
I see there are animation constraints mentioned in a beta build but can’t seem to find how to load them, as not in the extension manager for the USD composer Beta 2023.2.2 or 2023.2.5 release versions?
Seems there is a wiki example but not able to find where this can be tested?
If the constraint option does not work, is there any way just to direct connect transform attributes?
I’ve tested using an action graph, where on playback tick the transform of a ridged body is read and directly piped into the transform attribute of the target joint but there is no movement. Even though I can see in the graph both the correct value is being read and written in the nodes?
If i replace the target joint with a transform/another sphere it connected correctly.
Is there something that generally blocks connecting to skeleton joint transforms?
I could script something on tick to get the ridgid body transform value and set it on the joints, but seems over kill / odd way if there is no way just to connect things.
The desired result is just to have some rigid bodies using physX to simulate, and those transforms directly connected to a joint skeleton that is skinned to a mesh.
Thanks :D
In the image:
Read prim attribute getting the xformOp:translate of the sphere that starts by the hip joint but falls with physics
Write prim 1 that displays in the property panel the correct value , set to apply to the hip joint xformOp:translate but noting moves.
Write prim 2 sets xformOp:translate on cube, confirmed it follows the sphere perfectly.
So just seems setting joints xformOp:translate is not possible? or some reason its locked / is not compatible?
Relates to the same question here but was closed with no solution?
Closed as support for Audio 2 Face stopped? but the issue of not being able to transform OmniJoints still persists?
Also see two other requests for the same thing with no solutions?, starting to worry a simply connection/constraint from a transform to a OmniJoint is not possible?
Can’t believe if so!, would unlock so much potential and interesting use cases its mad to think its not supported?
If not I guess ill just have to export transform data to process in a 3D DCC (maya/blender). Seems such a odd omission given its such a basic but fundamental thing? (unless I’m missing something very obvious, but its taking too much time to research online or test I’m just going to opt to avoid using omniverse for this simple task).
Hi, I’ve attempted to use that to no avail, if you have any information how it should be connected?
The doc shows a target/source transform which makes sense assuming its to just apply one transform onto another. But the ActionGraph node looks different, there is no source/target transform. Just a transform, USD timecode and prim?
I’ve tried setting the Raw USD property prim path as the transform I want to drive,
Then I’ve got the rotate/translate from my source transform, used the “Make Transformation matrix from TRS” to convert the translate/rotate into a transform, and connected that.
I’ve also tried using the “get_prim_local_to_world_transform”
I just have 2 matrix constraints in the screen shot as one if driving the omniJoint, another just a cube to test if that connection is working. neither do so assuming the prim/connections are setup wrong.
Doesn’t seem very straight forward?
Does it need to be connected another way?
Or as there is no action/tick attribute to connect in, how does it evaluate if its not running on a playback tick?
Thanks
Ah I see above I was using the matrix constraint not the xform constraint (but would expect that to work as well?
I’ve tried looking for that extension in USD Composer 2023.1.1 (Beta) & 2023.2.5 Release but can’t seem to find it if you know what version it might be under?
I’ve also tried the Get/Set xfrom, which works when driving other objects like spheres/cubes, but it does not when setting the xform on a omniJoint.
So assuming possibly when i figure out how to access the xfrom constraint it may face the same issue?
The extension I have you uses no ActionGraph. It is just a simple drag and click tool. You just select two objects and bind them together. You are trying to make it too complex.
@vanessa.thompson just curious - are either versions a requirement in your workflow? because they are pretty dated at this point.
have you looked into the latest kit-app-templates (based on kit 107.3)? likely the extension mentioned is only in the later version and not backwards compatible with earlier builds. the mod also has a breakdown video regarding the installation of the latest Kit if you are looking for the instruction:
Hi, no worries, we have discussed with a Solutions Architect for a plan, for now we have just skipped this any used another DCC instead.
Out of date constraints / xfrom constraints in beta aside I would have expected this to be a very straight forward connection between 2 nodes that have transforms in them. So quite odd this is so hard to implement/relies on out dated or beta extensions?
I could try access the beta constraints or build a custom kit for the xform constraint?
But as other simpler methods have failed I don’t have the confidence to spend more time on this.
Defaulting to another DCC was faster and frictionless.
Love Omniverse but something so simple as this being missed has really surprised me!
I can think of many many use cases where this will cause a blocker or limit the usability of Omniverse!
We are glad you love OMNIVERSE and sorry to hear you are moving on to another software
We are not a DCC. Not at all. We are a more of a developer platform, and USD Composer is one of our templates.
As mentioned you were using a very old version with very limited abilities. The new version is free on Github as it is is 107.3
What you are trying to do is very simple and easy, but this would be done in ISAAC SIM. Not USD Composer.
Finally, we have a very extensive physics system, but not really between characters and objects. This is the problem. Robots and objects yes. Objects and objects yes. Not humans or characters that use the USDSkel system. Now if this was a robot, that would be easy.
Really see the value in OMNIVERSE hope I don’t come across negative :D, just flagging as a odd gap / may help for the future
Aware you are not a DCC, I would not see the above request as related to a DCC, more that I had to skip omniverse/revert back to a DCC in order to fix.
I was using USD composer (2023.2.2 or 2023.2.5) via the Omniverse launcher. Aware that will be deprecated soon, if features are available in a newer version ill check! By 107.3 did you have a link? At the moment I’ve been using the launcher so I just want to make sure I’m testing the right thing/latest version.
Isaac Sim 4.1.0. had the same blocking issues / no constraints. I opened the scene and the joints are abstracted away/hidden in the scene graph, so for now I’m just going to stick with the DCC option as hidden joints lead me to believe it’s going to be more complex / still not supported.
I understand your perspective, but as someone using omniverse for its intended use I see this as a missed opportunity
I don’t see the logic why these are separated! Just causes issues and a missed opportunity for easy solutions / features
Love using it for physics
This does not just apply to human character, I’m also working on robots where we have both physics rigs and VFX skeleton rigs. Some components don’t need to be simulated (even if they could it would not be the correct option in some use cases). But this limit is stopping us combining the two.
Even for pure robotics needs I have several current use cases where this would be a requirement (or would save a lot of hassle) and lots of future use cases.
Aware my example here is of a human character/skeleton but this applies to robotics as well in many many ways!
I think the language barrier here is that skeleton/skinned meshes in omniverse are mainly limited to human characters / game style animation clips blending to represent humans walking around a factory.
Whereas I’d see skeleton/skinned meshes very relevant to any robotics needs and many other omniverse use cases, so this blocker or convoluted way to connect the two would be an issue for me.
Happy to close the thread / just adding this for feedback :D
We are working with a Solutions Architect to demo our use cases/needs and experience (which is positive!) just this was a surprise / can be part of that feedback, as see how supporting this will unlock a lot of potential
Ok understood. But just to be clear, no one should be using USDSkel for robotics. That is a not real AI, RL, or ML. That is just pure VFX. If we make a robot walk at Nvidia, then it really learned to walk. And all of those are full joints driven by correct robotic kinematics. If you want a robot to pick up an object, then it REALLY has to pick up it, with hard work and training. Simply faking it with a bind it not we are all about. With respect.
For the new Kit versions, you have to use our “kit app template”. You have to do a small amount of building to use it, because we are moving to a developer focus, but it is easy.
1. Download "git" from https://git-scm.com/downloads
2. Start a windows command console by using running "cmd" from the Windows Start menu
3. Navigate to where you want to install the kit app template folder
4. Copy and run "git clone https://github.com/NVIDIA-Omniverse/kit-app-template.git"
5. run "cd kit-app-template"
6. run "repo template new"
7. select the “USD Composer” template
8. You can answer NO to the Application Layer, unless you need streaming.
9. run "repo build"
10. run "repo launch"
Thanks for the info, I was able to load up 107.3 & the Xform Constraint.
I can get it working as before for non omniJoint objects. But does not work when applied to an omniJoint, unless its the root/first joint under the skeleton type node. So still seems joints in the skeleton hierarchy are locked / unable to drive.
Image just shows the constraint setup, with two constraints, one for a Sphere another for a omniJoint.
With respect I fully understand and agree. I’m aware USDSkel is not designed for robotics.
I’m just flagging there may be use cases where combining the two is fruitful. Which in my case it was. I just needed to run a physics simulation using nvidia physics joints and colliders in your traditional very in-scope Nvidia way for an accurate simulation that works with all your workflows. I just wanted to map the resulting transforms of those nvidia physics joints onto a skinned skeleton. As the skinned mesh we did not require to be driven by Physx or a simulation itself, just to follow the result of the nvidia physics joints/rig. I just want it to follow along.
I’d assume just a simple xform / connection / get&set would work. But if there is a better/prefered way then I’m keen to know.
“If we make a robot walk at Nvidia, then it really learned to walk”
If you then want to capture synthetic data of that robot walking, but it needs to wear a jacket/cover that’s a deformable mesh that we do not want to simulate, but rather drive via skin weights / vertex influence per joint. How would you do that?
Aware we could use a cloth / deformable body, but if the requirement was not for that part to be simulated, or even we wanted to maintain/respect the per vertex influence weights. How would that be achieved?
I fully understand the use case of robotic, kinematic and physics joints. I don’t understand why skeleton/meshes are supported but simple connections to them are not. It’s ok if it’s not part of Nvidia’s use case, I’m not expecting it to cover used cases it’s not intended for. It’s not a DCC or designed for VFX. I can just see uses for skeleton meshes within the same target products and use cases Nvidia is going for.
I’m still learning so I’m happy to be wrong as well :D!
I can just see very legitimate use cases where a simple connection from one transform to another is blocked just because it’s an omniJoint type (But both source and target objects have transforms!).
I understand. I will pass along your comments as an area we can improve on.
If I had a fully simulated robot that was already set up and walking, I think it would be easier to just push a little more and actually fully simulate the clothing as well.
The clothing though is a little curious. Robots tend to either not wear clothing or if they do, it’s skin tight for a reason. They don’t want loose clothing to interfere with the kinematics.
The clothing example I was just expanding on your comment. If simulating gave better results for the use case we would use that. There are many robot types that require drapes/coverings. Getting into the weeds a little :D, aware cloth simulation may be better but there may be times we want to drive a mesh via vertex influence per joint (i.e. joints and skinning) if the requirement was to have a fixed unchanging bind between the physics joints and the vertex’s of a mesh, in this use case, as simulated mesh would not be the best option as it would break that ridged vertex bind to joint relationship.
We have some use cases we are showing to some Solutions Architect so can give more details there.
Thanks for following up and helping out. The USD composer 107.3 and Xform constraint was very helpful for other needs, thanks :D!
