Hi, My goal is to create a full scale scene with randomized weather, time of day, and vegetation. I figured out how to randomize the day time but I believe that there must be a third party tool for generating procedural grass, trees, vegetation (entire scene) and live syncing that into omniverse. After some researching I found PlantFactory and PlantCatalog. But I’m not sure if it is the industry standard for generating randomized scenes and syntactic data. I couldn’t find much information about this. What third party tools do you use (or know of) for generating procedural vegetation in Isaac Sim? Ideally this tool should have API for scene modification and live sync to Omniverse. Thank you.
i think the most robust tool on the market and in typical production house is Houdini: Houdini | 3D Procedural Software for Film, TV & Gamedev | SideFX
learning curve is quite steep but the entire software package and selling point is everything being fully procedural and node based; and, it already has connector available with OV.
there are other programs that could accomplish similar task but Houdini was the one that’s at the top of the list.
if you were referring to the creation of individual assets for plants, Speedtree plays a huge role in game development and film as well. however, it primarily does vegetations assets before it gets procedurally scattered using other tools down the pipeline.
a more lightweight alternative is Treeit.
In the archviz industry, GrowFX is often used as a plugin to 3DS Max.
Blender also has several options to choose from, one of them is the tree generator.
these tools aren’t meant to generate any data as they are purely visual assets. i suspect it’ll be a separate endeavor should there’s a need to do so.
@Simplychenable Thank you for your responce these tools seem great. Just to be more specific, I am a syntatic data generator. my goal to take X images of an outdoor environment and than randomize that scene and take another X images and so on for an unlimeted amout of time (days). There for I need a software where I do 2 things: 1-create a rebost scene with procedural vegetation (grass, trees, plants) and sky/clouds/weather. 2-have a script that changes the assets (randomly?) once in a while. Such that I wont have to manually change and export the scene/assets (The scripts should do everything). From your experience, which tool sounds like what I need. Thank you so much for your help.
frankly, i don’t feel like i am equipped to make a solid recommendation for your use case; i would defer to your peers in synthetic generation (so hopefully mods/devs can chime in here).
full transparency, i am a 3d artist with basic understanding of the guiding principle of SDG and the process it entails (mostly through learning OV, so somewhat limited). whilst i understand what you are looking to accomplish, there are certain aspects of your process that i cannot say with 100% certainly the applications mentioned above could accommodate.
to paraphrase your ask - you are looking to employ Isaac Sim and live sync as part of your pipeline while randomize programmatically the following aspects with API:
- time of day
- cloud cover
- weather
- vegetation assets (grass, trees, etc)
- placement/layout of vegetation
is the outdoor environment strictly landscape scenes? would there need to be other man-made elements that require additional logic? for example, building types, height of them, with sensible city/street layout?
to think out loud here, the first hurdle is finding connector(s) that supports live sync with OV because some DCCs are not yet supported with the feature despite them being considered more industry standard. one that i forgot to mention is another E-ON package: VUE, which does support live sync, but not sure if API is exposed and may require contacting the dev for confirmation.
@Simplychenable thank you for the quick response. Its great to get some help and insight. Your paraphrasing of my question is on point. in regards to men-made objects, I will need at some point that as well. However, I dont want to limit myself right now with the task at hand. If I find a complete solution to the nature elements, I want to use it, even if it does not contain man-made assets.
I came across VUE, PlantFactory, and PlantCatalog and I too am yet to find an API for basic UI manual opporations.
if you could get some synthetic generation devs to comment on here that would be great! Do you know any channels where I can directly contact that group?
Thank you,
Danielle
there’s a dedicated SDG channel, so it’s your call if you’d want to move the thread over: Synthetic Data Generation (SDG) - NVIDIA Developer Forums
one of the mods @pcallender in that channel could probably provide additional insight.
Hi, how did you manage to randomize the time of day?
@mhz this is only a speculation on my part, but one possibility is utilizing the premade procedural environment that ship with OV (there are seven of them and shows a play icon in the thumbnails, which you can find inside the Environment browser).
these procedural environment should allow control over the time of day, cloud coverage, etc. but i’ll let defer to the OP.
@danielle.sisserman This sounds like a really interesting project. Traditionally, Replicator has dealt with simpler scenes for domain randomizing. You might have a better time asking in the USD Composer forums on creating complex dynamic scenes. I will also ask around internally if there’s recommendations for a project of this type. I’ll circle back when I have some info.
Some comments of my own:
There really isn’t an industry standard way of doing this within an SDG context afaik. If I needed to tackle this I’d Look into two approaches:
-
I’d use tools from the VFX and Gaming industries, like blender, houdini. These are probably the closest thing to an industry standard.
-
I’d look into a custom python code where you need randomization within the scene at runtime. Something like a placement along spline or something, but right now this probably needs to be coded up by you.
Overall, we’re looking into making placement in SDG drastically simpler, but this is a longer term initiative.