I am trying to solve a problem using Replicator in Omniverse Code.
My situation is that I have 6 cameras, (6 Render Products) that need to be rendered on one frame after a physic sim. I can render 4 cameras at once, but using 6 will cause all the video ram to be used and it becomes very slow. I am using many annotators, (rgb, normals, bbox 3d, distance to camera). The resolution is 2560 x 1920. I realize that’s a lot for the video card to handle for that resolution and it makes sense that it will be painfully slow. I’m using the Path Tracer.
In my other extension I used omni.syntheticdata and rendered a camera then moved onto the next camera in a loop. But while using the new Replicator which uses nodes, I have yet to figure out a way to render the Render Products separately. It is my understanding that all the Render Products render at the same time and are subsequently waiting, (using a sync gate) until all products are rendered before sending to the Writer.
One solution which I have yet to figure out the best way is to render one product, then change the position of the camera, then render. I am using Ontime trigger which makes it difficult as the frame count has to be modified so that the ontime node does not trigger a new “scene”. As it is now I have to modify the frame count to accommodate the 32 samples needed. This issue could be addressed in it’s own post though.
Another solution may be to change the camera in the Render Product. How do I do this as there is the render product and the actual Hydra texture?
- What is the best way to change the camera of the Render Product while code is executing?
And while we are at it, it would be nice to render at a much lower resolution for the non hero frames during an ontime trigger scenario. For example if I have the ontime trigger using (interval=1, num=2) then there are 23 frames that are useless to me as they don’t get written. I have a custom writer subclassed from BasicWriter that does not write the non hero frame.
- Is there away to change the Render Product resolution on the fly?
I could run rep.orchestrator.preview(), then run through the physics process frame by frame. then start the render product process. But still have to loop through the cameras to produce the data and write them out. I don’t think I can use the Writer node at this point though which saves lots of coding as well as coding mistakes.
I can create my own nodes if needed. I have not done this yet but am ready and willing to start.
A solution that i don’t think will work is running the physics 6 times, (one for each camera) as I don’t trust the results of the sim will be the same every time. By the way it seems like rep.randomizer.instantiate does not take into account the global seed as it produces an output of different prims every time regardless of using rep.set_global_seed. While instantiate is in the unit tests, I don’t see a unit test that tests specifically for the consistency of using it with a global seed. I digress as this could be in another post.
- Is there another solution to render 6 cameras at high resolution with many annotators efficiently using the Replicator library?
I appreciate any help I can get.
Windows 11
Omniverse Code 2022.2.0
omni.replicator.core-1.4.7+wx64.r.cp37
RTX 3090
Thank you,
Mark