Originally published at: https://developer.nvidia.com/blog/bootstrapping-object-detection-model-training-with-3d-synthetic-data/
Learn step by step how to use NVIDIA Omniverse to generate your own synthetic dataset. Then fine-tune your computer vision model deployed in NVIDIA Triton for inference.
I’m a bit confused about which direction to take with Replicator, the way described in this tutorial or using Replicator Composer. Composer appears to be a higher level tool but I haven’t been able to find how to set the semantic labels of the items drawn. I don’t see anything in the Replicator Composer manual related to semantics.
Any pointers anyone?
Thanks,
Dave
Hi Dave, thanks for this question! Composer as it is now will be deprecated soon so utilizing a Python script with the Replicator API is how we would suggest you can get started today with generating your semantic labels on your dataset. However, there will be a Composer update coming soon that you might be interested in checking out! Hope that helps.
Thanks for the answer. In the meantime however I got the Replicator Composer (RC) warehouse demo working and was able to swap in my own assets, get annotations, etc.
So now that I have two ways of running Replicator, do you still recommend I stick with the Python more manual mode?
I am happy to hear that Replicator Composer is working for you. I think it depends on the timeline you are looking at in regard to your projects. Python API is well developed and can be used now which is a big benefit. Composer will see changes in the future which might affect your workflow as it stands now. Up to you which works best for your specific case though!
The problem is, and you can see in my other very long thread to tech support, the VS Code debugger is great but it crashes Code after a few iterations. So I’m stuck with no python debugger.
I REALLY REALLY wish you guys would prioritize fixing the stability of Code when using your own suggested extension debugging method. Not sure where it is on the queue but it’s incredibly painful to have to reboot Code after each try of my Replicator debug session.
Something I’ve noticed is the time to render grows exponentially with image number.
The only changes to the code I made were output image size (224x224), output directory and number of frames generated.
I doubt exponential growth in render times is NVIDIA’s intention. For example I generated 1000 224x224 images and it takes about 3 minutes for 100 frames. But took 17 hours for 1000 frames.
I am building the synthetic data following this article. However, I was stuck when I was coding the following code:
bbox2d_loose_file_name = “bounding_box_2d_tight_0.npy”
data = np.load(os.path.join(out_dir, bbox2d_tight_file_name))
bbox2d_tight_labels_file_name = “bounding_box_2d_tight_labels_0.json”
with open(os.path.join(out_dir, bbox2d_tight_labels_file_name), “r”) as json_data:
bbox2d_loose_id_to_labels = json.load(json_data)
colorize_bbox_2d(rgb_path, data, bbox2d_loose_id_to_labels, os.path.join(vis_out_dir, “bbox2d_tight.png”))
The output noticed me that there is no bounding_box_2d_tight_0.npy despite the fact that I already made BasicWritter code before. Can you show me how can I get that .npy file ?
Thanks for this question, it looks like you are trying to visualize the data. Do you have any files ending in .npy, or are you only seeing the .json and .png files?
Linking to workaround posted on the other blog: Randomizer based Replicator code gets slower every frame - #5 by jlafleche
That workaround doesn’t work as far as I can tell. See the end of that other thread.
I did find .npy, .json and .png in my created folder. Are they generated by the above codes, right? So .npy and .json shall be link via path to my folder and the .png are, too ?
Yes all three types of files should be in the same folder! The BasicWriter code is where the files are generated. The code snippet you included above is actually us labelling and example image with the generated bounding boxes and labels.
Has this been released?