I am working with Get3D to try to generate novel sculpture from my existing sculptures, they look like this. I altered the generator to help it generate the complex topology, and it works: Get3D can replicate sculptures faithfully and provide small neighborhoods around them.
However, interpolated forms, and all generated forms that do not replicate dataset items, are asymmetrical even if all dataset items have the same symmetry group. I would like to add a loss term to lock in the symmetry. Any ideas how to do this?
PS my version runs on one GTX1070 with 8GB, the generator has 6M parameters. That seems like enough, except for the symmetry issue it’s working, it can make motorbikes if needed. :-)
1 Like
Enforcing symmetry in the shape is possible by mirroring the signed distance function (SDF) values with respect to a plane of symmetry. For instance, if you’d like the shape to be symmetric with respect to the x-y plane, you can add a line after GET3D/networks_get3d.py at 2a09f5836e2a3ed49a37a6825a7139d11328446d · nv-tlabs/GET3D · GitHub to set sdf[ …, sdf.shape[-1]//2 + k] = sdf[ …,sdf.shape[-1]//2 - k], for k in range(sdf.shape[-1]//2). Additionally, you can also inference on only half of the grid to save memory.
1 Like
Thank you, I work with groups which include rotations and inversion transforms, in addition to reflection, but what you say can generalize.
Enforcing rigid symmetry in this way, or inferencing only on the fundamental unit of the grid, will clearly work. However it would create rigidly exact symmetric output, and I am looking for fully generated “fuzzy” symmetry which would (for example) allow interpolation between data items with different symmetry groups.
I tried comparing the generated mesh vertices to themselves under symmetry transforms using sliced optimal transport, and using that distance as a loss. I also tried doing a similarity loss on images from symmetrically related cameras. In both cases the generator could not learn to reduce asymmetry below random chance. I feel like it “should” be able to learn this, but I haven’t thought of the right way to train it.