hello, I’m currently learning the official case for channel flow (k-ep turbulence model).
I am very confused about the setting of the weight of the loss function. I really want to know what is the basis for the setting of these weights? 10, 100, 1000, 10000?
When I set all these weights to 1, the training results do not match the results of the standard case at all.
Then when I train a new object, how do I set these weights, and what indicators can I refer to?
If anyone can help me with this confusion, I would be very grateful!
Hi @12227126
These weighting of the different constraints are done out of intuition and simply training the model partially or to convergence and observing which parts of the loss failed to converge.
Roughly speaking its good to have all losses in the same order of magnitude, but sometimes one constraint can be significantly more strong than another for multiple reasons. So it comes down to testing and observing the model.
There is some literature on automatic weighting (e.g. neural tangent kernels which is in Modulus in some form) but wide spread adoption of these approaches has not occurred.
Thank you very much for your reply.
I am still more puzzled. For example, when all loss weights are set to 1 for training, how can I judge which ones have failed through the results or the loss curve? How should I adjust the magnitude of the weight through this result？I wonder if you can share some experience of your engineers when adjusting the magnitude of this weight?
The second doubt is that you said that all losses should be at the same magnitude as possible, but I have observed that in some cases that have converged, their losses are not necessarily at the same magnitude.
Regarding the automatic weighting method, I have also tried several methods, but none of these methods seem to have achieved better results, and even the results of some methods are completely wrong.