Working with custom constraints: criteria and outvar

Physics Simulation PhysicsNeMo-ML Model Framework Technical Support (PhysicsNeMo Only)
1

hello, I am trying to simulate a problem where we have to predict a damage in a solid mechanical model, but in doing so I am facing an problem with conditions on the same.

I need to solve the equation: self.equation["f"] = Max(|p| - r1, 0) + r2*f = 0, so I defined the same in modulus and in constraint section with outvar={"f":0}but I am facing the following error:

  1. I defined the function as it is (self.equation[“f”]) as mentioned above
[16:39:55] - JIT using the NVFuser TorchScript backend
[16:39:55] - JitManager: {'_enabled': True, '_arch_mode': <JitArchMode.ONLY_ACTIVATION: 1>, '_use_nvfuser': True, '_autograd_nodes': False}
[16:39:55] - GraphManager: {'_func_arch': False, '_debug': False, '_func_arch_allow_partial_hessian': True}
[16:40:06] - Installed PyTorch version 2.0.1+cu117 is not TorchScript supported in Modulus. Version 1.14.0a0+410ce96 is officially supported.
[16:40:06] - attempting to restore from: outputs/damage_model
[16:40:06] - Success loading optimizer: outputs/damage_model/optim_checkpoint.0.pth
[16:40:06] - Success loading model: outputs/damage_model/virtual_work_model.0.pth
[16:40:12] - [step:       1000] saved checkpoint to outputs/damage_model
[16:40:12] - [step:       1000] loss:  7.462e-02
[16:40:15] - Attempting cuda graph building, this may take a bit...
Error executing job with overrides: []

Traceback (most recent call last):
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/trainer.py", line 742, in _cuda_graph_training_step
    self.loss_static, self.losses_static = self.compute_gradients(
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/trainer.py", line 68, in adam_compute_gradients
    losses_minibatch = self.compute_losses(step)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/solver/solver.py", line 66, in compute_losses
    return self.domain.compute_losses(step)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/domain/domain.py", line 147, in compute_losses
    constraint.forward()
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/domain/constraint/continuous.py", line 130, in forward
    self._output_vars = self.model(self._input_vars)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/graph.py", line 234, in forward
    outvar.update(e(outvar))
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/utils/sympy/torch_printer.py", line 288, in forward
    output = self.torch_expr(args)
  File "<lambdifygenerated-13>", line 3, in _lambdifygenerated
    return (-10000.0*d__t + Max(0, Max(-10*d__x__x - 10*d__y__y - 0.5*sigma_xx*u__x - 0.5*sigma_xy*v__x - 0.5*sigma_yy*v__y, 10*d__x__x + 10*d__y__y + 0.5*sigma_xx*u__x + 0.5*sigma_xy*v__x + 0.5*sigma_yy*v__y) - 10000.0) - 10000.0)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/utils/sympy/torch_printer.py", line 154, in _max_torch
    value = (torch.zeros(tensor_shape) + value).to(device)
RuntimeError: CUDA error: operation not permitted when stream is capturing
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "damage_model.py", line 242, in run
    slv.solve()
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/solver/solver.py", line 173, in solve
    self._train_loop(sigterm_handler)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/trainer.py", line 535, in _train_loop
    loss, losses = self._cuda_graph_training_step(step)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/trainer.py", line 742, in _cuda_graph_training_step
    self.loss_static, self.losses_static = self.compute_gradients(
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/cuda/graphs.py", line 173, in __exit__
    self.cuda_graph.capture_end()
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/torch/cuda/graphs.py", line 79, in capture_end
    super().capture_end()
RuntimeError: CUDA error: operation failed due to a previous error during capture
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.

But applying the max operator is not helping, so I tried to separately define the 2 cases with criteria constraint.

....
    Equations = Model2D(E = E, nu = nu, lambda_=lambda_, mu = mu, rho = 1, beta = beta, flag = 1, r1 = r1, r2 = r2)
    ....

    DM1 = PointwiseInteriorConstraint(
        nodes=nodes,
        geometry=geo,
        outvar={"f1":0},
        criteria= Equations.equations["p"]<r1 and Equations.equations["p"]> -r1,
        batch_size=cfg.batch_size.lr_interior,
        bounds={x: bounds_x, y: bounds_y, t: bounds_t},
        lambda_weighting = {"f1": 1.0},
        parameterization={t: bounds_t, x: bounds_x, y: bounds_y},
        batch_per_epoch=500,
    ) 
    
    domain.add_constraint(DM1, "damage_model_1")

    DM2 = PointwiseInteriorConstraint(
        nodes=nodes,
        geometry=geo,
        outvar={"f2":0},
        criteria= Equations.equations["p"]>=r1 and Equations.equations["p"]=<-r1,
        batch_size=cfg.batch_size.lr_interior,
        bounds={x: bounds_x, y: bounds_y, t: bounds_t},
        lambda_weighting = {"f2": 1.0},
        parameterization={t: bounds_t, x: bounds_x, y: bounds_y},
        batch_per_epoch=500,
    ) 
   
   domain.add_constraint(DM2, "damage_model_2")

But this pops out the following error:

ret = run_job(
[09:11:40] - JIT using the NVFuser TorchScript backend
[09:11:40] - JitManager: {'_enabled': True, '_arch_mode': <JitArchMode.ONLY_ACTIVATION: 1>, '_use_nvfuser': True, '_autograd_nodes': False}
[09:11:40] - GraphManager: {'_func_arch': False, '_debug': False, '_func_arch_allow_partial_hessian': True}
Error executing job with overrides: []
Traceback (most recent call last):
  File "damage_model.py", line 220, in run
    DM1 = PointwiseInteriorConstraint(
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/domain/constraint/continuous.py", line 444, in __init__
    invar = geometry.sample_interior(
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/geometry/geometry.py", line 614, in sample_interior
    criteria_index, criteria(local_invar, local_params)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/geometry/helper.py", line 89, in criteria
    return fn_criteria(**inputs)
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/utils/sympy/numpy_printer.py", line 158, in grouped_lambda
    output.append(lambdify_f_i(**invar))
  File "/home/neo11/anaconda3/envs/test_env/lib/python3.8/site-packages/modulus/sym/utils/sympy/numpy_printer.py", line 142, in lambdify_f_i
    v = sp_lambdify_f_i(**x)
  File "<lambdifygenerated-15>", line 8, in _lambdifygenerated
    less(-0.5*sigma_xx(x, y, t)*Derivative(u(x, y, t), x) - 0.5*sigma_xy(x, y, t)*Derivative(v(x, y, t), x) - 0.5*sigma_yy(x, y, t)*Derivative(v(x, y, t), y) - 10*Derivative(d(x, y, t), (x, 2)) - 10*Derivative(d(x, y, t), (y, 2)), 10000.0))
NameError: name 'sigma_xx' is not defined

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

So I decided to tweak the script for PointwiseInteriorConstraint as follows:

    def __init__(
        ....
        c2: Union[Dict[str], Callable, None] = None, # intro to new custom-constraint
    ):

But even this didn’t helped and now the error is:

ret = run_job(
[08:29:57] - JIT using the NVFuser TorchScript backend
[08:29:57] - JitManager: {'_enabled': True, '_arch_mode': <JitArchMode.ONLY_ACTIVATION: 1>, '_use_nvfuser': True, '_autograd_nodes': False}
[08:29:57] - GraphManager: {'_func_arch': False, '_debug': False, '_func_arch_allow_partial_hessian': True}
Error executing job with overrides: []
Traceback (most recent call last):
  File "damage_model.py", line 220, in run
    DM1 = PointwiseInteriorConstraint(
TypeError: __init__() got an unexpected keyword argument 'c2'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Now I am out of ideas how I can make it work please help me with the same.

Thank you

Sincerely,
Nihal

2

Hi @nihalpushkar11

Starting off with the first error, try removing the =0. I.e.

self.equation["f"] = Max(|p| - r1, 0) + r2*f

The loss function of f will impose that this goes to zero. See the wave example for a sample implementation of a custom PDE if you have not already.

The second error seems to be because sigma_xx is not defined. Verify that you’ve defined this equation / symbol.

For the third error, verify that you have installed modulus with the custom implementation of the constraint. You could pip install your modified modulus version with a standard pip install or use a development mode / editable install.

3

hi @ngeneva, thank you for replying but could you please help me with these queries

  1. actually the constraint is f = -Max(|p| - r1, 0)/r2that needs to be satisfied everywhere. But the issue I am facing is with the Max operator as removing that solves the problem but with Max it pops out the error mentioned in part-1

  2. Where should I define them, like in the main script or the supporting scripts? {from where I import the equations}.

Could you please share some in-depth documentations on the same, so I can understand the structure and usage of the variables well.

  1. Also how can I introduce constraints for output variables as >=0, the dictionaries seem to equate the same to 0, but I want to put a constraint on quantities that they are negative or positive.

Thanks and regards

NIhal

4

As per your suggestion for part-2, I defined the symbol sigma_xx, but now facing a new error with the same:

line 376, in __nonzero__
    raise TypeError("cannot determine truth value of Relational")
TypeError: cannot determine truth value of Relational

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Please help