2021-06-14 20:29:44.409943: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0 WARNING:tensorflow:Deprecation warnings have been disabled. Set TF_ENABLE_DEPRECATION_WARNINGS=1 to re-enable them. WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/optimizer.py:353: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead. WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/optimizer.py:361: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead. CONFIGS: FullyConnectedArch, /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/architecture/fully_connected.py activation_fn: swish layer_size: 512 nr_layers: 6 skip_connections: False weight_norm: True adaptive_activations: False CONFIGS: ExponentialDecayLR, /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/learning_rate.py start_lr: 0.001 end_lr: 0.0 decay_steps: 4000 decay_rate: 0.95 CONFIGS: AdamOptimizer, /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/optimizer.py beta1: 0.9 beta2: 0.999 epsilon: 1e-08 amp: False WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/arch.py:36: The name tf.make_template is deprecated. Please use tf.compat.v1.make_template instead. CONFIGS: PFSolver, porous_flow.py network_dir: ./network_checkpoint_ldc_2d initialize_network_dir: added_config_dir: rec_results: True rec_results_cpu: False rec_results_freq: 1000 max_steps: 400000 save_filetypes: vtk,np xla: False inner_norm: 2 outer_norm: 2 save_network_freq: 1000 print_stats_freq: 100 tf_summary_freq: 500 optimizer_params_index: None initialize_network_params: None seq_train_domain: [] config: {'config': SimNetConfig(activation_fn='swish', adaptive_activations=False, added_config_dir='', amp=False, beta1=0.9, beta2=0.999, decay_rate=0.95, decay_steps=4000, end_lr=0.0, epsilon=1e-08, initialize_network_dir='', inner_norm=2, layer_size=512, max_steps=400000, network_dir='./network_checkpoint_ldc_2d', nr_layers=6, outer_norm=2, rec_results=True, rec_results_cpu=False, rec_results_freq=1000, run_mode='solve', save_filetypes='vtk,np', skip_connections=False, start_lr=0.001, weight_norm=True, xla=False)} arch: lr: optimizer: equations: [, , , ] nets: [] diff_nodes: [] WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/solver.py:224: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead. WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/solver.py:236: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2021-06-14 20:29:47.747460: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2400000000 Hz 2021-06-14 20:29:47.747844: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x53dfa00 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-06-14 20:29:47.747863: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-06-14 20:29:47.748854: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /simnet/external/lib:/usr/local/cuda/extras/CUPTI/lib64:/usr/local/cuda/compat/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64 2021-06-14 20:29:47.748869: E tensorflow/stream_executor/cuda/cuda_driver.cc:282] failed call to cuInit: UNKNOWN ERROR (303) 2021-06-14 20:29:47.748885: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see: * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md * https://github.com/tensorflow/addons * https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue. UNROLLING GRAPH: WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/learning_rate.py:65: The name tf.train.exponential_decay is deprecated. Please use tf.compat.v1.train.exponential_decay instead. WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/learning_rate.py:66: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead. Traceback (most recent call last): File "porous_flow.py", line 145, in ctr.run() File "/usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/controller.py", line 91, in run self.solver.solve() File "/usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/solver.py", line 399, in solve train_op = self.initialize_optimizer(losses, global_step, self.lr.get_lr(global_step['step'])) File "/usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/solver.py", line 301, in initialize_optimizer train_op = self.optimizer.initialize(lr=lr, File "/usr/local/lib/python3.8/dist-packages/simnet-21.6-py3.8.egg/simnet/optimizer.py", line 69, in initialize train_op = opt.minimize(loss, global_step=step, colocate_gradients_with_ops=True, var_list=params) File "/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/optimizer.py", line 415, in minimize grads_and_vars = self.compute_gradients( File "/usr/local/lib/python3.8/dist-packages/horovod/tensorflow/__init__.py", line 470, in compute_gradients gradients = self._optimizer.compute_gradients(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/tensorflow_core/python/training/optimizer.py", line 531, in compute_gradients raise ValueError("No variables to optimize.") ValueError: No variables to optimize.