WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/optimizer.py:353: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead. WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/optimizer.py:361: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead. /usr/lib/python3.7/importlib/_bootstrap.py:219: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. return f(*args, **kwds) CONFIGS: FullyConnectedArch, /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/architecture/fully_connected.py activation_fn: swish layer_size: 512 nr_layers: 6 skip_connections: False weight_norm: True adaptive_activations: False CONFIGS: ExponentialDecayLR, /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/learning_rate.py start_lr: 0.001 end_lr: 0.0 decay_steps: 15000 decay_rate: 0.95 CONFIGS: AdamOptimizer, /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/optimizer.py beta1: 0.9 beta2: 0.999 epsilon: 1e-08 amp: False WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/arch.py:36: The name tf.make_template is deprecated. Please use tf.compat.v1.make_template instead. CONFIGS: AneurysmSolver, aneurysm.py network_dir: ./network_checkpoint_aneurysm initialize_network_dir: added_config_dir: rec_results: True rec_results_cpu: True rec_results_freq: 1000 max_steps: 1500000 save_filetypes: vtk,np xla: False inner_norm: 2 outer_norm: 2 save_network_freq: 1000 print_stats_freq: 100 tf_summary_freq: 500 optimizer_params_index: None initialize_network_params: None seq_train_domain: [] config: {'config': SimNetConfig(activation_fn='swish', adaptive_activations=False, added_config_dir='', amp=False, beta1=0.9, beta2=0.999, decay_rate=0.95, decay_steps=15000, end_lr=0.0, epsilon=1e-08, initialize_network_dir='', inner_norm=2, layer_size=512, max_steps=1500000, network_dir='./network_checkpoint_aneurysm', nr_layers=6, outer_norm=2, rec_results=True, rec_results_cpu=True, rec_results_freq=1000, run_mode='solve', save_filetypes='vtk,np', skip_connections=False, start_lr=0.001, weight_norm=True, xla=False)} arch: lr: optimizer: equations: [, , , , ] nets: [] diff_nodes: [] WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/solver.py:224: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead. WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/solver.py:236: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2021-06-17 21:16:44.272573: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA 2021-06-17 21:16:44.277327: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2000179999 Hz 2021-06-17 21:16:44.277644: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55a33fe98680 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2021-06-17 21:16:44.277683: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2021-06-17 21:16:44.278850: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2021-06-17 21:16:44.388810: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-06-17 21:16:44.389617: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55a33fe984c0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2021-06-17 21:16:44.389650: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Tesla V100-SXM2-16GB, Compute Capability 7.0 2021-06-17 21:16:44.390847: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-06-17 21:16:44.391435: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: name: Tesla V100-SXM2-16GB major: 7 minor: 0 memoryClockRate(GHz): 1.53 pciBusID: 0000:00:04.0 2021-06-17 21:16:44.397439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0 2021-06-17 21:16:44.585185: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0 2021-06-17 21:16:44.659050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0 2021-06-17 21:16:44.687610: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0 2021-06-17 21:16:44.878635: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0 2021-06-17 21:16:44.988349: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0 2021-06-17 21:16:45.425648: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2021-06-17 21:16:45.425953: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-06-17 21:16:45.426672: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-06-17 21:16:45.427307: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1746] Adding visible gpu devices: 0 2021-06-17 21:16:45.427451: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0 2021-06-17 21:16:45.428877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-06-17 21:16:45.428907: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165] 0 2021-06-17 21:16:45.428915: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1178] 0: N 2021-06-17 21:16:45.429074: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-06-17 21:16:45.429677: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:983] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-06-17 21:16:45.430280: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:39] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0. 2021-06-17 21:16:45.430320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 15060 MB memory) -> physical GPU (device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:04.0, compute capability: 7.0) WARNING:tensorflow: The TensorFlow contrib module will not be included in TensorFlow 2.0. For more information, please see: * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md * https://github.com/tensorflow/addons * https://github.com/tensorflow/io (for I/O related ops) If you depend on functionality not listed there, please file an issue. WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/simnet-21.6-py3.7.egg/simnet/solver.py:175: get_or_create_global_step (from tensorflow.contrib.framework.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Please switch to tf.train.get_or_create_global_step tcmalloc: large alloc 2845573120 bytes == 0x55a46663e000 @ 0x7fa29e3f11e7 0x7fa29958546e 0x7fa2995d5c7b 0x7fa2995d5d18 0x7fa29967d010 0x7fa29967d73c 0x7fa29967d85d 0x55a33832ef68 0x7fa2995c2ef7 0x55a33832cc47 0x55a33832ca50 0x55a3383a0453 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b7ad 0x55a33826de2c 0x7fa2995c2ef7 0x55a33832cc47 0x55a33832ca50 0x55a3383a0453 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b7ad 0x55a33832e3ea 0x55a3383a07f0 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b4ae tcmalloc: large alloc 2845573120 bytes == 0x55a51087c000 @ 0x7fa29e3f11e7 0x7fa29958546e 0x7fa2995d5c7b 0x7fa2995d5d97 0x7fa29966e887 0x55a33832ccc0 0x55a33841dfed 0x55a3383a0988 0x55a33839b7ad 0x55a33832e3ea 0x55a3383a07f0 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b7ad 0x55a33832ec9f 0x55a33836fd79 0x55a33836ccc4 0x55a33832eea1 0x55a33839dbb5 0x55a33832e30a 0x55a33839c60e 0x55a33832e30a 0x55a33839c60e 0x55a33839b4ae 0x55a33839b1b3 0x55a338465182 0x55a3384654fd tcmalloc: large alloc 2845573120 bytes == 0x55a5ba23c000 @ 0x7fa29e3f11e7 0x7fa29958546e 0x7fa2995d5c7b 0x7fa2995d5d18 0x7fa2996a143b 0x7fa29966e2ab 0x55a33832ccc0 0x55a33841dfed 0x55a3383a0988 0x55a33839b7ad 0x55a33832e3ea 0x55a3383a07f0 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b7ad 0x55a33832ec9f 0x55a33836fd79 0x55a33836ccc4 0x55a33832eea1 0x55a33839dbb5 0x55a33832e30a 0x55a33839c60e 0x55a33832e30a 0x55a33839c60e 0x55a33839b4ae 0x55a33839b1b3 0x55a338465182 tcmalloc: large alloc 2845573120 bytes == 0x55a46663e000 @ 0x7fa29e3f3001 0x7fa29958554f 0x7fa2995d5b58 0x7fa2995d9b17 0x7fa299678203 0x55a33826e338 0x7fa20cc2c60d 0x7fa20cc324f5 0x55a33832cd54 0x55a33832ca50 0x55a3383a1105 0x55a33839b7ad 0x55a33832e3ea 0x55a3383a07f0 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b4ae 0x55a33832e3ea 0x55a33839d32a 0x55a33839b7ad 0x55a33832ec9f 0x55a33836fd79 0x55a33836ccc4 0x55a33832eea1 0x55a33839dbb5 0x55a33832e30a 0x55a33839c60e 0x55a33832e30a 0x55a33839c60e 0x55a33839b4ae [OptiX][ 4][ KNOBS]: All knobs on default. [OptiX][ 4][ DISK CACHE]: Opened database: "/var/tmp/OptixCache_root/cache7.db" [OptiX][ 4][ DISK CACHE]: Cache data size: "0 Bytes" terminate called after throwing an instance of 'std::runtime_error' what(): the provided PTX was compiled with an unsupported toolchain. [fc35a1064336:02488] *** Process received signal *** [fc35a1064336:02488] Signal: Aborted (6) [fc35a1064336:02488] Signal code: (-6) [fc35a1064336:02488] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7fa29e1b1980] [fc35a1064336:02488] [ 1] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0xc7)[0x7fa29d1f8fb7] [fc35a1064336:02488] [ 2] /lib/x86_64-linux-gnu/libc.so.6(abort+0x141)[0x7fa29d1fa921] [fc35a1064336:02488] [ 3] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x8c957)[0x7fa29cca2957] [fc35a1064336:02488] [ 4] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92ae6)[0x7fa29cca8ae6] [fc35a1064336:02488] [ 5] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92b21)[0x7fa29cca8b21] [fc35a1064336:02488] [ 6] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x92d54)[0x7fa29cca8d54] [fc35a1064336:02488] [ 7] /content/gdrive/My Drive/simnet/SimNet/external/pysdf/build/libsdf.so(+0x115bf)[0x7fa20cb4d5bf] [fc35a1064336:02488] [ 8] /content/gdrive/My Drive/simnet/SimNet/external/pysdf/build/libsdf.so(signedDistanceField+0x2fe)[0x7fa20cb51dfe] [fc35a1064336:02488] [ 9] /usr/local/lib/python3.7/dist-packages/pysdf-0.1-py3.7-linux-x86_64.egg/pysdf/sdf.cpython-37m-x86_64-linux-gnu.so(+0xb01f)[0x7fa20cc3201f] [fc35a1064336:02488] [10] python3(_PyMethodDef_RawFastCallKeywords+0x2e4)[0x55a33832cd54] [fc35a1064336:02488] [11] python3(_PyCFunction_FastCallKeywords+0x20)[0x55a33832ca50] [fc35a1064336:02488] [12] python3(_PyEval_EvalFrameDefault+0x5185)[0x55a3383a1105] [fc35a1064336:02488] [13] python3(_PyEval_EvalCodeWithName+0x5ed)[0x55a33839b7ad] [fc35a1064336:02488] [14] python3(_PyFunction_FastCallKeywords+0x1ea)[0x55a33832e3ea] [fc35a1064336:02488] [15] python3(_PyEval_EvalFrameDefault+0x4870)[0x55a3383a07f0] [fc35a1064336:02488] [16] python3(_PyEval_EvalCodeWithName+0x2ee)[0x55a33839b4ae] [fc35a1064336:02488] [17] python3(_PyFunction_FastCallKeywords+0x1ea)[0x55a33832e3ea] [fc35a1064336:02488] [18] python3(_PyEval_EvalFrameDefault+0x13aa)[0x55a33839d32a] [fc35a1064336:02488] [19] python3(_PyEval_EvalCodeWithName+0x2ee)[0x55a33839b4ae] [fc35a1064336:02488] [20] python3(_PyFunction_FastCallKeywords+0x1ea)[0x55a33832e3ea] [fc35a1064336:02488] [21] python3(_PyEval_EvalFrameDefault+0x13aa)[0x55a33839d32a] [fc35a1064336:02488] [22] python3(_PyEval_EvalCodeWithName+0x5ed)[0x55a33839b7ad] [fc35a1064336:02488] [23] python3(_PyObject_Call_Prepend+0x55f)[0x55a33832ec9f] [fc35a1064336:02488] [24] python3(+0x173d79)[0x55a33836fd79] [fc35a1064336:02488] [25] python3(+0x170cc4)[0x55a33836ccc4] [fc35a1064336:02488] [26] python3(PyObject_Call+0xb1)[0x55a33832eea1] [fc35a1064336:02488] [27] python3(_PyEval_EvalFrameDefault+0x1c35)[0x55a33839dbb5] [fc35a1064336:02488] [28] python3(_PyFunction_FastCallKeywords+0x10a)[0x55a33832e30a] [fc35a1064336:02488] [29] python3(_PyEval_EvalFrameDefault+0x68e)[0x55a33839c60e] [fc35a1064336:02488] *** End of error message *** [fc35a1064336:02510] *** Process received signal *** [fc35a1064336:02510] Signal: Segmentation fault (11) [fc35a1064336:02510] Signal code: Address not mapped (1) [fc35a1064336:02510] Failing at address: 0x7fc540df220d [fc35a1064336:02510] [ 0] /lib/x86_64-linux-gnu/libpthread.so.0(+0x12980)[0x7fc543c9f980] [fc35a1064336:02510] [ 1] /lib/x86_64-linux-gnu/libc.so.6(getenv+0xa5)[0x7fc5438de8a5] [fc35a1064336:02510] [ 2] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(_ZN13TCMallocGuardD1Ev+0x34)[0x7fc544149e44] [fc35a1064336:02510] [ 3] /lib/x86_64-linux-gnu/libc.so.6(__cxa_finalize+0xf5)[0x7fc5438df735] [fc35a1064336:02510] [ 4] /usr/lib/x86_64-linux-gnu/libtcmalloc.so.4(+0x13cb3)[0x7fc544147cb3] [fc35a1064336:02510] *** End of error message ***