Hi, everyone!
I want to use keras with multi-gpus to train network in PX2. But i meet the the ValueError,
ValueError: To call `multi_gpu_model` with `gpus=2`, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/gpu:0']. Try reducing `gpus`.
Using TensorFlow backend.
nvrm_gpu: Bug 200215060 workaround enabled.
2019-03-15 08:06:43.803666: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero
2019-03-15 08:06:43.804020: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 0 with properties:
name: Graphics Device major: 6 minor: 1 memoryClockRate(GHz): 1.29
pciBusID: 0000:04:00.0
totalMemory: 3.75GiB freeMemory: 3.68GiB
2019-03-15 08:06:43.901551: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:865] ARM64 does not support NUMA - returning NUMA node zero
2019-03-15 08:06:43.901759: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1212] Found device 1 with properties:
name: NVIDIA Tegra X2 major: 6 minor: 2 memoryClockRate(GHz): 1.275
pciBusID: 0000:00:00.0
totalMemory: 6.51GiB freeMemory: 3.52GiB
2019-03-15 08:06:43.901883: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1227] Device peer to peer matrix
2019-03-15 08:06:43.903048: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1233] DMA: 0 1
2019-03-15 08:06:43.903098: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1243] 0: Y N
2019-03-15 08:06:43.903141: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1243] 1: N Y
2019-03-15 08:06:43.903226: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1297] Ignoring visible gpu device (device: 1, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2) with Cuda multiprocessor count: 2. The minimum required count is 8. You can adjust this requirement with the env var TF_MIN_GPU_MULTIPROCESSOR_COUNT.
2019-03-15 08:06:43.903274: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1312] Adding visible gpu devices: 0
2019-03-15 08:06:46.437915: I tensorflow/core/common_runtime/gpu/gpu_device.cc:993] Creating TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3411 MB memory) -> physical GPU (device: 0, name: Graphics Device, pci bus id: 0000:04:00.0, compute capability: 6.1)
Create YOLOv3 model with 9 anchors and 13 classes.
Traceback (most recent call last):
File "train.py", line 199, in <module>
_main()
File "train.py", line 36, in _main
parallel_model=multi_gpu_model(model,gpus=2)
File "/usr/local/lib/python3.5/dist-packages/keras/utils/training_utils.py", line 138, in multi_gpu_model
available_devices))
ValueError: To call `multi_gpu_model` with `gpus=2`, we expect the following devices to be available: ['/cpu:0', '/gpu:0', '/gpu:1']. However this machine only has: ['/cpu:0', '/gpu:0']. Try reducing `gpus`.