Hi,
I tried to measure the mobilenet_v2 inference time on xavier. (Batch size = 1)
The result was
Experiements: cpu
START: mobilenet
3.234218406677246sec
Experiements: cuda
START: mobilenet
0.27671399116516116sec
When using only CPU for inference, it tooks 3.23 seconds, and, I think, it is too much time than expected.
In this paper, https://arxiv.org/pdf/2005.05085.pdf Fig.6, MobilenetV2 tooks less than 1 second on different other devices.
Then, why xavier-pytorch-cpu needs much more time on inferencing mobilenet_v2 than above devices/environments?
(p.s. I typed ‘sudo jetson_clocks’ before measure inference time.)
Any advice will be very appreciated.
Thanks in advance.
YJ.
Following code is what I used for measuring inference time on xaiver.
import sys
import time
import subprocess
import torch
import torchvision
def download_models():
models = {}
models["mobilenet"] = torchvision.models.mobilenet_v2(pretrained=True)
return models
if __name__ == "__main__":
models = download_models()
for device in ['cpu', 'cuda']:
print("Experiements:", device)
for name, model in models.items():
print("START:", name)
model = model.to(device)
model.eval()
n = 20
total_time = 0
for i in range(n):
img = torch.rand(1, 3, 224, 224)
if device == 'cuda':
img = img.cuda()
start = time.time()
output = model(img)
end = time.time()
total_time += (end - start)
print(total_time/n)