Torch model

[vovinsa]

Hi!
I have used this code:

while True:
    start = time()
    frame = cam.capture()
    frame = transforms.ToTensor()(frame)
    frame = frame.unsqueeze(0)
    frame = frame.to(DEVICE)
    print(frame.shape)
    out = net(frame)
    print(time() - start)

And have this time:

0.009600400924682617
torch.Size([1, 3, 224, 224])
0.010586738586425781
torch.Size([1, 3, 224, 224])
0.011191129684448242
torch.Size([1, 3, 224, 224])
0.3026876449584961
torch.Size([1, 3, 224, 224])
0.4999411106109619
torch.Size([1, 3, 224, 224])
0.5003457069396973
torch.Size([1, 3, 224, 224])
0.49947404861450195

Why model began to work slower?

[AastaLLL]

Hi,

Have you released the memory you uploaded to the GPU?

Thanks.

[vovinsa]

Using torch.cuda.empty_cache()?

[AastaLLL]

Hi,

It looks like you submit a bunch of cudaMemcpy in a short time.
If the jobs number is over than the computing channel, it’s possible the task needs to wait for the free channel.

Would you mind increasing the channel number to see if it helps?
https://elinux.org/Jetson/L4T/TRT_Customized_Example#Long_delays_when_submitting_several_cudaMemcpy

Thanks.