TrOCR model running slow on Jetson Nano

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hi, I’m using TrOCR on a Nvidia Jetson Nano, and I’m noticing that it’s taking a considerable amount of time for inference, causing the system to perform slowly. Is there a way to improve this?

import torch
from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image

processor = TrOCRProcessor.from_pretrained('microsoft/trocr-large-printed')
model = VisionEncoderDecoderModel.from_pretrained('microsoft/trocr-large-printed')

def perform_ocr(image_path):
    image = Image.open(image_path).convert("RGB")

    pixel_values = processor(images=image, return_tensors="pt").pixel_values.to(device)
    generated_ids = model.generate(pixel_values)
    generated_text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]

    return generated_text

Thanks

3

Hi,

Have you maximized the performance?

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

Thanks.

4

I have tried this no improvement in speed.
I’ve identified that the issue lies with my PyTorch version not being compatible with CUDA. However, I can’t find CUDA, cuDNN, and TensorRT in my current Jetpack Version 4.6. Could you please assist me in resolving this?

Currently, when I check for CUDA availability using:

import torch
print(torch.cuda.is_available())

it doesn’t seem to detect CUDA.

5

Hi

Do you mean there is no CUDA-related library in your environment?
If yes, you can install it via below command:

$ sudo apt-get install nvidia-jetpack

Thanks

6

I’m encountering memory issues when trying to install package. The command output is as follows:

After this operation, 6,305 MB of additional disk space will be used.
E: You don't have enough free space in /var/cache/apt/archives/.

I’ve checked the memory usage with the df -h command and received this output:

Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk0p1   14G   13G   32M 100% /
none            1.7G     0  1.7G   0% /dev
tmpfs           2.0G   60K  2.0G   1% /dev/shm
tmpfs           2.0G   21M  2.0G   2% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           2.0G     0  2.0G   0% /sys/fs/cgroup
/dev/loop0       50M   50M     0 100% /snap/core18/2810
/dev/loop1       34M   34M     0 100% /snap/snapd/21761
/dev/loop2       34M   34M     0 100% /snap/snapd/21467
/dev/loop3       50M   50M     0 100% /snap/core18/2826
tmpfs           396M  104K  396M   1% /run/user/1000

I’ve tried to free up disk space, but I can only manage to clear about 1.5 GB. Can you suggest any solutions to resolve this issue?

7

Hi,

You can setup an external drive to expand the storage.

Thanks.

8

I am using a Jetson Nano eMMC module with an A205 carrier board. Does this process apply to my setup as well, or is it specific to the devkit? I have not flashed the OS from an SD card.

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.