Resource usage issue jetson nano

Robotics & Edge Computing Jetson Systems Jetson Nano
1

Hello there, i am currently trying to get collision avoidance working using resnet18, the problem is, after a while, the blocked value updates way too slowly (initially it works, after loading the model), when i run a command to see cpu and gpu usage , the cpu and gpu also work in that initial state (they get to almost 100%) and the blocked value updates and then after a while it doesnt update as often and the usages drop to almost 10% on the cpu and 0% on gpu

I do get a warning: system throttled due to over-current
what i have tried: optimising the model as much as i can for lower inference time, changing termal paste, hooking up the jetson nano to a lab bench and running it at 5v+2.5A (to get 10W), running the jetson_clocks script
I am using a jetson nano 4gb devkit with a 100mb sd card

I should also mention the DC drivers are controlled using a arduino uno via serial

this is the code i am running:

import torch
import torchvision
import torchvision.transforms as transforms
import torch.nn.functional as F
import cv2
import PIL.Image
import numpy as np
import traitlets
from IPython.display import display
import ipywidgets.widgets as widgets
import threading
from torch2trt import TRTModule

# Import Robot class and other required modules
from Robot import robot_serial
from jetcam.utils import bgr8_to_jpeg
from jetcam.csi_camera import CSICamera

import time

# Initialize the camera
camera = CSICamera(width=224, height=224, capture_width=224, capture_height=224, capture_fps=5)  # Set capture_fps to 10

# Initialize the robot
robot = robot_serial()

device = torch.device('cuda')

# Initialize the model
model_trt = TRTModule()
model_trt.load_state_dict(torch.load('best_model_trt.pth'))

mean = torch.Tensor([0.485, 0.456, 0.406]).cuda().half()
std = torch.Tensor([0.229, 0.224, 0.225]).cuda().half()

normalize = transforms.Normalize(mean, std)

# Create widgets for display
image_w = widgets.Image(format='jpeg')
blocked_slider = widgets.FloatSlider(description='blocked', min=0.0, max=1.0, orientation='vertical')
speed_slider = widgets.FloatSlider(description='speed', min=0.0, max=1.0, value=0.0, step=0.01, orientation='horizontal')

# Display widgets
display(widgets.VBox([widgets.HBox([image_w, blocked_slider]), speed_slider]))

# Create a threading Event for synchronization
camera_event = threading.Event()

# Introduce frame skipping mechanism
frame_skip = 1  # Skip every 5th frame
frame_count = 0

def clamp(n, min, max):
    if n < min:
        return min
    elif n > max:
        return max
    else:
        return n

# Initialize cam_data and a lock for synchronization
cam_data = None
cam_data_lock = threading.Lock()

# Define the camera thread
def camera_thread():
    global cam_data, frame_count
    while True:
        new_cam_data = camera.read()
        with cam_data_lock:
            cam_data = new_cam_data
        image_w.value = bgr8_to_jpeg(new_cam_data)
        frame_count += 1

        # Signal camera update every frame_skip frames
        if frame_count % frame_skip == 0:
            camera_event.set()

# Start the camera thread
camera_thread = threading.Thread(target=camera_thread)
camera_thread.daemon = True
camera_thread.start()

def preprocess(image):
    # Ensure the image has a correct shape (HWC) and data type (uint8)
    if len(image.shape) == 4 and image.shape[0] == 1:
        image = image[0]

    image = PIL.Image.fromarray(image.astype('uint8'))
    image = transforms.functional.to_tensor(image).to(device).half()
    image.sub_(mean[:, None, None]).div_(std[:, None, None])
    return image[None, ...]

# Define the main loop
def main_loop():
    while True:
        camera_event.wait()  # Wait for camera update
        start_time = time.time()

        with cam_data_lock:
            x = np.array([cam_data], dtype=np.uint8)
        x = preprocess(x)

        with torch.no_grad():
            y = model_trt(x)
            y = F.softmax(y, dim=1)

        end_time = time.time()
        inference_time = end_time - start_time
        #print("Inference Time:", inference_time)

        prob_blocked = float(y[0, 0])
        blocked_slider.value = prob_blocked

        if prob_blocked < 0.5:
            robot.forward(0, clamp(int((speed_slider.value * 100) - 20), 0, 100))
        else:
            robot.turnLeft(0, clamp(int(speed_slider.value * 100), 0, 100), clamp(int((speed_slider.value * 100) - 10), 0, 100))
            
        time.sleep(0.03)
        
        # Clear the event for the next update and free up memory by deleting variables that are no longer needed.
        del x, y, start_time, end_time, inference_time
        torch.cuda.empty_cache()
        
        camera_event.clear()

# Start the main loop in a separate thread
main_thread = threading.Thread(target=main_loop)
main_thread.daemon = True
main_thread.start()

sorry if what i said is confusing, english is not my main language, i will clear things up if asked
thank you!

3

Hi,

How about memory?
Could you also check the memory status with sudo tegrastats at the initial and the time it gets stuck?

Thanks.

4

Hello, ive managed to fix it, the issue was that i kept spamming the commands to the arduino, ive changed it to remember the last command and now it works properly.