Hi, I am trying to write a script which uses jetson.utils videoSource and videoOutput to run inference with Yolov8. However, right now I am just focusing on getting the input stream, converting the cudaImages to Numpy and back (need this later for Yolo) and then stream it. When running it on the Jetson Orin Dev Kit, everything worked so far, but the predict function from Yolo didnt work anymore, because pytorch didnt have access to the GPU. So I switched to using the default jetson-inference container, installed a few more packages using pip and then commited this to a new image.
Now my current problem: when I run my script (below), I get these 2 errors:
[OpenGL] glDisplay – X screen 0 resolution: 1920x1080
[OpenGL] glDisplay – X window resolution: 1920x1080
[OpenGL] failed to create X11 Window.
failed to find/open file /proc/device-tree/model
I ran video-viewer, which works fine and can create a X11 window, so I dont know whats the problem.
import sys from jetson_utils import cudaImage, cudaToNumpy, cudaFromNumpy, cudaConvertColor, cudaAllocMapped, cudaResize, cudaDeviceSynchronize from jetson_utils import videoSource, videoOutput from ultralytics import YOLO import numpy as np import cv2 from torch import cuda # create video sources & outputs output = videoOutput("webrtc://@:8554/my_output") input = videoSource("/dev/video0") # load model model = YOLO("models/yolov8n.onnx") # capture frames until end-of-stream (or the user exits) while True: # get Frame from Source as cuda Image cuda_img = input.Capture(format='rgb32f', timeout=1000) if cuda_img is None: # if a timeout occurred continue # Resize the image to match YOLOv8's input shape (640x640) cuda_resized = cudaAllocMapped(width=640, height=640, format=cuda_img.format) cudaResize(cuda_img, cuda_resized) # convert imgage to np.ndarray np_img = cudaToNumpy(cuda_resized) cudaDeviceSynchronize() # convert type to float16 np_img = np_img.astype("float16") # transpose to match yolov8 input shape (3, 640, 640) np_img = np_img.transpose(2, 0, 1) # add extra dimension to represent batch size (1, 3, 640, 640) np_img = np_img[np.newaxis, :] # inference #preds = model.predict(np_img, iou=0.5, device=0) # convert results to np.ndarray # preds = preds.plot() # convert back to cuda Image # Revert the array to the original shape np_img = np_img.squeeze() # Remove the first dimension (1) np_img = np_img.transpose(1, 2, 0) #transpose back to 640,640,3 cudaDeviceSynchronize() image = cudaFromNumpy(np_img) # Resize possible, but decreases quality because no interpolation, image becomes more "cubic" # image_resized = cudaAllocMapped(width=1280, height=720, format=image.format) # cudaResize(image, image_resized) # render frame with labels output.Render(image) # exit on input/output EOS if not input.IsStreaming() or not output.IsStreaming(): break