[OpenGL] failed to create X11 Window when using videoOutput in container

Hi, I am trying to write a script which uses jetson.utils videoSource and videoOutput to run inference with Yolov8. However, right now I am just focusing on getting the input stream, converting the cudaImages to Numpy and back (need this later for Yolo) and then stream it. When running it on the Jetson Orin Dev Kit, everything worked so far, but the predict function from Yolo didnt work anymore, because pytorch didnt have access to the GPU. So I switched to using the default jetson-inference container, installed a few more packages using pip and then commited this to a new image.

Now my current problem: when I run my script (below), I get these 2 errors:
[OpenGL] glDisplay – X screen 0 resolution: 1920x1080
[OpenGL] glDisplay – X window resolution: 1920x1080
[OpenGL] failed to create X11 Window.


failed to find/open file /proc/device-tree/model

I ran video-viewer, which works fine and can create a X11 window, so I dont know whats the problem.

import sys
from jetson_utils import cudaImage, cudaToNumpy, cudaFromNumpy, cudaConvertColor, cudaAllocMapped, cudaResize, cudaDeviceSynchronize
from jetson_utils import videoSource, videoOutput
from ultralytics import YOLO
import numpy as np
import cv2
from torch import cuda

# create video sources & outputs
output = videoOutput("webrtc://@:8554/my_output")  
input = videoSource("/dev/video0")

# load model
model = YOLO("models/yolov8n.onnx")

# capture frames until end-of-stream (or the user exits)
while True:
    # get Frame from Source as cuda Image
    cuda_img = input.Capture(format='rgb32f', timeout=1000)
    if cuda_img is None:  # if a timeout occurred
    # Resize the image to match YOLOv8's input shape (640x640)
    cuda_resized = cudaAllocMapped(width=640, height=640, format=cuda_img.format)
    cudaResize(cuda_img, cuda_resized)
    # convert imgage to np.ndarray
    np_img = cudaToNumpy(cuda_resized)
    # convert type to float16
    np_img = np_img.astype("float16")
    # transpose to match yolov8 input shape (3, 640, 640)
    np_img = np_img.transpose(2, 0, 1)
    # add extra dimension to represent batch size (1, 3, 640, 640)
    np_img = np_img[np.newaxis, :]
    # inference
    #preds = model.predict(np_img, iou=0.5, device=0)
    # convert results to np.ndarray
    # preds = preds.plot()
    # convert back to cuda Image
    # Revert the array to the original shape
    np_img = np_img.squeeze()  # Remove the first dimension (1)
    np_img = np_img.transpose(1, 2, 0) #transpose back to 640,640,3
    image = cudaFromNumpy(np_img)
    # Resize possible, but decreases quality because no interpolation, image becomes more "cubic"
    # image_resized = cudaAllocMapped(width=1280, height=720, format=image.format)
    # cudaResize(image, image_resized)
    # render frame with labels

    # exit on input/output EOS
    if not input.IsStreaming() or not output.IsStreaming():

Hi @maxim.hansen2000, if you wish to use OpenGL display, try doing the import cv2 after you have created output = videoOutput("webrtc://@:8554/my_output")

cv2 seems to do it’s own initialization of OpenGL extensions which conflict with other libraries.

If you aren’t using docker/run.sh, see here in the script how this is setup:

Also note the -v /tmp/nv_jetson_model:/tmp/nv_jetson_model line in the docker run invocation.

So this actually helped, thanks a lot. I also had to move the ultralytics import, because that also includes opencv, but now the OpenGL window opens without a problem.

Here I still have the same problem, because I actually do use the run.sh script, I only added one more directory to be mounted with the other data volumes like so:

# generate mount commands
--volume $PWD/data:$DOCKER_ROOT/data \
--volume $PWD/$CLASSIFY_DIR/models:$DOCKER_ROOT/$CLASSIFY_DIR/models \

I also found one more error:

modprobe: FATAL: Module nvidia not found in directory /lib/modules/5.10.104-tegra

But I was able to fix this with the following flag in the docker/run.sh command and only want to mention this here as it might be part of the problem?:

-v /lib/modules/5.10.104-tegra:/lib/modules/5.10.104-tegra

Finally, it might be that my problem simply sits with Yolov8, because when I use the jetson.inference detectNet() with SSD-MobileNet, everything works as I want it to, however I still get the same error messages. So I guess if my code around Yolov8 would work, the whole application would work despite the error messages.

Thanks a lot for your help! :)

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.