Unable to run Visionworks through X forwarding

Robotics & Edge Computing Jetson Systems Jetson AGX Xavier
1

Hey,

I’m trying to do some development on my Xavier over SSH, using a CSI camera, Visionworks and OpenVX, and attempting to use X forwarding to view my programs output.

I am running into an issue that when connecting to the Xavier over SSH with -X or -Y.
When running my Visionworks program from the SSH session it loads correctly then I receive “Error: Cannot open source” from the following code snippet:

    nvxio::Application &app = nvxio::Application::get();
    ovxio::printVersionInfo();
    ovxio::FrameSource::Parameters config;

    std::string resolution = "1920x1080", input = "device:///nvcamera?index=0";

    app.setDescription("This sample captures frames from NVIDIA GStreamer camera");
    app.addOption('r', "resolution", "Input frame resolution", nvxio::OptionHandler::oneOf(&resolution,
        { "2592x1944", "2592x1458", "1920x1080", "1280x720", "640x480" }));

    app.init(argc, argv);

    parseResolution(resolution, config);

    std::unique_ptr<ovxio::FrameSource> source(ovxio::createDefaultFrameSource(context, input));

    if (!source) {

        std::cout << "Error: cannot open source!" << std::endl;

        return nvxio::Application::APP_EXIT_CODE_NO_RESOURCE;

    }

However, when I connect to the SSH session without -X, the Visionworks code correctly connects to the CSI camera source, but fails to create the default render as there is no attached display.

    std::unique_ptr<ovxio::Render> render(ovxio::createDefaultRender(
                context, "OpenVX Testing", config.frameWidth, config.frameHeight));
    if (!render) {
        std::cout << "Error: Cannot open default render!" << std::endl;
        return nvxio::Application::APP_EXIT_CODE_NO_RENDER;
    }

I am assuming that the input source “device:///nvcamera” is being overwritten or changed when connecting with X forwarding, however I’m unsure as to how I would go about correcting the issue.
And I can confirm that X forwarding is functioning correctly, can work with gvim, openCV imview() etc etc

I also see the same results when attempting to load the camera stream using gstreamer:

    chris@chris-desktop:~/Documents/Spec-Rite/build$ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw    (memory:NVMM), width=(int)1928, height=(int)1208, framerate=30/1' ! nvvidconv flip-method=0 ! 'video/x-raw, format=(string)I420' ! xvimagesink -e
    No protocol specified
    nvbuf_utils: Could not get EGL display connection
    Setting pipeline to PAUSED ...
    No protocol specified
    ERROR: Pipeline doesn't want to pause.
    ERROR: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: Could not initialise Xv output
    Additional debug info:
    xvimagesink.c(1773): gst_xv_image_sink_open (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
    Could not open display (null)
    Setting pipeline to NULL ...
    Freeing pipeline ...

I am running on 32.1.0, would upgrading help? The latest drivers for my camera are only 32.3.1.

# R32 (release), REVISION: 1.0, GCID: 14531094, BOARD: t186ref, EABI: aarch64, DATE: Wed Mar 13 07:41:08 UTC 2019

Cheers!

2

I can’t say for VisionWorks, but for gstreamer I may help.
Please send the output of (assuming your camera is /dev/video0):

sudo v4l2-ctl -d0 --list-formats-ext

for better advice. (You may have to install v4l-utils package through APT)

3

Thanks for the reply:

sudo v4l2-ctl -d0 --list-formats-ext
[sudo] password for chris:
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'BA12'
        Name        : 12-bit Bayer GRGR/BGBG
                Size: Discrete 1928x1208
                        Interval: Discrete 0.033s (30.000 fps)

Also when i run using v4l2 it just gives me this output then nothing:

v4l2-ctl -V --set-fmt-video=width=1928,height=1208,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=3 --stream-to=ar0231-dual.raw -d /dev/video0

Format Video Capture:
        Width/Height      : 1928/1208
        Pixel Format      : 'RG12'
        Field             : None
        Bytes per Line    : 3856
        Size Image        : 4658048
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :
4

Sorry, I have no experience with 12 bits bayer sensors going through nvarguscamerasrc.
Did you get all the SDK files including ISP config for that sensor? You may check with your sensor provider.
Can you decode the 3 bayer frames you’ve recorded ?
Probably someone else may also better advice.

5

No worries, thanks for you help anyways.
It’s frustrating as it works perfectly fine on the device when I plug in the HDMI monitor, it’s only over SSH this happens.
Cheers!

6

If it works fine with a locally attached monitor and just fails as remote X, you may try to unset DISPLAY in case it would be set. Not tried myself, but some user told this once.

7

Tried that, as well as redoing all my xhost permissions. Thanks again

8

Where @Honey_Patouceul had mentioned this:

sudo v4l2-ctl -d0 --list-formats-ext

…see how those formats differ when directly logged in to the Jetson via GUI, and how this changes when using “ssh -Y” or “ssh -X”. When you use forwarding of X events it implies much of the hardware and software is offloaded from the Jetson to the host PC. You might have everything you need on the Jetson, but if the host PC does not match this, then it could fail due to forwarding no longer running some of the content on the Jetson.

9

I’ll look into it, thanks.

10

You would check the caps used in local case with -v flag of gst-launch-1.0, and try to specify same caps from remote host if host display is able to do the same.