e-con Systems See3CAM_CU40 camera + Jetson

We are working on a computer vision related project. Our hardware platform is NVidia Jetson TX2 with two See3CAM_CU40 USB cameras connected.

We encountered a problem while trying to use V4L to get an image from the camera. We followed the instructions from the “See3CAMs on Jetson TK1 (ARM) over USB 3.0” page (https://www.e-consystems.com/Articles/Camera/USB-3-Camera-on-JetsonTK1-kit.asp), we successfully built and launched L4T 19.3.0 with reconfigured kernel with HID support, including USB 3.0 support in the bootloader. And we also tried the Grinch kernel version 19.3.8 with similar results.

The problem is in VIDIOC_STREAMON system call, that return -1 (with errno set to 16).
Same code running on x86 without any errors.

Camera initialization finishes with no problem:

int xioctl(int fd, int request, void *arg) {
    int r;
    do {
        r = ioctl (fd, request, arg);
    } while (-1 == r && EINTR == errno);

    return r;
}

void cameraInit(const std::string &device, int &deviceFile, void *&buffer){
    deviceFile = open(device.c_str(), O_RDWR);

    if(deviceFile == -1) {
        throw std::runtime_error("Couldn't open device file: " + device);
    } else {
        std::cout<<"Successfully open " + device<<std::endl;
    }

    v4l2_format format = {};
    format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    format.fmt.pix.width = grabber_width;
    format.fmt.pix.height = grabber_height;

    if(xioctl(deviceFile, VIDIOC_S_FMT, &format) == -1) {
        throw std::runtime_error("Setting frame format for camera " + device + " failed");
    } else {
        std::cout<<"Successfully set video capture format to "<<grabber_width<<" x "<<grabber_height<<std::endl;
    }

    v4l2_requestbuffers req_buf = {};
    req_buf.count = 1;
    req_buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    req_buf.memory = V4L2_MEMORY_MMAP;

    if(xioctl(deviceFile, VIDIOC_REQBUFS, &req_buf) == -1) {
        throw std::runtime_error("Requesting buffer for camera " + device + " failed");
    } else {
        std::cout<<"Successfully requested frame buffer"<<std::endl;
    }

    v4l2_buffer buf = {};
    buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    buf.memory = V4L2_MEMORY_MMAP;
    buf.index = 0;

    if(xioctl(deviceFile, VIDIOC_QUERYBUF, &buf) == -1) {
        throw std::runtime_error("Querying buffer from " + device + " failed. errno = " + std::to_string(errno));
    } else {
        std::cout<<"Successfully got buffer. Buffer length is "<<buf.length<<std::endl;
    }

    buffer = mmap(NULL, buf.length, PROT_READ | PROT_WRITE, MAP_SHARED, deviceFile, buf.m.offset);
    if(buffer == MAP_FAILED){
        throw std::runtime_error("Buffer mmaping failed (errno = " + std::to_string(errno) + ")");
    } else {
        std::cout<<"Successfully mmaped frame buffer"<<std::endl;
    }
}

Grabbing the frame throws an error on VIDIOC_STREAMON call with errno set to 16.

cv::Mat getFrame(int deviceFile, void *buffer) {
    v4l2_buffer buf = {};
    buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    buf.memory = V4L2_MEMORY_MMAP;
    buf.index = 0;

    if(xioctl(deviceFile, VIDIOC_QBUF, &buf) == -1) {
        throw std::runtime_error("Querying buffer failed. errno = " + std::to_string(errno));
    } else {
        std::cout<<"Buffer queried: OK "<<std::endl;
    }

    if(xioctl(deviceFile, VIDIOC_STREAMON, &buf.type) == -1) {// <--------- HERE
        throw std::runtime_error("Starting capture failed. errno = " + std::to_string(errno));
    } else {
        std::cout<<"Capture started: OK "<<std::endl;
    }

    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(deviceFile, &fds);
    struct timeval tv = {2, 0};
    int r = select(deviceFile + 1, &fds, NULL, NULL, &tv);

    if(r == -1){
        throw std::runtime_error("Waiting for frame failed. errno = " + std::to_string(errno));
    } else {
        std::cout<<"Frame captured: OK "<<std::endl;
    }

    if(xioctl(deviceFile, VIDIOC_DQBUF, &buf) == -1) {
        throw std::runtime_error("Retrieving frame failed. errno = " + std::to_string(errno));
    } else {
        std::cout<<"Frame retrieved: OK "<<std::endl;
    }

    cv::Mat input_image(grabber_height, grabber_width, CV_16UC1, buffer);
    cv::Mat temp_RGIR_image(grabber_height, grabber_width, CV_8UC1);

    //According to https://www.e-consystems.com/blog/camera/accessing-see3cam-custom-format-with-opencv/
    //10 bit pixels are placed into 16 bit values, just scaling down to 8 bit
    cv::convertScaleAbs(input_image, temp_RGIR_image, 255.0/1024);

    //simply pick one channel and scale down whole image
    return ConvertRGIR2Gray(temp_RGIR_image);
}

Tried on TK1 with same result, but works fine on x86. Is there something wrong with v4l on Jeston?

hello a.kazbekov,

if you got an VIDIOC_STREAMON system call failed, then you should check from the sensor driver side to ensure the sensor streaming.

could you please share more details,
for example,

  1. which JetPack release you’re working with.
  2. please access camera sensor with v4l2 standard controls, please also share the kernel messages if you got failed.
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
  1. Personally I’m never flashed this TX2. /etc/nv_tegra_release starts with:
# R28 (release), REVISION: 1.0, GCID: 9436269, BOARD: t186ref, EABI: aarch64, DATE: Fri Jul 28 17:04:29 UTC 2017
  1. v4l2-ctl gives me:
nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.09 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.04 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<
nvidia@tegra-ubuntu:~$

so, nothing wrong with camera, and the problem is in my code?

hello a.kazbekov,

v4l2 standard controls verified camera streaming works as expected.
please review your implementation,
thanks