Can not open the camera on TX1 via OpenCV

Hi,
I am trying to open the camera on Jetson TX1 developmen kit via OpenCV and I can compile the code.But when I run it,I got nothing.someone said “OpenCV is not using Gstreamer path and pipeline string is being interpreted as V4L2 device”,but I do not konow how to do it.The code is like that:

#include <stdio.h>
#include <opencv2/opencv.hpp>

using namespace cv;
using namespace std;

int main(int argc, char** argv )
{
        VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), 
width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 
! video/x-raw, format=(string)BGRx ! videoconvert ! 'video/x-raw, format=(string)BGR' ! appsink"); // open the camera

    if(!cap.isOpened()){ 
 // check if we succeeded
        return -1;
}

    Mat edges;
    namedWindow("edges",1);
    for(;;)
    {
        Mat frame;
        cap >> frame; // get a new frame from camera
        cvtColor(frame, edges, COLOR_BGR2GRAY);
        GaussianBlur(edges, edges, Size(7,7), 1.5, 1.5);
        Canny(edges, edges, 0, 30, 3);
        imshow("edges", edges);
       // if(waitKey(30) >= 0) break;
    }
    return 0;
}

I have got the code from the website:https://devtalk.nvidia.com/default/topic/904949/how-to-get-tx1-camera-in-opencv/
Any suggestion will be appreciated,
Thanks

Hi Jackwang,

It seems geoffreywall could make it work, see https://devtalk.nvidia.com/default/topic/904949/jetson-tx1/how-to-get-tx1-camera-in-opencv/post/4787122/#4787122

Please change your code:

VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), 
width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 
! video/x-raw, format=(string)BGRx ! videoconvert ! 'video/x-raw, format=(string)BGR' ! appsink"); // open the camera

To following:

char* gst = "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! videoconvert ! appsink";

    VideoCapture cap(gst);

And see if it can work.

Thanks

Hi kayccc,
Thanks for your response.I had modified my code as what you said.But It did not work.I got nothing just as before.By the way.I have flashed my board with tegra 23.2.

I followed this thread’s instructions and had to add in a

if (!frame.empty())
{
    continue;
}

in the for(;;) to keep it from throwing scn == 3 || scn == 4 errors.

Now it just hangs in an infinite loop, waiting for an image to be passed (it hangs because all images being passed are empty)

How do I fix it?

EDIT

The terminal says that there is an error near unexpected token ‘(’ in this command:

nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! videoconvert ! appsink

when run from the command line

Hi,

You may want to try this:

VideoCapture capture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

I took the code from https://devtalk.nvidia.com/default/topic/904949/jetson-tx1/how-to-get-tx1-camera-in-opencv/post/4823344/#4823344
It works for me. However, I lowered the resolution to avoid delay.

Thanks.

-Leow

Well, I’m currently getting stuck with this same situation even after updating JetPack to 2.1.
I’m trying to use the embedded CSI sensor with OpenCV.

After I could successfully run the pre-installed application ‘nvgstcapture-1.0’ with an option ‘-m 2’, and then type ‘gcdn’, it returns

‘/dev/video0’ but actually there is no ‘video0’ under ‘dev’ folder(only ‘nvgstcapture-1.0’ works, ‘nvgstcapture’ and ‘nvgstcapture-0.1’

don’t work, I don’t know why).

I tried to apply the newly released patch provided by dusty_nv(https://devtalk.nvidia.com/default/topic/920739/jetson-tx1/l4t-r23-2-

for-jetson-tx1-released/post/4824232/#4824232), but I think JetPack 2.1 already provides drivers or kernel sources that the patch is

applied.

Then I also tried to apply GST_EXPORT := 3, but still same situation.

I thought it would be better to use “nvcamerasrc” pipeline instead of the default OpenCV cam value(-1 or 0).
So I tried to use the above line

VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)

24/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! ‘video/x-raw, format=(string)BGR’ ! appsink");

or just cap(0), cap(-1 ) or cap(1) but it always fails to find the camera.

Is it real that everybody doen not see this problem anymore and only me is the one facing this embarrassing situation? do I need to

remove the exisitng Opencv4Tegra and install the common openCV 2.4 as geoffreyall did?

Please if someone has any advice, let me know it.

Hi SeunghyunLee,

Actually, the OpenCV4Tegra is not well optimized to support camera on Jetson TX1 at present version to cause the video capturing (live streaming) problem.
We’re working this issue, and going to have an update version in coming release soon.

Instead of using OpenCV4Tegra, you could try to remove the exisitng Opencv4Tegra and install the common openCV 2.4 as geoffreyall did or to compile and install OpenCV 3.1.0 by referring the below thread:
https://devtalk.nvidia.com/default/topic/917386/jetson-tx1/usb-3-0-port-unstable-on-jetson-tx1-/post/4835793/#4835793

This should work as temporary solution on your case.

Thanks

I’m having this issue as well. I’m working on a time critical demo. The camera works fine with the v4l-2 interface but openCV fails sometimes with error, sometimes without error.

Can you provide a time line on the fix to the NVidia optimized openCV? I need to figure out if I’m going to have to use lib4vl for my demo.

Thanks,

Brian

I am a newer TX1.I am trying to use a USB camera on tx1 boardwith opencv2.4.10. Here is the c++ code.

int main()
{

//char* gst = "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink";

  //  VideoCapture cap(gst);

//VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");
VideoCapture cap(0);

Mat frame;
if(cap.isOpened())
{cout<<"1"<<endl;
while(1)
{
namedWindow("1",1);
cap>>frame;
imshow("1",frame);
waitKey(30);

}

}

return 0;

}

and there is the error:
(DEBUG) try_init_v4l2 VIDIOC_ENUMINPUT “/dev/video0”: Inappropriate ioctl for device
Unable to stop the stream.: Device or resource busy
Unable to stop the stream.: Bad file descriptor
1
VIDIOC_STREAMON: Inappropriate ioctl for device

I also try with gstreamer and nvgstcapture-1.0-sample,

gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvoverlaysink -e

Also the error:
Invalid FPSRange Input
Setting pipeline to PAUSED …
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingSocket read error. Camera Daemon stopped functioning…
gst_nvcamera_open() failed ret=0
ERROR: Pipeline doesn’t want to pause.
Setting pipeline to NULL …
Freeing pipeline …

And still cannot get the stream from camera,please help me!
Thanks!

I am a newer TX1.I am trying to use a USB camera on tx1 boardwith opencv2.4.10. Here is the c++ code.

int main()
{

//char* gst = "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink";

  //  VideoCapture cap(gst);

//VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)320, height=(int)240, format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");
VideoCapture cap(0);

Mat frame;
if(cap.isOpened())
{cout<<"1"<<endl;
while(1)
{
namedWindow("1",1);
cap>>frame;
imshow("1",frame);
waitKey(30);

}

}

return 0;

}

and there is the error:
(DEBUG) try_init_v4l2 VIDIOC_ENUMINPUT “/dev/video0”: Inappropriate ioctl for device
Unable to stop the stream.: Device or resource busy
Unable to stop the stream.: Bad file descriptor
1
VIDIOC_STREAMON: Inappropriate ioctl for device

I also try with gstreamer and nvgstcapture-1.0-sample,

gst-launch-1.0 nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvvidconv flip-method=2 ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvoverlaysink -e

Also the error:
Invalid FPSRange Input
Setting pipeline to PAUSED …
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingSocket read error. Camera Daemon stopped functioning…
gst_nvcamera_open() failed ret=0
ERROR: Pipeline doesn’t want to pause.
Setting pipeline to NULL …
Freeing pipeline …

And still cannot get the stream from camera,please help me!
Thanks!

same here, could nvidia really fix this annoying thing? We already wait for 3 months! NV keeps saying there will be an upgrade, there will be an upgrade, now where is it? 24.2 got even more bug! Seriously, we all hope NV could comes up with some new system, or release a step-by-step tutorial about how to load default Ubuntu and how to install those CUDA and things like them, how to set up so we can optimized CPU and GPU power.

Hi Gary_4001,

Regarding the issue that met, we ave answered in another topic 961154:
https://devtalk.nvidia.com/default/topic/961154/problem-with-tx1-usb-camera-/?offset=1#4963564

The USB camera should work with L4T 24.1 + OpenCV4Tegra 2.4.13.

About the onboard camera, assuming you’re using L4T 24.1, installing OpenCV 3.1.0 may work around this issue. See this thread: https://devtalk.nvidia.com/default/topic/943129/jetson-tx1/highgui-error-v4l-v4l2-while-opening-camera-with-opencv4tegra-l4t-r24/post/4921383/#4921383

Would you please file another topic with specific issue you met?

Thanks