OpenCV camera w/TX1 works with C++ but not Python

Jary316 · August 4, 2016, 6:11pm

I am interested in reading the Jetson TX1 camera from the camera module that ships with the board. I have installed the latest release, R24.1 32 bits, CUDA 7 and manually installed OpenCV 3.1 with Python.

When I run the following excerpt of C++, the camera turns on:

VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

  if (!cap.isOpened()) {
    printf("Camera did not open\n");
    return -1;
  }

I am never hitting the if statement in this code block, but when running python (version 2.7.6) with the following code:

cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")

if not cap.isOpened():
  print("Could not open camera")
  sys.exit(1)

the if statement always triggers. The string passed to VideoCapture is identical, and I don’t think there are 2 versions of OpenCV on my machine. I am confused why the C++ implementation works but not the python implementation.

Just to verify in python:

>>> import cv2
>>> cv2.__version__
'3.1.0'

The linked .so files for C++ are also 3.1.0. Does anyone know why Python is not working please?

Thanks!

linuxdev · August 4, 2016, 8:47pm

I’m curious if permissions might be getting in the way. Have you tried with “sudo” for both programs?

Jary316 · August 4, 2016, 9:14pm

Thank you for the suggestion linuxdev, you may be on to something:

The C++ program opens the camera without sudo but does not with sudo.
The python script does not open the camera with or without sudo.

It did not help for the python script, but does indeed fail with sudo (surprisingly?) with the C++ program.

linuxdev · August 4, 2016, 10:14pm

I am thinking about the requirement to be in group “video”. One last permissions experiment…try both programs with “sudo -u ubuntu ”. The idea is to specifically make sure everything is forced to user ubuntu (known to be in group video)…it should already be that way if logged in as ubuntu, but this is not always “exactly” what really goes on. Though unlikely, there may be a difference between with and without “sudo -u ubuntu ”. How environment is inherited in a script may change slightly.

Jary316 · August 4, 2016, 10:41pm

I have tried both the python script and the C++ program with sudo -u ubuntu but both fail (the same as with plain sudo).

One thing that I did notice, when the C++ program works (without sudo), the following gets printed:

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0xf09208a10 isAohdr=0
2592 x 1458 FR=30.000000 CF=0xf09208a10 isAohdr=0
…

The same lines appear when calling gst-launch-1.0 nvcamerasrc (with or without sudo). Could the permission be causing issue with gstreamer or the nvcamerasrc plugin? I am not very familiar with how those work.

In terms of group, I believe I am logged in as ubuntu and also in the group video:

ubuntu@tegra-ubuntu:~/Workspace$ groups
ubuntu adm dialout sudo audio video
ubuntu@tegra-ubuntu:~/Workspace$ groups ubuntu
ubuntu : ubuntu adm dialout sudo audio video

Any idea please?

Jary316 · August 4, 2016, 10:45pm

Just for experiments, I checked which user python and sudo python showed up as:

ubuntu@tegra-ubuntu:~/Workspace$ python
...
>>> getpass.getuser()
'ubuntu'
ubuntu@tegra-ubuntu:~/Workspace$ sudo python
...
>>> getpass.getuser()
'root'

I added root to the group video:

ubuntu@tegra-ubuntu:~/Workspace$ groups root
root : root video

but sudo python script.py still fails to open the camera.

linuxdev · August 4, 2016, 10:56pm

Root wouldn’t need to be in group video for permissions. Looks like this isn’t the issue, but for trivia, if you were to look at environment under “sudo” for a system where root is actually allowed to log in, some environment will inherit from the user prior to becoming root; using “su -” instead would be purely root environment. The D-Bus message system itself sees some of the differences between running something “sudo -u” versus pure “su” versus running as self from a GUI login (which can for example cause differences in fonts displayed). The parts which are exported go on to effect shell commands differently from those parts not exported.

Sadly, the original question about Python-initiated CUDA versus C++ CUDA still goes unanswered.

Jary316 · August 4, 2016, 11:01pm

Thank you for the extra info linuxdev! Thank you for the help thus far.

I’ll report back if I make any progress and keep monitoring the thread. Thank you.

Jary316 · August 5, 2016, 1:23am

After a bit more digging, I am wondering if the issue leads with gstreamer, and here is why:

When setting logging with gstreamer (via export of GTS_DEBUG), my console spews when using my c++ program. When opening it with python, I see no log, even with the verbose setting.

Now I added the following code to open gstreamer myself:

import gst

pipeline = gst.Pipeline()
bin = gst.parse_bin_from_description("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink", False)

pipeline.add(bin)
pipeline.set_state(gst.STATE_PLAYING)

and it is now spewing a logs like the C++ program. My new error is with nvcamerasrc plugin not existing when opening this way: glib.GError: no element “nvcamerasrc”

Jary316 · August 5, 2016, 5:32am

I managed to get it working in Python! The sample of code above used gstreamer 0.1, which I had to replace with 1.0. In a gist, VideoCapture() doesn’t seem to be calling gstreamer, so I had to do it directly, like so:

pipeline = Gst.parse_launch("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)120/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink name=sink")

  appsink = pipeline.get_by_name("sink")

Then I could call emit ‘pull-sample’ on appsink. Fixed my issue, but it’s quite a bit slower as I have to convert the data for numpy / openCV.

JerryJia · December 21, 2016, 6:44am

Thanks for all the information from this thread, anyway, I just got my python works to get one frame from TX1 camera module.

The important thing to get it work is that we need install OpenCV 3.1.0, you can follow the official build method and replace python cv2.so lib to version 3.1.0. Original l4t OpenCV is 2.4.

And the other important thing is to use correct nvcamerasrc; try
cap = cv2.VideoCapture(“nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! nvtee ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM), format=(string)I420 ! nvoverlaysink -e ! appsink”)

It works on my TX1, just for sharing here.

pnambiar · July 6, 2017, 4:37pm

I have this script working with sudo…Is there a way to get it working without sudo? (>>> import cv2

cv2.version ‘3.2.0-dev’) I have already tried adding user to group video (gpasswd -a sidmeister video)

import sys
import cv2

def read_cam():
cap = cv2.VideoCapture(“nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink”)

if cap.isOpened():
    cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
    while True:
        ret_val, img = cap.read();
        cv2.imshow('demo',img)
        cv2.waitKey(10)
else:
 print "camera open fail"
 

cv2.destroyAllWindows()

WayneWWW · July 13, 2017, 9:29am

I don’t think it requires to be root to use this script. Do you edit it as a superuser?

pnambiar · July 13, 2017, 5:28pm

@WayneWWW
Initially I had to edit it as superuser. I changed permission of the folder and I don’t have to Sudo to edit the script.

Thanks,
-P

jkjung13 · October 21, 2017, 4:04am

OpenCV’s cv::VideoCapture() call actually allows a 2nd argument which could be used to specify “apiPreference”. In case of cv::VideoCapture() with a gstreamer pipeline, we’d like to specify “CAP_GSTREAMER” there.

The related OpenCV documentation could be found here:

https://docs.opencv.org/3.3.0/d8/dfe/classcv_1_1VideoCapture.html#a85b55cf6a4a50451367ba96b65218ba1
https://docs.opencv.org/3.3.0/d4/d15/group__videoio__flags__base.html#ga023786be1ee68a9105bf2e48c700294d

I also shared my python source code with the above-mentioned enhancement in the following blog post.

“How to Capture and Display Camera Video with Python on Jetson TX2”: https://jkjung-avt.github.io/tx2-camera-with-python/

def open_cam_onboard(width, height):
    # On versions of L4T previous to L4T 28.1, flip-method=2
    # Use Jetson onboard camera
    gst_str = "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! videoconvert ! appsink".format(width, height)
    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

jkjung13 · October 21, 2017, 4:06am