Posenet not connecting to builtin camera

Hi All,

We’re proof-concepting TX2 & Posenet, and I’m running into a few problems, depending on the camera I’m using.

  1. USB camera is OK, but slow framerate (6 FPS) and high CPU usage (40%),
    but this is off-topic here.

  2. posenet returns failure when attempting to read the stream from the builtin camera

This is the error logged by posenet:

01/20/2020 09:33:17 am 93.64.54.53: ezio - [ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Cannot identify device ‘/dev/video0’.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
VIDEOIO ERROR: V4L: can’t open camera by index 0
Traceback (most recent call last):
File “csv-demo.py”, line 101, in
main()
File “csv-demo.py”, line 59, in main
cap, scale_factor=args.scale_factor, output_stride=output_stride)
File “/home/ezio/tensorflow/GIT/posenet/posenet/utils.py”, line 28, in read_cap
raise IOError(“webcam failure”)
OSError: webcam failure

Let me add the following:

  • Camera is OK with nvgstcapture
  • OS: Linux JTX2 4.9.140-tegra
  • JetPack 4.3
  • OpenCV 4.11

How can I fix the problem?
Thank you for your attention,

                  Ezio

I don’t think v4l2src0 can open build-in raw bayer sensor.
You can reference to below link to use nvarguscamerasrc to open build-in camera.

https://github.com/JetsonHacksNano/CSI-Camera

Thank you Shane for your reply.

I don’t think v4l2src0 can open build-in raw bayer sensor.

Ouch! Do you mean OpenCV cannot open the builtin camera?

So, I can use posenet only for USB cameras?

Or, should I hack the posenet/OpenCV code to use this? https://github.com/JetsonHacksNano/CSI-Camera

Ciao,

            Ezio

If you need to open on board CSI camera you reference to the link to use the correct gstreamer pipeline to open it via openCV.

Hi Shane, thank you for your reply.

Well, this command, that you reported in the reply works like a charm:

gst-launch-1.0 nvarguscamerasrc ! ‘video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=960, height=616’ ! nvvidconv ! nvegltransform ! nveglglessink -e

So, we hacked it into our code, I’m pasting the snippet at the end …

Camera is recognized, and this is a great improvement, but we get a nice coredump:

2020-01-21 16:43:39.691479: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1304] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3598 MB memory) -> physical GPU (device: 0, name: NVIDIA Tegra X2, pci bus id: 0000:00:00.0, compute capability: 6.2)
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2592 x 1944 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 2
Output Stream W = 1280 H = 720
seconds to Run = 0
Frame Rate = 120.000005
GST_ARGUS: PowerService: requested_clock_Hz=24192000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
Segmentation fault (core dumped)

heres’ the code snippet:

def gstreamer_pipeline(
capture_width=1280,
capture_height=720,
display_width=1280,
display_height=720,
framerate=60,
flip_method=0,
):
return (
"nvarguscamerasrc ! "
"video/x-raw(memory:NVMM), "
"width=(int)%d, height=(int)%d, "
"format=(string)NV12, framerate=(fraction)%d/1 ! "
"nvvidconv flip-method=%d ! "
"video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! "
"videoconvert ! "
“video/x-raw, format=(string)BGR ! appsink”
% (
capture_width,
capture_height,
framerate,
flip_method,
display_width,
display_height,
)
)

def main():
with tf.Session() as sess:
model_cfg, model_outputs = posenet.load_model(args.model, sess)
output_stride = model_cfg[‘output_stride’]

            if args.csicamera>0:
                    cap = cv2.VideoCapture(gstreamer_pipeline(capture_width=args.cam_width, capture_height=args.cam_height), cv2.CAP_GSTREAMER)