I am using the following app: https://github.com/naisy/realtime_object_detection
The interesting part is, it works on my Xavier but doesn’t work on the TX2. I think this is because of the specific open CV install and maybe the g-streamer compatibility?
According to FLIR this is what they say about the USB webcam:
Boson is capable of providing digital data as a USB Video Class (UVC) compliant device. Two output options are provided. Note the options are not selected via the CCI but rather by the video capture or viewing software selected by the user. The options are:
■ Pre-AGC (16-bit): The output is linearly proportional to the flux incident on each pixel in the array; output resolution is 320x256 for the 320 configuration, 640x512 for the 640 configuration. Note that AGC settings, zoom settings, and color-encoding settings have no effect on the output signal at this tap point. This option is identified with a UVC video format 4CC code of “Y16 ” (16-bit uncompressed greyscale image)
■ Post-Colorize, YCbCrb: The output is transformed to YCbCr color space using the specified color palette (see Section 6.7). Resolution is 640x512 for both the 320 and 640 configurations. Three options are provided, identified via the UVC video format 4CC code:
• I420: 8 bit Y plane followed by 8 bit 2x2 subsampled U and V planes
• NV12: 8-bit Y plane followed by an interleaved U/V plane with 2x2 subsampling
• NV21: same as NV12 except reverse order of U and V planes
I am guessing I need to make sure that one of the three 4CC codes are compatible, I just have no idea where to look?
Looking at what is below I am guessing open CV is trying to open the 16bit monochrome. Googling the error it seems like most people were working with monochrome images, so I guess I am looking for a way to force it to use the color version?
https://www.flir.com/support-center/oem/is-there-a-way-to-maximize-the-video-display-on-the-boson-app-for-windows-pc-to-full-screen/
When I would use a simple open CV program to display the webcam it would give me the same error, but then display the video, my guess is it tries the monochrome gets and error then moves onto the color version. I am looking for a way for open CV to not fail when it gets the wrong format.
When I run the code below prior to installing pyyaml it just doesn’t work and says:
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp, line 890
VIDEOIO(cvCreateCapture_GStreamer(CV_CAP_GSTREAMER_V4L2, reinterpret_cast<char *>(index))): raised OpenCV exception:
/home/nvidia/src/opencv-3.4.0/modules/videoio/src/cap_gstreamer.cpp:890: error: (-2) GStreamer: unable to start pipeline
in function cvCaptureFromCAM_GStreamer
OpenCV Error: Assertion failed (scn == 3 || scn == 4) in cvtColor, file /home/nvidia/src/opencv-3.4.0/modules/imgproc/src/color.cpp, line 11111
Traceback (most recent call last):
File "test.py", line 11, in <module>
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cv2.error: /home/nvidia/src/opencv-3.4.0/modules/imgproc/src/color.cpp:11111: error: (-215) scn == 3 || scn == 4 in function cvtColor
But after it gives the same errors as above but then displays an image.
Open CV webcam display code
import numpy as np
import cv2
cap = cv2.VideoCapture(0)
while(True):
# Capture frame-by-frame
ret, frame = cap.read()
# Our operations on the frame come here
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
# Display the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
Using that same code I can not get the FLIR to display on the Xavier, but it does work with the above program.