I created a library for the nvivafilter plugin to realtine remap a distorted image from a camera with a wide-angle lens. It works sucessfully via gst-launch:
gst-launch-1.0 nvarguscamerasrc sensor-mode=0 ! "video/x-raw(memory:NVMM), width=(int)3264, height=(int)2464, format=(string)NV12, framerate=(fraction)21/1" ! nvvidconv ! nvivafilter customer-lib-name=./gst-remap_cudaprocess.so cuda-process=true ! 'video/x-raw(memory:NVMM), format=RGBA, width=3264, height=2464' ! nvegltransform ! nveglglessink
Now I want to use it in the OpenCV VideoCaputure. My original VideoCapture instantiation is:
source = "nvarguscamerasrc sensor_mode=0 ! video/x-raw(memory:NVMM), width=(int)3264, height=(int)2464, format=(string)NV12, framerate=(fraction)21/1 ! nvvidconv flip-method=2 ! video/x-raw, width=(int)3264, height=(int)2464, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink"
cap = cv2.VideoCapture(source, cv2.CAP_ANY)
How should I transform it by adding the nvivafilter plugin? I made various attempts with no success. This is the last one:
source = "nvarguscamerasrc sensor_mode=0 ! video/x-raw(memory:NVMM), width=(int)3264, height=(int)2464, format=(string)NV12, framerate=(fraction)21/1 ! nvvidconv flip-method=2 ! nvivafilter customer-lib-name=./gst-remap_cudaprocess.so cuda-process=true ! video/x-raw(memory:NVMM), format=RGBA, width=(int)3264, height=(int)2464 ! appsink"
cap = cv2.VideoCapture(source, cv2.CAP_ANY)
Output:
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1640 x 1232 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 3264 H = 2464
seconds to Run = 0
Frame Rate = 21.000000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
(python:11337): GStreamer-CRITICAL **: 14:51:47.310:
Trying to dispose element capsfilter1, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.
(python:11337): GStreamer-CRITICAL **: 14:51:47.311:
Trying to dispose element capsfilter0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.
(python:11337): GStreamer-CRITICAL **: 14:51:47.311:
Trying to dispose element nvivafilter0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.
(python:11337): GStreamer-CRITICAL **: 14:51:47.313:
Trying to dispose element pipeline0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.
[ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /tmp/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
(python:11337): GStreamer-CRITICAL **: 14:51:47.314:
Trying to dispose element appsink0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.
CRITICAL:root:Error opening input video: nvarguscamerasrc sensor_mode=0 ! video/x-raw(memory:NVMM), width=(int)3264, height=(int)2464, format=(string)NV12, framerate=(fraction)21/1 ! nvvidconv flip-method=2 ! nvivafilter customer-lib-name=./gst-remap_cudaprocess.so cuda-process=true ! video/x-raw(memory:NVMM), format=RGBA, width=(int)3264, height=(int)2464 ! appsink
(python:11337): GStreamer-CRITICAL **: 14:51:47.327: gst_element_post_message: assertion 'GST_IS_ELEMENT (element)' failed
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
CONSUMER: Done Success
WARNING Argus: 6 client objects still exist during shutdown:
548301591968 (0x7f00000c70)
548301592128 (0x7f00001500)
548301592320 (0x7f000013f0)
548301597424 (0x7f00003500)
548301605448 (0x7f00003618)
548301781760 (0x7f00000ee0)