Problemes IMX219 Camera + OpenCV

Hello,

i’m new with the Jetson Nano. First my hardware/software:

Camera: Waveshare IMX219-160IR Camera Supports NVIDIA Jetson Nano Developer Kit 8 Megapixels Infrared Night Vision 160° FOV Suits for AI Projects

Open CV: 4.1

I want to capture the Videostream from the camera with open cv. My code for the camera is the follwing.

cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)3280, height=(int)2464,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")

But if i start the application, i get follwing error:

[INFO] Computer Vision wird gestartet
[INFO] loading facial landmark predictor…
[INFO] starting video stream thread…
^C
(python3:10301): GStreamer-CRITICAL **: 12:08:56.710:
Trying to dispose element capsfilter2, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

Traceback (most recent call last):
File “schnittstelle.py”, line 27, in

(python3:10301): GStreamer-CRITICAL **: 12:08:56.710:
Trying to dispose element capsfilter1, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python3:10301): GStreamer-CRITICAL **: 12:08:56.711:
Trying to dispose element capsfilter0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python3:10301): GStreamer-CRITICAL **: 12:08:56.711:
Trying to dispose element nvvconv0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python3:10301): GStreamer-CRITICAL **: 12:08:56.711:
Trying to dispose element nvarguscamerasrc0, but it is in PLAYING instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python3:10301): GStreamer-CRITICAL **: 12:08:56.711:
Trying to dispose element pipeline0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

ergebnis_cv = queue_CV.get()

File “/usr/lib/python3.6/multiprocessing/queues.py”, line 94, in get
Process Process-1:
res = self._recv_bytes()
File “/usr/lib/python3.6/multiprocessing/connection.py”, line 216, in recv_bytes
buf = self._recv_bytes(maxlength)
File “/usr/lib/python3.6/multiprocessing/connection.py”, line 407, in _recv_bytes
buf = self._recv(4)
File “/usr/lib/python3.6/multiprocessing/connection.py”, line 379, in _recv
chunk = read(handle, remaining)
KeyboardInterrupt
Traceback (most recent call last):
File “/usr/lib/python3.6/multiprocessing/process.py”, line 258, in _bootstrap
self.run()
File “/usr/lib/python3.6/multiprocessing/process.py”, line 93, in run
self._target(*self._args, **self._kwargs)
File “/home/mufade2/Dokumente/Code/Drowiness_Detection_CV/Code_Drowiness_CV.py”, line 71, in drowiness_detection_CV
cap = cv2.VideoCapture(“nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)3280, height=(int)2464,format=(string)NV12, framerate=(fraction)20/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink”)
KeyboardInterrupt

Does someone know, why i get this error?

Best regards

Could you try to use the pipeline on gst-launch first instead of directly put it in opencv?

Hi WayneWWW,

i used this to use the pipline on gst-launch (directly from Waveshare).

DISPLAY=:0.0 gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM), width=3280, height=2464, format=(string)NV12, framerate=(fraction)20/1' ! nvoverlaysink -e

I get this output:

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21,000000 fps Duration = 47619048 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28,000001 fps Duration = 35714284 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29,999999 fps Duration = 33333334 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59,999999 fps Duration = 16666667 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120,000005 fps Duration = 8333333 ; Analog Gain range min 1,000000, max 10,625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 3264 H = 2464
seconds to Run = 0
Frame Rate = 21,000000
GST_ARGUS: PowerService: requested_clock_Hz=35358400
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

And then nothing happens.

Sry for my bad english. I’m from germany ;)

I tried it now a seconds time, the output was now:

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
(Argus) Error Timeout: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function openSocketConnection(), line 215)
(Argus) Error Timeout: Cannot create camera provider (in src/rpc/socket/client/SocketClientDispatch.cpp, function createCameraProvider(), line 102)
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:515 Failed to create CameraProvider
Got EOS from element “pipeline0”.
Execution ended after 0:01:00.072032420
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

hello sven_hendrik.vennebusch,

according to the sensor capability, the maximum resolution is 3264x2464@21-fps.
could you please revise your gst commands to match the same configuration.

also,
could you please share which JetPack release you’re working with.
thanks

Hi JerrsyChang,

I tried it and it happend the same.

My Jetpack Version:

  • NVIDIA Jetson NANO/TX1
    • Jetpack 4.2.1 [L4T 32.2.0]
    • CUDA GPU architecture 5.3
  • Libraries:
    • CUDA 10.0.326
    • cuDNN 7.5.0.56-1+cuda10.0
    • TensorRT 5.1.6.1-1+cuda10.0
    • Visionworks 1.6.0.500n
    • OpenCV 4.1.0 compiled CUDA: YES
  • Jetson Performance: inactive

Thank you very much.

hello sven_hendrik.vennebusch,

according to your messages in comment#3, assume your sensor already enable for sending streaming.

GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

am also not sure what had you done between 1st and 2nd trials.
could you please have alternative way to enable camera sensor for capture a jpeg images.
for example,

$ gst-launch-1.0 nvarguscamerasrc num-buffers=1 ! 'video/x-raw(memory:NVMM), width=3264, height=2464' ! nvjpegenc ! filesink location=sample.jpg

Hi JerryChang,

The jpeg image was empty.

The different between the 1st and 2nd trial was that it was the second try to open the stream.
The output from #3 i only get after restart the Jetson Nano.
Also i get the outout from #3 if there is no camera connected? I think thats not normal…

hello sven_hendrik.vennebusch,

please also check similar issue, Topic 1050743.
you might also check these modules using IMX219 that works with Jetson-Nano, https://github.com/NVIDIA-AI-IOT/jetbot/issues/29
thanks