Trying to use the NOIR raspberry pi camera with my jetson nano

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hello, I’m new to the jetson and I have a NOIR camera but I don’t know how to get it to work. Every tutorial I see uses the normal v2 camera but I can’t get it to work and I can’t find tutorials or pages which work with the NOIR.

2

You would check what sensor your camera has. If it is a RPi v2 cam with IMX219 sensor, it would just work as the standard version with IR filter. The software on Jeston has no knowledge about the IR filter being removed or not from the sensor. Connect your cam to CSI0 connector of devkit while it is not powered, boot your jetson and you should be able to run:

gst-launch-1.0 nvarguscamerasrc ! nvoverlaysink

for displaying your camera feed into monitor.

You may get some pink image in low lightening conditions, but if using a NOIR sensor you would have an IR light source for dark and would manage your image mainly from red.
The hard thing would be dealing with ISP settings that are closed source and need a Jetson partner if you want to use it with Argus. You can also get frames from V4L, but it would be bayer RG10, you would have to manage demosaicing by yourself.

3

You probably just want to filter the output to grayscale

496×745 70.4 KB

4

For testing if converting to greyscale as proposed by @Nick_H does fit your need, you would try:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! 'video/x-raw(memory:NVMM), format=GRAY8' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420' ! nvoverlaysink

# or if you have GUI running :
gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! 'video/x-raw, format=GRAY8' ! videoconvert ! xvimagesink

@2024a, is it expected that:

  • nvoverlaysink fails to display GRAY8 from NVMM memory (nor standard memory, gst-inspect-1.0 nvoverlaysink says no format restriction).
  • the first pipeline works with NV12->GRAY8->I420 but fails with NV12->GRAY8->NV12.
  • the second pipeline requires videoconvert
  • closing the window of xvimagesink results in seg fault (this may not be jetson specific).
5

When I run the command you gave me, it displays this:

Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:656 Failed to create CaptureSession

(gst-launch-1.0:8039): GStreamer-CRITICAL **: 12:15:20.978: gst_mini_object_set_qdata: assertion ‘object != NULL’ failed
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.011564518
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

6

Do you have a monitor connected to Jetson or are running headless ?

If you have a local display, be sure that no other process is using camera. Suggest to retry after a fresh boot.
Be sure you have a /dev/video0 node created by the IMX219 driver. If not, the camera has failed to be probed by driver.

If running headless, nvoverlaysink wouldn’t work.
You would connect from host to Jetson with ssh -X and in Jetson use:

gst-launch-1.0 nvarguscamerasrc ! nvvidconv ! xvimagesink