Opencv nvarguscamerasrc: hard LOCKUP on cpu 0

I am currently writing a script to do some image processing in opencv, but i have gotten quite stuck.

Im using a Jetson Nano 4GB developer kit, and a raspberry pi camera 2 connected to the CSI port.
but whenever i try to open a video capture with opencv, the system freezes, and reboots.

when i run “nvgstcapture-1.0” everything works fine.

i’m not quite sure about the nvarguscamerasrc pipeline, i got it from an example by JetsonHacks.

example code

#include <string>
#include <opencv2/opencv.hpp>

std::string gstreamer_pipeline(int sensor_id, int sensor_mode, int framerate, int flip_method, int display_width, int display_height) {
    return "nvarguscamerasrc sensor-id=" + std::to_string(sensor_id) + " sensor-mode=" + std::to_string(sensor_mode) + " ! video/x-raw(memory:NVMM), format=(string)NV12, framerate=(fraction)" + std::to_string(framerate) + "/1 ! nvvidconv flip-method=" + std::to_string(flip_method) + " ! video/x-raw, width=(int)" + std::to_string(display_width) + ", height=(int)" + std::to_string(display_height) + ", format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink";
}

int main(int, char**) {
    auto pipeline = gstreamer_pipeline(
        0,
        3,
        30,
        0,
        1280,
        720
    );
    auto capture = new cv::VideoCapture(pipeline, cv::CAP_GSTREAMER);

    return 0;
}

output:

GST_ARGUS: Creating output stream

CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 3 
   Output Stream W = 1280 H = 720 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

then it wait a few seconds and everything freezes

[  624.351579] Kernel panic - not syncing: Watchdog detected hard LOCKUP on cpu 0
[  624.358793] CPU: 3 PID: 0 Comm: swapper/3 Not tainted 4.9.201-tegra #1
[  624.365305] Hardware name: NVIDIA Jetson Nano Developer Kit (DT)
[  624.371294] Call trace:
[  624.373737] [<ffffff800808b9f8>] dump_backtrace+0x0/0x198
[  624.379123] [<ffffff800808bfbc>] show_stack+0x24/0x30
[  624.384161] [<ffffff800845abe8>] dump_stack+0xa0/0xc8
[  624.389199] [<ffffff80081c0a00>] panic+0x12c/0x2a8
[  624.393980] [<ffffff800818169c>] watchdog_check_hardlockup_other_cpu+0x11c/0x120
[  624.401358] [<ffffff8008180810>] watchdog_timer_fn+0x98/0x2c0
[  624.407090] [<ffffff8008138b60>] __hrtimer_run_queues+0xd8/0x360
[  624.413081] [<ffffff80081394b0>] hrtimer_interrupt+0xa8/0x1e0
[  624.418814] [<ffffff8008c00760>] tegra210_timer_isr+0x38/0x48
[  624.424546] [<ffffff8008121610>] __handle_irq_event_percpu+0x68/0x288
[  624.430970] [<ffffff8008121858>] handle_irq_event_percpu+0x28/0x60
[  624.437134] [<ffffff80081218e0>] handle_irq_event+0x50/0x80
[  624.442693] [<ffffff80081256f4>] handle_fasteoi_irq+0xd4/0x1c0
[  624.448510] [<ffffff80081205c4>] generic_handle_irq+0x34/0x50
[  624.454241] [<ffffff8008120cb0>] __handle_domain_irq+0x68/0xc0
[  624.460058] [<ffffff8008080d44>] gic_handle_irq+0x5c/0xb0
[  624.465442] [<ffffff8008082c28>] el1_irq+0xe8/0x194
[  624.470308] [<ffffff8008ba6948>] cpuidle_enter_state+0xb8/0x380
[  624.476213] [<ffffff8008ba6c84>] cpuidle_enter+0x34/0x48
[  624.481511] [<ffffff800811104c>] call_cpuidle+0x44/0x70
[  624.486722] [<ffffff80081113c8>] cpu_startup_entry+0x1b0/0x200
[  624.492540] [<ffffff8008091cc8>] secondary_start_kernel+0x190/0x1f8
[  624.498790] [<0000000084f671a4>] 0x84f671a4
[  624.502963] SMP: stopping secondary CPUs
[  625.571863] SMP: failed to stop secondary CPUs 0,3
[  625.576644] Kernel Offset: disabled
[  625.580123] Memory Limit: none
[  625.590300] Rebooting in 5 seconds..
[  630.594165] SMP: stopping secondary CPUs
[  631.663157] SMP: failed to stop secondary CPUs 0,3
[0000.159] [L4T TegraBoot] (version 00.00.2018.01-l4t-e82258de)

reboot:

[0000.164] Processing in cold boot mode Bootloader 2
[0000.169] A02 Bootrom Patch rev = 1023
[0000.173] Power-up reason: software reset
[0000.176] No Battery Present
[0000.179] pmic max77620 reset reason
[0000.182] pmic max77620 NVERC : 0x0
[0000.186] RamCode = 0
[0000.188] Platform has DDR4 type RAM
[0000.191] max77620 disabling SD1 Remote Sense
[0000.195] Setting DDR voltage to 1125mv
[0000.199] Serial Number of Pmic Max77663: 0xd08b2
[0000.207] Entering ramdump check
[0000.210] Get RamDumpCarveOut = 0x0
[0000.213] RamDumpCarveOut=0x0,  RamDumperFlag=0xe59ff3f8
[0000.218] Last reboot was clean, booting normally!
[0000.223] Sdram initialization is successful 
[0000.227] SecureOs Carveout Base=0x00000000ff800000 Size=0x00800000
[0000.233] Lp0 Carveout Base=0x00000000ff780000 Size=0x00001000
[0000.239] BpmpFw Carveout Base=0x00000000ff700000 Size=0x00080000
[0000.245] GSC1 Carveout Base=0x00000000ff600000 Size=0x00100000
[0000.251] GSC2 Carveout Base=0x00000000ff500000 Size=0x00100000
[0000.257] GSC4 Carveout Base=0x00000000ff400000 Size=0x00100000
[0000.262] GSC5 Carveout Base=0x00000000ff300000 Size=0x00100000
[0000.268] GSC3 Carveout Base=0x000000017f300000 Size=0x00d00000
[0000.284] RamDump Carveout Base=0x00000000ff280000 Size=0x00080000
[0000.290] Platform-DebugCarveout: 0

thank you for your help

Hi,
There are two python samples to demonstrate nvarguscamerasrc, please check if you can run these successfully:
[Gstreamer] nvvidconv, BGR as INPUT - #4 by DaneLLL
OpenCV Video Capture with GStreamer doesn't work on ROS-melodic - #3 by DaneLLL

Thank you for your answer :)

The examples both work

but when i add in the sensor-mode argument it craches again

Hi,
It looks similar to
OpenCV ops blocking for multi-camera capture with Gstreamer - #14 by DaneLLL

We suggest run 30fps case in OpenCV. For high frame rate case, suggest run gst-launch-1.0 command.

Another issue, now the image looks like this:
example

nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)I420 ! appsink

the image uploaded here has been scaled down to 640x360, but it also happens on the original image.

You are displaying the I420 frame from opencv, but opencv imshow interprets the 1-channel image as monochrome.
For color, you would capture as BGR frames with:

nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink
1 Like

Thank you
Does nvarguscamerasrc have any documentation? I haven’t been able to find much.

Where can I find the available formats?

Is it possible to just capture in single channel, black/white? That’s all I need

The documentation is embedded into the plugin, you can use gst-inspect:

gst-inspect-1.0 nvarguscamerasrc

This will list the SRC caps (ie outputs available formats).
It should provide NV12 only, but if you just want a monochrome image into opencv, you would use nvvidconv for converting to GRAY8:

nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=GRAY8 ! appsink