Hello,
Just to provide an update, I did resolve the issue preventing me from collecting raw images. Specifically, the impact of the null pointer exceptions.
It appears there is an issue caused by the vi_channel_error_recover function in vi5_fops.c even as of R35.5.0. Whenever the CSI subdev is unable to close a stream due to the channel pointer becoming null, the channel capture_state remains in CAPTURE_ERROR, and the logic in vi5_fops.c prevents it from entering the CAPTURE_GOOD state.
This is probably intentional and needs to be resolved, but it seems that not closing an NVCSI stream upon one error does not prevent collecting additional frames. After making the workaround fix, I think I am able to collect raw frames using v4l2-ctl:
user@tegra-ubuntu:~$ v4l2-ctl -v width=1920,height=1080,pixelformat='GB12' --set-ctrl=bypass_mode=0,frame_rate=30000000 -c sensor_mode=0 --stream-mmap -d 0 --stream-to=video0.raw
<<<<<<<<<<<<<<< 13.55 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 21.64 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 24.40 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 25.79 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 26.62 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 27.18 fps
<<<<<<317440 != 4147200
<4096 != 4147200
...
<4096 != 4147200
< 27.58 fps
4096 != 4147200
<4096 != 4147200
...
<4096 != 4147200
< 27.89 fps
4096 != 4147200
...
<4096 != 4147200
< 30.00 fps
4096 != 4147200
<4096 != 4147200
< 30.00 fps
4096 != 4147200
<4096 != 4147200
And the trace is attached below.
trace.log (737.9 KB)
I am aware that there are CHANSEL_SHORT_FRAME errors even after boosting clocks, but that is minor for now.
At this point, I would like to be able to do the following:
-
Optionally, convert the raw capture into an encoded video, given the video format of the sensor
-
Use nvarguscamerasrc (with the econ provided ISP settings, camera_overrides.isp) to get an encoded video in gstreamer
I am trying the following reference , which has been tested in Jetpack 4.6 with these cameras and confirmed to work:
Unfortunately, I am still receiving the following error according to the Gstreamer console logs:
user@tegra-ubuntu:~$ DISPLAY=:0 GST_DEBUG=3 gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=0 ! "video/x-raw(memory:NVMM), width=(int)1920,height=(int)1080, format=(string)NV12, framerate=30/1" ! nvv4l2h265enc ! "video/x-h265, stream-format=(string)byte-stream" ! h265parse ! matroskamux ! queue ! filesink location=video0.mkv
No protocol specified
No EGL Display
nvbufsurftransform: Could not get EGL display connection
Setting pipeline to PAUSED …
No protocol specified
nvbuf_utils: Could not get EGL display connection
Opening in BLOCKING MODE
0:00:00.933197449 2616 0xaaaaead0b0a0 WARN v4l2 gstv4l2object.c:2398:gst_v4l2_object_add_interlace_mode:0xaaaaeace64f0 Failed to determine interlace mode
0:00:00.933303691 2616 0xaaaaead0b0a0 WARN v4l2 gstv4l2object.c:2398:gst_v4l2_object_add_interlace_mode:0xaaaaeace64f0 Failed to determine interlace mode
0:00:00.933360684 2616 0xaaaaead0b0a0 WARN v4l2 gstv4l2object.c:2398:gst_v4l2_object_add_interlace_mode:0xaaaaeace64f0 Failed to determine interlace mode
0:00:00.933415021 2616 0xaaaaead0b0a0 WARN v4l2 gstv4l2object.c:2398:gst_v4l2_object_add_interlace_mode:0xaaaaeace64f0 Failed to determine interlace mode
0:00:00.933549648 2616 0xaaaaead0b0a0 WARN v4l2 gstv4l2object.c:4512:gst_v4l2_object_probe_caps:nvv4l2h265enc0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
Pipeline is live and does not need PREROLL …
0:00:00.935743257 2616 0xaaaaead14cc0 FIXME default gstutils.c:3980:gst_pad_create_stream_id_internal:nvarguscamerasrc0:src Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/nvv4l2h265enc:nvv4l2h265enc0.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstH265Parse:h265parse0.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-h265, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
Redistribute latency…
NvMMLiteOpen : Block : BlockType = 8
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
0:00:00.959353336 2616 0xaaaaead14cc0 WARN v4l2bufferpool gstv4l2bufferpool.c:1114:gst_v4l2_buffer_pool_start:nvv4l2h265enc0:pool:src Uncertain or not enough buffers, enabling copy threshold
/GstPipeline:pipeline0/nvv4l2h265enc:nvv4l2h265enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1
GST_ARGUS: Creating output stream
No protocol specified
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 31.622776; Exposure Range min 450000, max 400000000;GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 31.622776; Exposure Range min 450000, max 400000000;
GST_ARGUS: 1280 x 720 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 31.622776; Exposure Range min 450000, max 400000000;
GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 31.622776; Exposure Range min 450000, max 400000000;
GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 0
Output Stream W = 1920 H = 1080
seconds to Run = 0
Frame Rate = 29.999999
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: dmabuf_fd -1 mapped entry NOT found
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadExecute:694 NvBufSurfaceFromFd Failed.
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, threadFunction:247 (propagating)
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: DISCONNECTED
Additional debug info:
Argus Error Status
Execution ended after 0:00:01.487375112
Setting pipeline to NULL …
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 277)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 379)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
GST_ARGUS: Cleaning up
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
(Argus) Error InvalidState: Receive thread is not running cannot send. (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 96)
(Argus) Error InvalidState: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)
Caught SIGSEGV
./test_gstargus.sh: line 4: 2616 Segmentation fault (core dumped) GST_DEBUG=3 gst-launch-1.0 -v nvarguscamerasrc sensor-id=0 sensor-mode=0 ! “video/x-raw(memory:NVMM), width=(int)1920,height=(int)1080, format=(string)NV12, framerate=30/1” ! nvv4l2h265enc ! “video/x-h265, stream-format=(string)byte-stream” ! h265parse ! matroskamux ! queue ! filesink location=video0.mkv
The dmesg log just shows:
[ 317.539901] tegra-camrtc-capture-vi tegra-capture-vi: corr_err: discarding frame 0, flags: 0, err_data 131072
[ 317.641815] eimx462 33-0043: cam_stream_on 1106MCU Stream On Success !!
[ 328.179367] eimx462 33-0043: cam_stream_off 1178 MCU Get CMD Stream off Success !!
While we wait for econ you might be able to provide an update based on the above information.
Thank you.