capturing raw video using gstreamer

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I am just trying to capture the camera raw to a file at a certain resolution and frame rate . Could you please tell me how this command could do that capture?

nvidia@tegra-ubuntu:~/scripts/scfile$ sh -x ./e.sh

  • gst-launch-1.0 nvcamerasrc fpsRange=30 30 intent=3 ! video/x-raw, format=(string)I420, width=(int)1280, height=(int)720, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)I420, width=(int)1280, height=(int)720 ! filesink location=raw.420 -e
    WARNING: erroneous pipeline: could not link nvcamerasrc0 to nvvconv0
    nvidia@tegra-ubuntu:~/scripts/scfile$

Thank you,
Y-

2

hello Yile,

since gstreamer go through ISP, you’re not able to capture the camera raw if you’re going through gstreamer pipeline.
the other way to capture camera raw files would be v4l2-ctl, which we called VI-mode.
here’s command for your reference.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw

however, there’s a known issue of TX2 that camera sensor with embedded metadata might got PXL_SOF failure.
here’s topic for your reference: https://devtalk.nvidia.com/default/topic/1007058/
thanks

3

@JerryChang does the v4l2-ctl work with bayer’s camera presented at the developers board?
By now it was found that recording from it could be captured with e.g:

nvgstcapture -m 2 -k 0

Reference: https://devtalk.nvidia.com/default/topic/1010795/jetson-tx2/v4l2-on-jetson-tx2/post/5156813/#5156813

4

hello Andrey1984,

i had reply the topic, [url]https://devtalk.nvidia.com/default/topic/1010795/[/url]
thanks

5

I was attempting to capture the camera to a ‘raw’ file and playback the file using gstreamer.

I use:
gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’! filesink location=test.raw

to capture the file and :

gst-launch-1.0 filesrc location=test.raw ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvoverlaysink -e

to display the file.

so far display just runs and exits. no errors.

Y-

6

You may try this for recording:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1' ! nvvidconv flip-method=6 ! 'video/x-raw, width=1920, height=1080, format=I420, framerate=30/1' ! filesink location=test.I420

Be careful that this may quickly generate a huge file and fill your jetson MMC, so if you intend to capture a long sequence, record it on a big disk.

and this for playing back:

gst-launch-1.0 filesrc location=test.I420 ! videoparse width=1920 height=1080 framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM), width=1920, height=1080, format=I420, framerate=30/1' ! nvoverlaysink
7

Thank you that worked.

I downloaded a file: https://media.xiph.org/video/derf/y4m/ducks_take_off_420_720p50.y4m

When I run this command the screen rolls to the right. Any ideas?

gst-launch-1.0 filesrc location=/home/nvidia/ducks_take_off_420_720p50.y4m ! videoparse width=1280 height=720 framerate=50/1 ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=1280, height=720, format=I420, framerate=50/1’ ! nvoverlaysink

Y-

8

For videoparse, I420 is the default format. You can use other formats, see available ones with

gst-inspect-1.0 videoparse

under properties/format.
For y4m, you may have to decode it first. With gstreamer, check y4mdec gstreamer plugin.
Otherwise, maybe ffmpeg can convert a y4m file to a I420 file.

9

There isn’t much documentation on how to use y4mdec. Could you give an example ?
Thx,
Y-

10

Got it working thanks:
gst-launch-1.0 filesrc location=/home/nvidia/ducks_take_off_420_720p50.y4m ! y4mdec ! nvoverlaysink

11 
gst-launch-1.0 -v filesrc location=example.y4m ! y4mdec ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420' ! nvoverlaysink

Note that y4mdec plugin is in gstreamer-plugins-bad, if you can’t find it with gst-inspect-1.0, install it with:

sudo apt-get install gstreamer1.0-plugins-bad
12

Thanks that worked.

Note: when I run the bigger file I get the following output:

  • gst-launch-1.0 -v filesrc location=/home/nvidia/ducks_take_off_2160p50.y4m ! y4mdec ! nvvidconv ! video/x-raw(memory:NVMM), format=I420 ! nvoverlaysink
    Setting pipeline to PAUSED …
    Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingPipeline is PREROLLING …
    /GstPipeline:pipeline0/GstY4mDec:y4mdec0.GstPad:src: caps = “video/x-raw,\ format=(string)I420,\ width=(int)3840,\ height=(int)2160,\ interlace-mode=(string)progressive,\ pixel-aspect-ratio=(fraction)1/1,\ chroma-site=(string)mpeg2,\ colorimetry=(string)bt2020,\ framerate=(fraction)50/1”
    /GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = “video/x-raw(memory:NVMM),\ width=(int)3840,\ height=(int)2160,\ interlace-mode=(string)progressive,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)50/1,\ format=(string)I420”
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = “video/x-raw(memory:NVMM),\ width=(int)3840,\ height=(int)2160,\ interlace-mode=(string)progressive,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)50/1,\ format=(string)I420”
    /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0.GstPad:sink: caps = “video/x-raw(memory:NVMM),\ width=(int)3840,\ height=(int)2160,\ interlace-mode=(string)progressive,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)50/1,\ format=(string)I420”
    /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = “video/x-raw(memory:NVMM),\ width=(int)3840,\ height=(int)2160,\ interlace-mode=(string)progressive,\ pixel-aspect-ratio=(fraction)1/1,\ framerate=(fraction)50/1,\ format=(string)I420”
    /GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = “video/x-raw,\ format=(string)I420,\ width=(int)3840,\ height=(int)2160,\ interlace-mode=(string)progressive,\ pixel-aspect-ratio=(fraction)1/1,\ chroma-site=(string)mpeg2,\ colorimetry=(string)bt2020,\ framerate=(fraction)50/1”
    Pipeline is PREROLLED …
    Setting pipeline to PLAYING …
    New clock: GstSystemClock
    WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
    There may be a timestamping problem, or this computer is too slow.
    WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
    There may be a timestamping problem, or this computer is too slow.
    WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
    There may be a timestamping problem, or this computer is too slow.
    WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
    There may be a timestamping problem, or this computer is too slow.
    WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped.
    Additional debug info:
    gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0:
    There may be a timestamping problem, or this computer is too slow.
    ^Chandling interrupt.
    Interrupt: Stopping pipeline …
    Execution ended after 0:00:12.979387839
    Setting pipeline to PAUSED …
    Setting pipeline to READY …
    Setting pipeline to NULL …
    Freeing pipeline …
13

Have you boosted your TX2 ?

sudo /usr/sbin/nvpmodel -m0
sudo /home/ubuntu/jetson_clocks.sh
14

Ran those two commands above. Still got the same results playing the video.
Thx,