Converting h264 to RGB on Jetson Nano

I’m attempting to convert an mp4 to RGB using GST. I get the following data when piping to the fake sink.

$ gst-launch-1.0 -v filesrc location=./test.mp4 ! decodebin ! fakesink silent=False
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = video/quicktime
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:src_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)constrained-baseline, codec_data=(buffer)014240150301000b6742402895a01e0089f95001000468ce3c80, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)constrained-baseline, codec_data=(buffer)014240150301000b6742402895a01e0089f95001000468ce3c80, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstMultiQueuePad:sink_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)constrained-baseline, codec_data=(buffer)014240150301000b6742402895a01e0089f95001000468ce3c80, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)6250/207
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)449ab9831a644fbdbee1615ed92715dd9f4a93b7922395ec46b6172391c3ce98/001, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)2;) 0x7f8400c010
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)“video/x-raw(memory:NVMM),\ format=(string)NV12,\ width=(int)1920,\ height=(int)1080,\ interlace-mode=(string)progressive,\ multiview-mode=(string)mono,\ multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono,\ pixel-aspect-ratio=(fraction)1/1,\ chroma-site=(string)mpeg2,\ colorimetry=(string)bt709,\ framerate=(fraction)6250/207”;) 0x7f8400c6c0
/GstPipeline:pipeline0/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)6250/207
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad2: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)6250/207
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)“GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)5596000000, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;”;) 0x7f8400c320
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: tag (20510), GstTagList-stream, taglist=(taglist)“taglist,\ video-codec=(string)"H.264\\ /\\ AVC",\ bitrate=(uint)2590869;”;) 0x7f78008490
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: tag (20510), GstTagList-global, taglist=(taglist)“taglist,\ datetime=(datetime)2023-11-19T21:05:39Z,\ container-format=(string)Quicktime;”;) 0x7f8400c240
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: tag (20510), GstTagList-stream, taglist=(taglist)“taglist,\ video-codec=(string)"H.264\\ \(Constrained\\ Baseline\\ Profile\)",\ bitrate=(uint)2590869;”;) 0x7f78008570
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = preroll *******
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1008 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.033120000, offset: -1, offset_end: -1, flags: 00000440 discont header , meta: GstVideoMeta) 0x7f88030e70
(omitting lines for brevity)
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: tag (20510), GstTagList-stream, taglist=(taglist)“taglist,\ video-codec=(string)"H.264\\ \(Constrained\\ Baseline\\ Profile\)",\ bitrate=(uint)2590869,\ minimum-bitrate=(uint)3184541,\ maximum-bitrate=(uint)3184541;”;) 0x7f78008ed0
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = chain ******* (fakesink0:sink) (1008 bytes, dts: none, pts: 0:00:00.295333333, duration: 0:00:00.033120000, offset: -1, offset_end: -1, flags: 00000000 , meta: GstVideoMeta) 0x7f88030e70
(omitting lines for brevity)
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: stream-group-done (26894), GstEventStreamGroupDone, group-id=(uint)2;) 0x7f780c5570
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: eos (28174), ) 0x7f8400c420
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.537895455
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

From this output, it seems I could add videoconvert ! video/x-raw,format=RGB and convert to RGB. However, I get the following error when doing so:

$ gst-launch-1.0 -v filesrc location=./test.mp4 ! decodebin ! videoconvert ! video/x-raw,format=RGB ! fakesink silent=False
Setting pipeline to PAUSED …
Pipeline is PREROLLING …
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = video/quicktime
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = NULL
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:src_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)constrained-baseline, codec_data=(buffer)014240150301000b6742402895a01e0089f95001000468ce3c80, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)constrained-baseline, codec_data=(buffer)014240150301000b6742402895a01e0089f95001000468ce3c80, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstMultiQueuePad:sink_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)2.1, profile=(string)constrained-baseline, codec_data=(buffer)014240150301000b6742402895a01e0089f95001000468ce3c80, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4, profile=(string)constrained-baseline, width=(int)1920, height=(int)1080, framerate=(fraction)6250/207, pixel-aspect-ratio=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
/GstPipeline:pipeline0/GstDecodeBin:decodebin0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)6250/207
/GstPipeline:pipeline0/GstFakeSink:fakesink0: last-message = event ******* (fakesink0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)449ab9831a644fbdbee1615ed92715dd9f4a93b7922395ec46b6172391c3ce98/001, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)2;) 0x7f9000e010
/GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstDecodePad:src_0.GstProxyPad:proxypad2: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)6250/207
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1:
not negotiated
ERROR: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0: Internal data stream error.
Additional debug info:
qtdemux.c(6073): gst_qtdemux_loop (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstQTDemux:qtdemux0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to NULL …
Freeing pipeline …

Any suggestions for getting this decoded to RGB?

Hi,
Please try the commands one by one:

$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! fakesink
$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw, format=(string)RGBA ! fakesink
$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw, format=(string)RGBA ! videoconvert ! video/x-raw,format=RGB ! fakesink

Hello, Here is the output. I would expect to see some frame data from fakesink silent=False, but it doesn’t show much.

Also, the video is about 5 seconds long. It seems unexpected to see a very short execution time for the pipeline in the output below. How should I interpret that execution time?

$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! fakesink silent=False
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is PREROLLING …
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.571422080
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! “video/x-raw, format=(string)RGBA” ! fakesink silent=False
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is PREROLLING …
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Got EOS from element “pipeline0”.
Execution ended after 0:00:01.299858404
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! “video/x-raw, format=(string)RGBA” ! videoconvert ! video/x-raw,format=RGB ! fakesink silent=False
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is PREROLLING …
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
Pipeline is PREROLLED …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Got EOS from element “pipeline0”.
Execution ended after 0:00:02.731016199
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Hi,
We are not sure how silent=False works. This would need other users to share experience.

There is no error in the prints and it looks like the frames are decoded. Please try the command and check whether RGB data is saved:

$ gst-launch-1.0 filesrc location=test.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! nvvidconv ! “video/x-raw, format=(string)RGBA” ! videoconvert ! video/x-raw,format=RGB ! filesink location=dump.RGB

Looks like it works. Was able to see the data using appsink through python. Thanks for the help.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.