Nvjpegdec produces transparent frames

tomskih_pa · August 5, 2022, 6:11am

• Hardware Platform (Jetson / GPU)
GPU NVIDIA GeForce RTX 2080

• DeepStream Version
6.1.0

• JetPack Version (valid for Jetson only)
• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)
515.48.07

• Issue Type( questions, new requirements, bugs)
bug

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Launch a pipeline that converts JPEG image to PNG.

gst-launch-1.0 -v \
    filesrc location=test.jpg ! \
    nvjpegdec ! \
    nvvideoconvert ! \
    pngenc ! \
    filesink location=nvjpegdec.png

test.jpg:
test

nvjpegdec.png is transparent. The value of alpha channel byte is 0x01 (should be 0xff). Probably nvjpegdec sets alpha channel value in range 0 - 1 (float) rather than 0 - 255 (int).
nvjpegdec.png:

Log:

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstJpegDec:jpegdec0.GstPad:src: caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)0/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, block-linear=(boolean)false
/GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)0/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, block-linear=(boolean)false
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:sink: caps = video/x-raw, format=(string)I420, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)1:4:0:0, framerate=(fraction)0/1
/GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:src: caps = image/png, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, interlace-mode=(string)progressive, colorimetry=(string)sRGB, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = image/png, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)0/1, interlace-mode=(string)progressive, colorimetry=(string)sRGB, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.000095077
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Content of the frame produced by nvjpegdec. Note 01 byte on alpha channel.

$ gst-launch-1.0 \
    filesrc location=test.jpg ! \
    nvjpegdec ! \
    nvvideoconvert ! \
    'video/x-raw,format=RGBA' ! \
    fakesink dump=true
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Using GPU 0 (NVIDIA GeForce RTX 2080, 46 SMs, 1024 th/SM max, CC 7.5, ECC off)
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
00000000 (0x7ff75ec4e220): eb eb eb 01 eb eb eb 01 eb eb eb 01 eb eb eb 01  ................
00000010 (0x7ff75ec4e230): eb eb eb 01 eb eb eb 01 eb eb eb 01 eb eb eb 01  ................
...
0012bfe0 (0x7ff75ed7a200): 7d 7d 7d 01 d0 d0 d0 01 3e 3e 3e 01 8e 8e 8e 01  }}}.....>>>.....
0012bff0 (0x7ff75ed7a210): 65 65 65 01 48 48 48 01 1f 1f 1f 01 cd cd cd 01  eee.HHH.........
Got EOS from element "pipeline0".
Execution ended after 0:00:00.557729472
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

yingliu · August 7, 2022, 12:29pm

Please try below command
gst-launch-1.0 filesrc location=test.jpeg ! jpegparse ! nvv4l2decoder ! nvvideoconvert ! pngenc ! filesink location=test.png

tomskih_pa · August 8, 2022, 2:30am

gst-launch-1.0 filesrc location=test.jpeg ! jpegparse ! nvv4l2decoder ! nvvideoconvert ! pngenc ! filesink location=test.png works fine, test.png isn’t transparent.

Logs

gst-launch-1.0 -v filesrc location=test.jpeg ! jpegparse ! nvv4l2decoder ! nvvideoconvert ! pngenc ! filesink location=test.png   
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
/GstPipeline:pipeline0/GstJpegParse:jpegparse0.GstPad:src: caps = image/jpeg, parsed=(boolean)true, format=(string)I420, width=(int)640, height=(int)480, framerate=(fraction)1/1
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:sink: caps = image/jpeg, parsed=(boolean)true, format=(string)I420, width=(int)640, height=(int)480, framerate=(fraction)1/1
/GstPipeline:pipeline0/nvv4l2decoder:nvv4l2decoder0.GstPad:src: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)1/1
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:src: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, block-linear=(boolean)false
/GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:sink: caps = video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)1/1, pixel-aspect-ratio=(fraction)1/1, format=(string)RGBA, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, block-linear=(boolean)false
/GstPipeline:pipeline0/Gstnvvideoconvert:nvvideoconvert0.GstPad:sink: caps = video/x-raw(memory:NVMM), format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)1/1
/GstPipeline:pipeline0/GstPngEnc:pngenc0.GstPad:src: caps = image/png, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)sRGB, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = image/png, width=(int)640, height=(int)480, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)1/1, interlace-mode=(string)progressive, colorimetry=(string)sRGB, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.000236902
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

yingliu · August 8, 2022, 8:41am

Still checking, but not sure if it is an issue in nvjepgdec as if I switch “pngenc” to “jpegenc” and the correct jpeg file can be created.

tomskih_pa · August 9, 2022, 2:18am

The issue can’t be reproduced with “jpegenc” since jpeg format doesn’t support transparency.

yingliu · August 9, 2022, 2:24am

Can you let us know if it blocks your scenario or found it is a potential bug? If you need create png pictures you can use nvv4l2decoder with pngenc plugin, currently we still need some time to figure out what happends on this case, thank you.

tomskih_pa · August 9, 2022, 2:37am

It doesn’t block our scenario.

yuweiw · August 11, 2022, 5:44am

Hi @tomskih_pa , we are analyzing this problem. You can just use the nvv4l2decoder instead of the nvjpegdec to transcode the image temporary. Thanks

finch_1024 · August 6, 2023, 2:27am

error happened using nvv4l2decoer and how to solve it?

yuweiw · August 7, 2023, 1:41am

Your platform may not support hard decoding. Could you file a new topic to describe your problem in detail?

yingliu · August 15, 2023, 5:48am

This issue is fixed in DS6.3.

yingliu · August 15, 2023, 5:48am

Please file a new topic for the issue you met, thanks.

yingliu · August 15, 2023, 5:49am