I’m using gstreamer to decode 10M+ single image into numpy array, the pipeline and code i use is like this:
>>> cap = cv2.VideoCapture("filesrc location=vYYBAFqIM56ALxqmAQaLE_hA0Ag565.jpg ! image/jpeg,width=6912, height=9216,framerate=1/1! nvjpegdec ! imagefreeze ! videoconvert! video/x-raw,format=BGR,width=6912, height=9216,framerate=1/1 ! appsink", cv2.CAP_GSTREAMER)
[ WARN:0] global /opt/workspace/opencv-4.5.2/modules/videoio/src/cap_gstreamer.cpp (1081) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=2, duration=-1
>>> cap = cv2.VideoCapture("filesrc location=test_1.jpg ! nvjpegdec ! imagefreeze ! videoconvert! appsink", cv2.CAP_GSTREAMER)[ WARN:0] global /opt/workspace/opencv-4.5.2/modules/videoio/src/cap_gstreamer.cpp (1081) open OpenCV | GStreamer warning: Cannot query video position: status=1, value=2, duration=-1
>>> a, b = cap.read()
>>> a
True
>>> b.shape
(9216, 6912, 3)
>>> np.nonzero(b)
(array([], dtype=int64), array([], dtype=int64), array([], dtype=int64))
which means I do get the shape of the image, but value of it are all 0, what’s wrong with this pipeline?
jtop shows that NVJPG is working…
this is build info of opencv
>>> print(cv2.getBuildInformation())General configuration for OpenCV 4.5.2 ===================================== Version control: unknown
Extra modules:
Location (extra): /opt/workspace/opencv_contrib-4.5.2/modules
Version control (extra): unknown
Platform:
Timestamp: 2021-07-28T07:22:24Z
Host: Linux 4.9.201-tegra aarch64
CMake: 3.10.2
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE
CPU/HW features:
Baseline: NEON FP16
C/C++:
Built as dynamic libs?: YES
C++ standard: 11
C++ Compiler: /usr/bin/c++ (ver 7.5.0)
C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
C Compiler: /usr/bin/cc
C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release): -Wl,--gc-sections -Wl,--as-needed
Linker flags (Debug): -Wl,--gc-sections -Wl,--as-needed
ccache: NO
Precompiled headers: NO
Extra dependencies: m pthread cudart_static dl rt nppc nppial nppicc nppicom nppidei nppif nppig nppim nppist nppisu nppitc npps cublas cudnn cufft -L/usr/local/cuda-10.2/lib64 -L/usr/lib/aarch64-linux-gnu
3rdparty dependencies:
OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dnn_superres dpm face features2d flann freetype fuzzy gapi hfs highgui img_hash imgcodecs imgproc intensity_transform line_descriptor mcc ml objdetect optflow phase_unwrapping photo plot python2 python3 quality rapid reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking video videoio videostab wechat_qrcode xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: alphamat cnn_3dobj cvv hdf java julia matlab ovis sfm ts viz
Applications: apps
Documentation: NO
Non-free algorithms: NO
GUI:
GTK+: YES (ver 2.24.32)
GThread : YES (ver 2.56.4)
GtkGlExt: NO
VTK support: NO
Media I/O:
ZLib: /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
JPEG: /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
WEBP: build (ver encoder: 0x020f)
PNG: /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
TIFF: /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.0.9)
JPEG 2000: build (ver 2.4.0)
OpenEXR: build (ver 2.3.0)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: YES (2.2.5)
FFMPEG: YES
avcodec: YES (57.107.100)
avformat: YES (57.83.100)
avutil: YES (55.78.100)
swscale: YES (4.8.100)
avresample: NO
GStreamer: YES (1.14.5)
v4l/v4l2: YES (linux/videodev2.h)
Parallel framework: pthreads
Trace: YES (with Intel ITT)
Other third-party libraries:
Lapack: NO
Eigen: NO
Custom HAL: YES (carotene (ver 0.0.1))
Protobuf: build (3.5.1)
NVIDIA CUDA: YES (ver 10.2, CUFFT CUBLAS)
NVIDIA GPU arch: 53 62 72
NVIDIA PTX archs:
cuDNN: YES (ver 8.0.0)
OpenCL: YES (no extra features)
Include path: /opt/workspace/opencv-4.5.2/3rdparty/include/opencl/1.2
Link libraries: Dynamic load
Python 2:
Interpreter: /usr/bin/python2.7 (ver 2.7.17)
Libraries: /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.17)
numpy: /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
install path: lib/python2.7/dist-packages/cv2/python-2.7
Python 3:
Interpreter: /usr/bin/python3 (ver 3.6.9)
Libraries: /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
numpy: /usr/lib/python3/dist-packages/numpy/core/include (ver 1.13.3)
install path: lib/python3.6/dist-packages/cv2/python-3.6
Python (for build): /usr/bin/python2.7
Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Install to: /usr/local
-----------------------------------------------------------------
>>>
Hi,
Please run
$ gst-launch-1.0 filesrc location=vYYBAFqIM56ALxqmAQaLE_hA0Ag565.jpg ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=RGBA ! videoconvert! video/x-raw,format=BGR ! filesink location=test.BGR
And check if you get correct BGR data. If yes, please replace location=test.BGR with appsink and apply to cv2.VideoCapture().
Hi,
Thanks for reply!
I ran the command above and got 0 size test.BGR file：
$ gst-launch-1.0 filesrc location=vYYBAFqIM56ALxqmAQaLE_hA0Ag565.jpg ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=RGBA ! videoconvert! video/x-raw,format=BGR ! filesink location=test.BGR
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Opening in BLOCKING MODE
Pipeline is PREROLLING ...
ERROR: from element /GstPipeline:pipeline0/GstJpegParse:jpegparse0: Internal data stream error.
Additional debug info:
gstbaseparse.c(3611): gst_base_parse_loop (): /GstPipeline:pipeline0/GstJpegParse:jpegparse0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Freeing pipeline ...
I’ve connected Nano to a monitor and test the example code before like:
gst-launch-1.0 filesrc location=vYYBAFqIM56ALxqmAQaLE_hA0Ag565.jpg ! nvjpegdec ! \
imagefreeze ! xvimagesink -e
it did jump out the picture.
Hi,
Please check if you can decode the sample JPEG file. We can decode it successfully:
nvidia@tegra-ubuntu:~$ gst-launch-1.0 filesrc location= /usr/src/jetson_multimedia_api/data/Picture/nvidia-logo.jpg ! jpegparse ! nvv4l2decoder mjpeg=1 ! nvvidconv ! 'video/x-raw,format=RGBA' ! videoconvert ! video/x-raw,format=BGR ! filesink location=a.BGR
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Pipeline is PREROLLING ...
NvMMLiteOpen : Block : BlockType = 277
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 277
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Got EOS from element "pipeline0".
Execution ended after 0:00:00.011535326
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
Form the print it looks like the file cannot be recognized by jpegparse plugin. May also try without jpegparse
gst-launch-1.0 filesrc location=vYYBAFqIM56ALxqmAQaLE_hA0Ag565.jpg ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=RGBA ! videoconvert! video/x-raw,format=BGR ! filesink location=test.BGR