I am trying to record videos from a connected camera. I am trying to use a python script but alternative method is fine if someone has a suggestion. With script, which works on other PCs does not work on the Xavier. I get an output video of only a few Mbs and the video is only the first frame. I have tried different codec/output formats with no success. Any advice on the code here or an alternative suggestion for recording video?
recordVideo.py (440 Bytes)
Hi,
On Jetson platforms, generally we run gstreamer pipelines in cv2.VideoCapture() and cv2.VideoWriter(). Please refer to this sample:
Displaying to the screen with OpenCV and GStreamer - #9 by DaneLLL
Ok I tried switching it to follow that example but I simply get failed to open output. This is a CSI camera running 1920x1080 30fps YUV. The camera runs and displays fine with the code but I cannot get video to record. Updated code with attempt to use GST attached.
recordVideo.py (793 Bytes)
You may try:
gst_out = "appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc ! video/x-h264,format=byte-stream ! h264parse ! matroskamux ! filesink location=test-nvh264-writer.mkv "
out = cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, float(cap.get(cv2.CAP_PROP_FPS)), (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))))
This still does not work. As is I get an error V4L: unable to get FPS Failed to open output. Manually setting the FPS to 30 I just get failed to open output. With my code the camera opens and streams but the recorded files are extremely small and only show one frame.
This is using a leopard GMSL camera and leopard adapter board directly connected to the CSI port on the bottom of the Xavier. It is outputting at 30FPS YUV frames. I can stream and I can perform CV functions but I cannot get it to record.
If you are still using it, try commenting imshow call. It is not efficient on Jetson and may slow down.
If you can post the ouput of:
v4l2-ctl -d0 --list-formats-ext
(v4l2-ctl command is provided by apt package v4l-utills), it may be possible to debug this further.
This is the ouput of that command:
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Index : 1
Type : Video Capture
Pixel Format: 'YUYV'
Name : YUYV 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Ok, so first try to display your camera in gstreamer:
# The following expects a X display
gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! xvimagesink
# If not running X but having a monitor able to diplay this mode connected to Jeston:
gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! nvovelaysink
If it works, you may try to record H264 encoded video into matroskamux container file with:
gst-launch-1.0 -ev v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvv4l2h264enc ! h264parse ! matroskamux ! filesink location=test_h264.mkv
If this works, you may try to change your opencv VideoCapture from V4L backend to gstreamer backend:
cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1 ", cv2.CAP_GSTREAMER)
first command works no issues, video streams and is running at 30fps.
Second command I never see stream, hangs with this as the output:
Setting pipeline to PAUSED …
Opening in BLOCKING MODE
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, format=(string)NV12
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, format=(string)NV12
/GstPipeline:pipeline0/nvv4l2h264enc:nvv4l2h264enc0.GstPad:src: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, level=(string)NULL, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:sink: caps = video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, profile=(string)NULL, level=(string)NULL, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2
Redistribute latency…
NvMMLiteOpen : Block : BlockType = 4
===== NVMEDIA: NVENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
/GstPipeline:pipeline0/nvv4l2h264enc:nvv4l2h264enc0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, format=(string)NV12
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, framerate=(fraction)30/1, interlace-mode=(string)progressive, format=(string)NV12
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)30/1, colorimetry=(string)2:4:7:1, interlace-mode=(string)progressive
H264: Profile = 66, Level = 0
/GstPipeline:pipeline0/GstH264Parse:h264parse0.GstPad:src: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)constrained-baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, codec_data=(buffer)01424028ffe1000b67424028965403c0113f2a01000468ce3c80
/GstPipeline:pipeline0/GstMatroskaMux:matroskamux0.GstMatroskamuxPad:video_0: caps = video/x-h264, stream-format=(string)avc, alignment=(string)au, profile=(string)constrained-baseline, level=(string)4, width=(int)1920, height=(int)1080, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)30/1, interlace-mode=(string)progressive, colorimetry=(string)bt709, chroma-site=(string)mpeg2, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true, codec_data=(buffer)01424028ffe1000b67424028965403c0113f2a01000468ce3c80
/GstPipeline:pipeline0/GstMatroskaMux:matroskamux0.GstPad:src: caps = video/x-matroska
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-matroska
/GstPipeline:pipeline0/GstMatroskaMux:matroskamux0.GstPad:src: caps = video/x-matroska, streamheader=(buffer)< 1a45dfa301000000000000144282896d6174726f736b610042878102428581021853806701ffffffffffffff114d9b74010000000000008c4dbb010000000000001253ab841549a96653ac88ffffffffffffffff4dbb010000000000001253ab841654ae6b53ac88ffffffffffffffff4dbb010000000000001253ab841043a77053ac88ffffffffffffffff4dbb010000000000001253ab841c53bb6b53ac88ffffffffffffffff4dbb010000000000001253ab841254c36753ac88ffffffffffffffff1549a966010000000000007473a490105b52499570ed2534d4c0497328ac592ad7b1830f424044898800000000000000004d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d757865720044618808f09aa13bf058001654ae6b010000000000006eae0100000000000065d7810183810173c588d6d0e8c136319da923e3838401fca055536e86566964656f00e0010000000000000bb0820780ba8204389a81028690565f4d504547342f49534f2f4156430063a29a01424028ffe1000b67424028965403c0113f2a01000468ce3c80 >
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-matroska, streamheader=(buffer)< 1a45dfa301000000000000144282896d6174726f736b610042878102428581021853806701ffffffffffffff114d9b74010000000000008c4dbb010000000000001253ab841549a96653ac88ffffffffffffffff4dbb010000000000001253ab841654ae6b53ac88ffffffffffffffff4dbb010000000000001253ab841043a77053ac88ffffffffffffffff4dbb010000000000001253ab841c53bb6b53ac88ffffffffffffffff4dbb010000000000001253ab841254c36753ac88ffffffffffffffff1549a966010000000000007473a490105b52499570ed2534d4c0497328ac592ad7b1830f424044898800000000000000004d80a54753747265616d6572206d6174726f736b616d75782076657273696f6e20312e31342e35005741994753747265616d6572204d6174726f736b61206d757865720044618808f09aa13bf058001654ae6b010000000000006eae0100000000000065d7810183810173c588d6d0e8c136319da923e3838401fca055536e86566964656f00e0010000000000000bb0820780ba8204389a81028690565f4d504547342f49534f2f4156430063a29a01424028ffe1000b67424028965403c0113f2a01000468ce3c80 >
I dont see a way of stopping this cleanly, killing the process with ctrl+C gives me an output file that is reasonable sized (10s of Mbs depending on length) but I suspect it is corrupt due to how I am closing it. The video cannot be played by VLC.
You may have to issue a second termination signal (first try one or two seconds later) when using EOS with -e.
Is the saved file ok ? Note that Ubuntu totem may not work correctly with Jetsons, so better read your file from gstreamer on Jetson, or upload to an host for checking.
opening the file on a different PC works!
What do you mean by second termination signal? How am I supposed to be terminating the recording from the terminal command?
If I try to move on to your third suggestion of switching cap to gstreamer backend as you wrote what do I need for my gst_out and out commands?
Glad to see it worked out.
The second term signal may not be be relevant for your case (it is sometimes required for terminating a pipeline with EOS and nvarguscamerasrc).
You would just change the VideoCapture from V4L VideoCapture(0) to the gstreamer pipeline capture. Then I would expect the framerate could be read. Be sure to comment imshow. If you need display, it should be possible to use one from VideoWriter.
recordVideo.py (958 Bytes)
Attached is what I attempted based on what I think you are suggesting, but I still get a failed to open output error msg.
I added the following after the CAP to see what it was returning:
w = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
h = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)
fps = cap.get(cv2.CAP_PROP_FPS)
print('Src opened, %dx%d @ %d fps' % (w, h, fps))
The result is: Src opened, 0x0 @ 0 fps
so it doesn’t look like it is opening the capture source.
This is what I am using but I have tried many modifications and so far no success.
cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1 ", cv2.CAP_GSTREAMER)
Seems the capture pipeline failed to start.
You would try this customization of your code:
import cv2
# Get information about your opencv build options
print(cv2.getBuildInformation())
# Open camera
cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1 ", cv2.CAP_GSTREAMER)
if not cap.isOpened():
print("Failed to open camera")
exit()
# Open writer to H264/MKV file
gst_out = "appsrc ! queue ! videoconvert ! video/x-raw,format=BGRx ! nvvidconv ! nvv4l2h264enc ! video/x-h264,format=byte-stream ! h264parse ! matroskamux ! filesink location=test-nvh264-writer.mkv "
out = cv2.VideoWriter(gst_out, cv2.CAP_GSTREAMER, 0, float(cap.get(cv2.CAP_PROP_FPS)), (int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))))
if not out.isOpened():
print("Failed to open output")
exit()
# Loop
while(cap.isOpened()):
ret, frame = cap.read()
if ret==True:
out.write(frame)
#cv2.imshow('frame',frame)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
else:
break
cap.release()
out.release()
cv2.destroyAllWindows()
and post the output for giving some details.
Output:
General configuration for OpenCV 4.0.0 =====================================
Version control: unknown
Extra modules:
Location (extra): /home/name/opencv_contrib/modules
Version control (extra): unknown
Platform:
Timestamp: 2021-01-15T19:02:50Z
Host: Linux 4.9.140-tegra aarch64
CMake: 3.13.3
CMake generator: Unix Makefiles
CMake build tool: /usr/bin/make
Configuration: RELEASE
CPU/HW features:
Baseline: NEON FP16
required: NEON
disabled: VFPV3
C/C++:
Built as dynamic libs?: YES
C++ Compiler: /usr/bin/c++ (ver 7.5.0)
C++ flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG -DNDEBUG
C++ flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -g -O0 -DDEBUG -D_DEBUG
C Compiler: /usr/bin/cc
C flags (Release): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -O3 -DNDEBUG -DNDEBUG
C flags (Debug): -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -fvisibility=hidden -g -O0 -DDEBUG -D_DEBUG
Linker flags (Release):
Linker flags (Debug):
ccache: NO
Precompiled headers: YES
Extra dependencies: dl m pthread rt
3rdparty dependencies:
OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core datasets dnn dnn_objdetect dpm face features2d flann freetype fuzzy gapi hdf hfs highgui img_hash imgcodecs imgproc java_bindings_generator line_descriptor ml objdetect optflow phase_unwrapping photo plot python3 python_bindings_generator reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
Disabled: world
Disabled by dependency: -
Unavailable: cnn_3dobj cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cvv java js matlab ovis python2 sfm viz
Applications: tests perf_tests examples apps
Documentation: NO
Non-free algorithms: YES
GUI:
GTK+: YES (ver 3.22.30)
GThread : YES (ver 2.56.4)
GtkGlExt: NO
VTK support: NO
Media I/O:
ZLib: /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
JPEG: /usr/lib/aarch64-linux-gnu/libjpeg.so (ver 80)
WEBP: build (ver encoder: 0x020e)
PNG: /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
TIFF: /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 / 4.0.9)
JPEG 2000: build (ver 1.900.1)
OpenEXR: build (ver 1.7.1)
HDR: YES
SUNRASTER: YES
PXM: YES
PFM: YES
Video I/O:
DC1394: NO
FFMPEG: YES
avcodec: YES (ver 57.107.100)
avformat: YES (ver 57.83.100)
avutil: YES (ver 55.78.100)
swscale: YES (ver 4.8.100)
avresample: NO
GStreamer: NO
v4l/v4l2: linux/videodev2.h
Parallel framework: pthreads
Trace: YES (built-in)
Other third-party libraries:
Lapack: NO
Eigen: NO
Custom HAL: YES (carotene (ver 0.0.1))
Protobuf: build (3.5.1)
OpenCL: YES (no extra features)
Include path: /home/name/opencv/3rdparty/include/opencl/1.2
Link libraries: Dynamic load
Python 3:
Interpreter: /home/name/.local/bin/.virtualenvs/env1/bin/python3 (ver 3.6.9)
Libraries: /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.9)
numpy: /home/name/.local/bin/.virtualenvs/env1/lib/python3.6/site-packages/numpy/core/include (ver 1.16.1)
packages path: lib/python3.6/site-packages
Python (for build): /usr/bin/python2.7
Java:
ant: NO
JNI: NO
Java wrappers: NO
Java tests: NO
Install to: /usr/local
-----------------------------------------------------------------
I am on it. I will report back results.
Thank you so much for the support these past few days.
Ok, So recompiled CV2 with Gstreamer. I now get this with build information:
Video I/O:
DC1394: NO
FFMPEG: YES
avcodec: YES (ver 57.107.100)
avformat: YES (ver 57.83.100)
avutil: YES (ver 55.78.100)
swscale: YES (ver 4.8.100)
avresample: NO
GStreamer:
base: YES (ver 1.14.5)
video: YES (ver 1.14.5)
app: YES (ver 1.14.5)
riff: YES (ver 1.14.5)
pbutils: YES (ver 1.14.5)
I have tried multiple configurations of my cap= statement but no matter what I get the following error:
(python3:8867): GStreamer-CRITICAL **: 09:38:33.873: gst_element_get_state: assertion 'GST_IS_ELEMENT (element)' failed
Failed to open camera
Any ideas?
Ok, so this seems to be the winner:
cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,width=1920,height=1080,format=YUY2,framerate=30/1 ! videoconvert ! video/x-raw,format=BGR ! appsink drop=1", cv2.CAP_GSTREAMER)
This now opens the stream and records the video. This is the same as the video recorded using the above GST-launch command. I cannot open with VLC on the Xavier but can play it with a different PC. Both of these however still have one issue. They are played back at 2x speed.
On windows, if I look at the video properties it shows frame rate of 15fps.
Printing the output of cv2.CAP_PROP_FPS gives me 30FPS.