# Timeshift Encode/Decode and OCV pipeline **URL:** \ **Category:** Jetson TX2\ **Created:** [August 24, 2017, 9:46am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861 "2017-08-24T09:46:47Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![dumbogeorge](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@dumbogeorge](https://forums.developer.nvidia.com/u/dumbogeorge)\ **Post date:** [August 24, 2017, 9:46am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/1 "2017-08-24T09:46:47Z")
Hi Folks, I am looking to create ‘time shift pipeline’ using Gstreamer. This refers to actual recording of scene using cameras, putting the resultant bitstream on disk , and then re-reading the same video (after some delay) for further OpenCV processing, while recording is going on. We are looking to record videos on tegra hardware (Tx1/Tx2) and then process the recorded videos further. Could someone please give any pointer/gstreamer code/Argus/Visionworks example for a time shift pipe ? Thanks ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [August 25, 2017, 6:06am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/2 "2017-08-25T06:06:54Z")
Hi, For encoding, I think you can use hlssink gst-launch-1.0 nvcamerasrc num-buffers=300 ! ‘video/x-raw(memory:NVMM),width=1280,height=720’ ! omxh264enc ! mpegtsmux ! hlssink It encodes into multiple files every 15 second(by default). You can process, delete, and process the next video files. ---
**Author:** ![dumbogeorge](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@dumbogeorge](https://forums.developer.nvidia.com/u/dumbogeorge)\ **Post date:** [August 31, 2017, 6:36pm UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/3 "2017-08-31T18:36:05Z")
Hi DaneLLL Any idea how to install hlssink on Tx2 ? I ran following command line - ```auto ubuntu@tegra-ubuntu:~/work/Experiment$ gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh265enc ! 'video/x-h265, stream-format=(string)byte-stream' ! h265parse ! matroskamux !hlssink bash: !hlssink: event not found ``` Thanks ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 1, 2017, 2:08am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/4 "2017-09-01T02:08:21Z")
hlssink should be in gstreamer1.0-plugins-bad ---
**Author:** ![dumbogeorge](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@dumbogeorge](https://forums.developer.nvidia.com/u/dumbogeorge)\ **Post date:** [September 1, 2017, 1:08pm UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/5 "2017-09-01T13:08:31Z")
Hi DaneLLL I downloaded gstreamer1.0-plugins-bad and I do not see hlssink in there. ```auto sudo find / -name '*sink' /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tools/element-templates/basesink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tools/element-templates/videosink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tools/element-templates/audiosink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tests/examples/avsamplesink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tests/examples/waylandsink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tests/examples/waylandsink/gtkwaylandsink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tests/examples/waylandsink/.libs/gtkwaylandsink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tests/examples/gtk/gtksink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/tests/examples/gtk/gtkglsink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/sys/d3dvideosink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/sys/dshowvideosink /home/ubuntu/Downloads/gst-plugins-bad-1.8.3/ext/apexsink /home/ubuntu/.config/pulse/ae15719763c84b35196c20a95728b806-default-sink ``` ```auto sudo find / -name '*sink' ``` ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 7, 2017, 8:37am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/6 "2017-09-07T08:37:16Z")
Steps to install gstreamer1.0-plugins-bad [url][https://devtalk.nvidia.com/default/topic/1023577/jetson-tx2/hlssink/post/5207537/#5207537[/url]](https://devtalk.nvidia.com/default/topic/1023577/jetson-tx2/hlssink/post/5207537/#5207537%5B/url%5D) ---
**Author:** ![dumbogeorge](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@dumbogeorge](https://forums.developer.nvidia.com/u/dumbogeorge)\ **Post date:** [September 7, 2017, 6:41pm UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/7 "2017-09-07T18:41:45Z")
Thanks DaneLLL I am able to install and run hlssink. Going back to my original topic of timeshift, I am looking to encode from camera, write in a file , and then read it back to decode. Is there a way command line like this can work ? ```auto gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh265enc ! 'video/x-h265, stream-format=(string)byte-stream' ! h265parse ! matroskamux ! filesink location=/home/ubuntu/junk.mkv filesrc location=/home/ubuntu/junk.mkv ! decodebin ! nvoverlaysink -e ``` When I tried encode followed by decode without file - it seems to work - although with lots of jerks/frame drops … ```auto gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh265enc ! 'video/x-h265, stream-format=(string)byte-stream' ! h265parse ! matroskamux ! decodebin ! nvoverlaysink -e ``` With aforesaid command line I see frame drops and following messages … ```auto ubuntu@tegra-ubuntu:~$ gst-launch-1.0 -e nvcamerasrc fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! omxh265enc ! 'video/x-h265, stream-format=(string)byte-stream' ! h265parse ! matroskamux ! decodebin ! nvoverlaysink -e Setting pipeline to PAUSED ... Available Sensor modes : 3864 x 2174 FR=60.000000 CF=0x1009208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10 Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 0 WxH = 3864x2174 FrameRate = 60.000000 ... New clock: GstSystemClock Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 8 ===== MSENC ===== NvMMLiteBlockCreate : Block : BlockType = 8 ===== NVENC blits (mode: 1) into block linear surfaces ===== NvMMLiteOpen : Block : BlockType = 279 TVMR: NvMMLiteTVMRDecBlockOpen: 7907: NvMMLiteBlockOpen NvMMLiteBlockCreate : Block : BlockType = 279 TVMR: cbBeginSequence: 1223: BeginSequence 1920x1088, bVPR = 0 TVMR: LowCorner Frequency = 180000 TVMR: cbBeginSequence: 1622: DecodeBuffers = 3, pnvsi->eCodec = 10, codec = 9 TVMR: cbBeginSequence: 1693: Display Resolution : (1920x1080) TVMR: cbBeginSequence: 1694: Display Aspect Ratio : (1920x1080) TVMR: cbBeginSequence: 1762: ColorFormat : 5 TVMR: cbBeginSequence:1776 ColorSpace = NvColorSpace_YCbCr601 TVMR: cbBeginSequence: 1904: SurfaceLayout = 3 TVMR: cbBeginSequence: 2005: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5 TVMR: cbBeginSequence: 2007: BeginSequence ColorPrimaries = 2, TransferCharacteristics = 2, MatrixCoefficients = 2 Allocating new output: 1920x1088 (x 10), ThumbnailMode = 0 OPENMAX: HandleNewStreamFormat: 3464: Send OMX_EventPortSettingsChanged : nFrameWidth = 1920, nFrameHeight = 1088 TVMR: FrameRate = 1000 TVMR: NVDEC LowCorner Freq = (576000 * 1024) WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped. Additional debug info: gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: There may be a timestamping problem, or this computer is too slow. ---> TVMR: Video-conferencing detected !!!!!!!!! WARNING: from element /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: A lot of buffers are being dropped. Additional debug info: gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0: There may be a timestamping problem, or this computer is too slow. ^Chandling interrupt. Interrupt: Stopping pipeline ... EOS on shutdown enabled -- Forcing EOS on the pipeline Waiting for EOS... TVMR: NvMMLiteTVMRDecDoWork: 6768: NVMMLITE_TVMR: EOS detected TVMR: TVMRBufferProcessing: 5723: Processing of EOS TVMR: TVMRBufferProcessing: 5800: Processing of EOS Done Got EOS from element "pipeline0". EOS received - stopping pipeline... Execution ended after 0:00:04.030194830 Setting pipeline to PAUSED ... Setting pipeline to READY ... TVMR: TVMRFrameStatusReporting: 6369: Closing TVMR Frame Status Thread ------------- TVMR: TVMRVPRFloorSizeSettingThread: 6179: Closing TVMRVPRFloorSizeSettingThread ------------- TVMR: TVMRFrameDelivery: 6219: Closing TVMR Frame Delivery Thread ------------- TVMR: NvMMLiteTVMRDecBlockClose: 8105: Done Setting pipeline to NULL ... Freeing pipeline ... ``` What would be right way to encode → to a file → read from the same file → decode → feed to opencv pipe using code given by you here … [https://devtalk.nvidia.com/default/topic/1011376/jetson-tx1/gstreamer-decode-live-video-stream-with-the-delay-difference-between-gst-launch-1-0-command-and-appsink-callback/post/5160929/#5160929](https://devtalk.nvidia.com/default/topic/1011376/jetson-tx1/gstreamer-decode-live-video-stream-with-the-delay-difference-between-gst-launch-1-0-command-and-appsink-callback/post/5160929/#5160929) Thanks, ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 11, 2017, 1:49am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/8 "2017-09-11T01:49:32Z")
Hi dumbogeorge, I am not sure if it works by reading/writing the same file, so I suggest use hlssink. Some other users may share experience. If you don’t need it to be mp4/mkv, you can consider use tegra\_multimedia\_api. Have a thread store into multiple h264 files. Have another thread read the files for analysis and then delete the processed files. ---
**Author:** ![dumbogeorge](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@dumbogeorge](https://forums.developer.nvidia.com/u/dumbogeorge)\ **Post date:** [September 11, 2017, 11:50am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/9 "2017-09-11T11:50:24Z")
Hi DaneLLL Just thinking out loud, how to feed multiple files generated by hlssink to decoder element in subsequent part of gstreamer pipeline ? Would the decoderbin not complain if a data from file\_00 is interrupted midway and later supplied from file\_01 ? I do need mp4/mkv and cannot delete the intermediate recorded files. But would like to explore generation of raw H264/H265 bitstreams (without mp4/mkv container) such that I can try to use tegra\_multimedia\_api examples. How would encoder generate raw bitstreams - any example gstreamer pipelines ? Thanks, ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 12, 2017, 7:14am UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/10 "2017-09-12T07:14:18Z")
Hi dumbogeorge, tegra\_multimedia\_api is not gstreamer based. You can install samples via Jetpack and check tegra\_multimedia\_api/samples/00\_video\_decode/ tegra\_multimedia\_api/samples/01\_video\_encode/ Looks like hlssink is not 100% fit your case. Maybe you can try to get help from [http://gstreamer-devel.966125.n4.nabble.com/](http://gstreamer-devel.966125.n4.nabble.com/) ---
**Author:** ![nadeemm](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/nadeemm/32/367020_2.png) [@nadeemm](https://forums.developer.nvidia.com/u/nadeemm)\ **Post date:** [October 18, 2021, 6:26pm UTC](https://forums.developer.nvidia.com/t/timeshift-encode-decode-and-ocv-pipeline/52861/11 "2021-10-18T18:26:54Z")