gstreamer NVMM <-> opencv gpuMat

For 1, you may read [url]https://developer.ridgerun.com/wiki/index.php/Embedded_GStreamer_Performance_Tuning[/url].

For 2, I don’t really understand your case… As the CV code is custom, you may use it in another C++ project…

Thanks again for your reply. What I meant is say I want to to write a c++ program that launches the camera, captures 100 camera frames and save them. how do I do that ?

You may check tegra MM API. Have a look to:

/home/nvidia/tegra_multimedia_api/samples/12_camera_v4l2_cuda

(If not in /home/nvidia, it may be in /home/ubuntu).

Thanks again Honey_Patouceul. I could not find there how you actually save the frames. However I am almost done with my application except for one thing. I would be grateful if you help me with it.

In my application I have a loop that at every iteration outputs a cuda::GpuMat. The data in it is in YUYV format. I want to save the frames into a file.
To do that I need to download that GpuMat into a cv::Mat, and then convert it to RGB, then I am using Gstreamer within OpenCV to save it into a file.

video.open("appsrc ! videoconvert ! omxh265enc ! matroskamux ! filesink location=anas.mkv", cv::CAP_GSTREAMER, 0, (double)25, pano_gpu.size(),true );   
for (;;)
{
..
pano_gpu.download(pano, stream);
cvtColor(pano, pano_rgb, COLOR_YUV2BGR_UYVY);
video.write(pano);
..
}

The problem her is that it is slow. My questions are

  1. can I use Gstreamer to save the GpuMat’s to file directly without downloading to cpu ? If so how?
  2. can I use Gstreamer to save to file with downloading to cpu but without converting the color ? If so how?
  3. the color code (COLOR_YUV2BGR_UYVY) is not implemented in CUDA::cvtcolor, so I can’t convert it on the gpu. Is there an alternative ?

Many thanks again

I think that filesystem access needs to be done from cpu…gpu has no knowledge of disk drivers and attached disks, AFAIK.
But you may not really need to download, if can you provide your own buffer for GpuMat (look for constructor with data ptr, it just creates a header with your buffer) allocated with Unified Memory. Such buffer can be accessed from both CPUs and GPU, with same address, so you don’t really need to copy, as the same RAM is shared by CPUs and GPU. Just create also a Mat on cpu side with same data pointer.

Filesink is only writing its input on disk, but has no restriction about its input format. Your pipeline does more, such as H265 encoding. Maybe point 3 would give you enough to move forward.

I wasn’t aware of this. Probably one format is available for conversion that matches one of videoconvert sink(input) capabilities:

gst-inspect-1.0 videoconvert
...
Pad Templates:
  SINK template: 'sink'
    Availability: Always
    Capabilities:
      video/x-raw
                 format: { I420, YV12, YUY2, UYVY, AYUV, RGBx, BGRx, xRGB, xBGR, RGBA, BGRA, ARGB, ABGR, RGB, BGR, Y41B, Y42B, YVYU, Y444, v210, v216, NV12, NV21, NV16, NV61, NV24, GRAY8, GRAY16_BE, GRAY16_LE, v308, RGB16, BGR16, RGB15, BGR15, UYVP, A420, RGB8P, YUV9, YVU9, IYU1, ARGB64, AYUV64, r210, I420_10LE, I420_10BE, I422_10LE, I422_10BE, Y444_10LE, Y444_10BE, GBR, GBR_10LE, GBR_10BE, NV12_64Z32, A420_10LE, A420_10BE, A422_10LE, A422_10BE, A444_10LE, A444_10BE }
                  width: [ 1, 2147483647 ]
                 height: [ 1, 2147483647 ]
              framerate: [ 0/1, 2147483647/1 ]
...

Hi Honey_Patoceul,

In context of questions is #44 and your response there after, I would like to ask -

  1. How can I allocated a unified memory pointer ? Like -

unsigned char ptr ;
unsigned int size = 1920
1080*1.5 ; // image would be YUV420/I420
cudaMallocManaged (ptr, size, cudaMemAttachGlobal);

  1. ‘Receive’ a frame from camera in this pointer (ptr) using Argus ?

  2. ‘Receive’ a frame from camera in this pointer (ptr) using Gstreamer/appsink ?

  3. After I have received the frame from camera in unified memory - can initialize, cv::Mat and cv::cuda::GpuMat with this pointer (like you suggest) and divide processing of frame between CPU and GPU.

Thanks,

For questions 1 to 3, sorry I am not the best person for answering this, and I’m currently very busy. More experienced users or nvidia folks may share their experience, otherwise you may search this forum or nvidia developer site.

Anyway, for question 4, don’t think that unified memory will allow to efficiently process the same memory location concurrently from GPU and CPU, AFAIK it would make page faults.
I would suggest to check Unified memory doc for details (there was a good slideshow for this in nvidia site but cannot find the link now) and split your data into separate memory locations for doing so.

Hi Honey_Patouceul,
Thanks, for #4 , I should clarify that I do not intend to touch unified mem pixels “concurrently” from CPU and GPU. I would like to process them one after another - such that I do not need to copy them around.
Thanks

Hi Honey_Patouceul,

I tried the code from post #19 https://devtalk.nvidia.com/default/topic/1022543/jetson-tx2/gstreamer-nvmm-lt-gt-opencv-gpumat/post/5208232/#5208232 but I keep getting the error

gst-launch-1.0: symbol lookup error: ./lib-gst-custom-opencv_cudaprocess.so: undefined symbol: _ZN2cv4cuda6GpuMatC1EiiiPvm

I’m using opencv 4.0.0 and had to change the namespace of the gpu functions, but the code is much the same.

/*
     * Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
     *
     * Redistribution and use in source and binary forms, with or without
     * modification, are permitted provided that the following conditions
     * are met:
     *  * Redistributions of source code must retain the above copyright
     *    notice, this list of conditions and the following disclaimer.
     *  * Redistributions in binary form must reproduce the above copyright
     *    notice, this list of conditions and the following disclaimer in the
     *    documentation and/or other materials provided with the distribution.
     *  * Neither the name of NVIDIA CORPORATION nor the names of its
     *    contributors may be used to endorse or promote products derived
     *    from this software without specific prior written permission.
     *
     * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
     * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
     * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
     * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR
     * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
     * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
     * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
     * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
     * OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
     * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
     * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
     */

    #include <stdio.h>
    #include <stdlib.h>

    #include <cuda.h>
    #include <opencv2/opencv.hpp>
    #include <opencv2/core/cuda.hpp>

    #include "cudaEGL.h"
    #if defined(__cplusplus)
    extern "C" void Handle_EGLImage (EGLImageKHR image);
    extern "C" {
    #endif

    typedef enum {
      COLOR_FORMAT_Y8 = 0,
      COLOR_FORMAT_U8_V8,
      COLOR_FORMAT_RGBA,
      COLOR_FORMAT_NONE
    } ColorFormat;

    typedef struct {
      /**
      * cuda-process API
      *
      * @param image   : EGL Image to process
      * @param userPtr : point to user alloc data, should be free by user
      */
      void (*fGPUProcess) (EGLImageKHR image, void ** userPtr);

      /**
      * pre-process API
      *
      * @param sBaseAddr  : Mapped Surfaces(YUV) pointers
      * @param smemsize   : surfaces size array
      * @param swidth     : surfaces width array
      * @param sheight    : surfaces height array
      * @param spitch     : surfaces pitch array
      * @param sformat    : surfaces format array
      * @param nsurfcount : surfaces count
      * @param userPtr    : point to user alloc data, should be free by user
      */
      void (*fPreProcess)(void **sBaseAddr,
                          unsigned int *smemsize,
                          unsigned int *swidth,
                          unsigned int *sheight,
                          unsigned int *spitch,
                          ColorFormat *sformat,
                          unsigned int nsurfcount,
                          void ** userPtr);

      /**
      * post-process API
      *
      * @param sBaseAddr  : Mapped Surfaces(YUV) pointers
      * @param smemsize   : surfaces size array
      * @param swidth     : surfaces width array
      * @param sheight    : surfaces height array
      * @param spitch     : surfaces pitch array
      * @param sformat    : surfaces format array
      * @param nsurfcount : surfaces count
      * @param userPtr    : point to user alloc data, should be free by user
      */
      void (*fPostProcess)(void **sBaseAddr,
                          unsigned int *smemsize,
                          unsigned int *swidth,
                          unsigned int *sheight,
                          unsigned int *spitch,
                          ColorFormat *sformat,
                          unsigned int nsurfcount,
                          void ** userPtr);
    } CustomerFunction;

    void init (CustomerFunction * pFuncs);

    #if defined(__cplusplus)
    }
    #endif


    /**
      * Dummy custom pre-process API implematation.
      * It just access mapped surface userspace pointer &
      * memset with specific pattern modifying pixel-data in-place.
      *
      * @param sBaseAddr  : Mapped Surfaces pointers
      * @param smemsize   : surfaces size array
      * @param swidth     : surfaces width array
      * @param sheight    : surfaces height array
      * @param spitch     : surfaces pitch array
      * @param nsurfcount : surfaces count
      */
    static void
    pre_process (void **sBaseAddr,
                    unsigned int *smemsize,
                    unsigned int *swidth,
                    unsigned int *sheight,
                    unsigned int *spitch,
                    ColorFormat  *sformat,
                    unsigned int nsurfcount,
                    void ** usrptr)
    {
      /* add your custom pre-process here */
    }

    /**
      * Dummy custom post-process API implematation.
      * It just access mapped surface userspace pointer &
      * memset with specific pattern modifying pixel-data in-place.
      *
      * @param sBaseAddr  : Mapped Surfaces pointers
      * @param smemsize   : surfaces size array
      * @param swidth     : surfaces width array
      * @param sheight    : surfaces height array
      * @param spitch     : surfaces pitch array
      * @param nsurfcount : surfaces count
      */
    static void
    post_process (void **sBaseAddr,
                    unsigned int *smemsize,
                    unsigned int *swidth,
                    unsigned int *sheight,
                    unsigned int *spitch,
                    ColorFormat  *sformat,
                    unsigned int nsurfcount,
                    void ** usrptr)
    {
      /* add your custom post-process here */
    }



    static void cv_process(void *pdata, int32_t width, int32_t height)
    {
        /* Create a GpuMat with data pointer */
        cv::cuda::GpuMat d_mat(height, width, CV_8UC4, pdata);
        
        /* Apply Sobel filter */
        cv::Sobel(d_mat, d_mat, CV_8UC4, 1, 1, 1, 1, 1, cv::BORDER_DEFAULT);
    }

    /**
      * Performs CUDA Operations on egl image.
      *
      * @param image : EGL image
      */
    static void
    gpu_process (EGLImageKHR image, void ** usrptr)
    {
      CUresult status;
      CUeglFrame eglFrame;
      CUgraphicsResource pResource = NULL;

      cudaFree(0);
      status = cuGraphicsEGLRegisterImage(&pResource, image, CU_GRAPHICS_MAP_RESOURCE_FLAGS_NONE);
      if (status != CUDA_SUCCESS) {
        printf("cuGraphicsEGLRegisterImage failed : %d \n", status);
        return;
      }

      status = cuGraphicsResourceGetMappedEglFrame( &eglFrame, pResource, 0, 0);
      if (status != CUDA_SUCCESS) {
        printf ("cuGraphicsSubResourceGetMappedArray failed\n");
      }

      status = cuCtxSynchronize();
      if (status != CUDA_SUCCESS) {
        printf ("cuCtxSynchronize failed \n");
      }

      if (eglFrame.frameType == CU_EGL_FRAME_TYPE_PITCH) {
        if (eglFrame.eglColorFormat == CU_EGL_COLOR_FORMAT_RGBA) {
    	/* Apply CV gpu processing */
    	cv_process(eglFrame.frame.pPitch[0], eglFrame.width, eglFrame.height);
        } else {
    	printf ("Invalid eglcolorformat for opencv\n");
        }
      }
      else {
         printf ("Invalid frame type for opencv\n");
      }

      status = cuCtxSynchronize();
      if (status != CUDA_SUCCESS) {
        printf ("cuCtxSynchronize failed after memcpy \n");
      }

      status = cuGraphicsUnregisterResource(pResource);
      if (status != CUDA_SUCCESS) {
        printf("cuGraphicsEGLUnRegisterResource failed: %d \n", status);
      }
    }

    extern "C" void
    init (CustomerFunction * pFuncs)
    {
      pFuncs->fPreProcess = pre_process;
      pFuncs->fGPUProcess = gpu_process;
      pFuncs->fPostProcess = post_process;
    }

Thanks in advance for any help

What gives:

ldd ./lib-gst-custom-opencv_cudaprocess.so

If it shows libopencv_core is not found, you may have to add opencv libs path into environment variable LD_LIBRARY_PATH.

It gives

linux-vdso.so.1 =>  (0x0000007f7f328000)
	libcuda.so.1 => /usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1 (0x0000007f7e8c0000)
	librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f7e8a9000)
	libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f7e87d000)
	libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f7e869000)
	libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f7e6da000)
	libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f7e6b9000)
	libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f7e571000)
	/lib/ld-linux-aarch64.so.1 (0x0000005576d20000)
	libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f7e4c4000)
	libnvrm_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so (0x0000007f7e48f000)
	libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f7e456000)
	libnvidia-fatbinaryloader.so.28.2.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvidia-fatbinaryloader.so.28.2.0 (0x0000007f7e3f1000)
	libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f7e3d3000)

I created the environment variable

echo $LD_LIBRARY_PATH
/usr/local/lib/

and I get the same output as above. I also edited the paths in the makefile to

# Location of the CUDA Toolkit
CUDA_PATH ?= /usr/local/cuda-9.0
INCLUDE_DIR = /usr/local/include/opencv4
LIB_DIR = /usr/local/lib
TEGRA_LIB_DIR = /usr/lib/aarch64-linux-gnu/tegra

and still the same. Is there another possible issue?

Not sure what is the cause, but it seems something is wrong…Your custom lib has no dependency to opencv.

Looking further at the code you showed, it was for opencv-2.4…For opencv3 there was another version in post #23.

Furthermore, I have not tested this example on R28.2.1 version with nvarguscamerasrc, quickly tried with R31.1 on Xavier but I haven’t yet succeeded.

I got it to compile with the example from post #23 but I edited the make file includes to be

# Location of the CUDA Toolkit
CUDA_PATH ?= /usr/local/cuda-9.0
INCLUDE_DIR = /usr/local/include/opencv4
LIB_DIR = /usr/local/lib
TEGRA_LIB_DIR = /usr/lib/aarch64-linux-gnu/tegra

# This is typical install path of opencv4tegra
OPENCV_DIR = /usr/local

and ran the pipeline with

gst-launch-1.0 nvcamerasrc fpsRange="30 30" sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name=./lib-gst-custom-opencv_cudaprocess.so ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! nvegltransform ! nveglglessink

and it prints

Invalid eglcolorformat for opencv
Invalid eglcolorformat for opencv
Invalid eglcolorformat for opencv
Invalid eglcolorformat for opencv
Invalid eglcolorformat for opencv
Invalid eglcolorformat for opencv
Invalid eglcolorformat for opencv

repeatedly. If I set format=(string)RGBA, the error doesn’t print, but either way, it doesn’t apply the sobel filter to the video and prints

Pipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock

If you want to do opencv processing, BGRA format is probably what you would use because most of opencv algorithms are for RGB format.
So you’d try such pipeline:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), format=I420' ! nvivafilter customer-lib-name="./lib-gst-custom-opencv_cudaprocess.so" cuda-process=true ! 'video/x-raw(memory:NVMM), format=RGBA' ! nvegltransform ! nveglglessink

I did try that pipeline

gst-launch-1.0 nvcamerasrc fpsRange="30 30" sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name=./lib-gst-custom-opencv_cudaprocess.so ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! nvegltransform ! nveglglessink

but the pipeline doesn’t seem to apply the sobel filter to the video

Possibly it failed to load your custom lib and in such case it may silently fall back to /usr/lib/aarch64-linux-gnu/libnvsample_cudaprocess.so.
Check with ldd if your custom lib can load all its dependencies.

Running

ldd ./lib-gst-custom-opencv_cudaprocess.so

returns

linux-vdso.so.1 =>  (0x0000007f78d6d000)
        libopencv_core.so.4.0 => /usr/local/lib/libopencv_core.so.4.0 (0x0000007f7871e000)
        libopencv_cudafilters.so.4.0 => /usr/local/lib/libopencv_cudafilters.so.4.0 (0x0000007f74c96000)
        libcuda.so.1 => /usr/lib/aarch64-linux-gnu/tegra/libcuda.so.1 (0x0000007f742b8000)
        librt.so.1 => /lib/aarch64-linux-gnu/librt.so.1 (0x0000007f742a0000)
        libpthread.so.0 => /lib/aarch64-linux-gnu/libpthread.so.0 (0x0000007f74274000)
        libdl.so.2 => /lib/aarch64-linux-gnu/libdl.so.2 (0x0000007f74261000)
        libstdc++.so.6 => /usr/lib/aarch64-linux-gnu/libstdc++.so.6 (0x0000007f740d1000)
        libgcc_s.so.1 => /lib/aarch64-linux-gnu/libgcc_s.so.1 (0x0000007f740b0000)
        libc.so.6 => /lib/aarch64-linux-gnu/libc.so.6 (0x0000007f73f69000)
        /lib/ld-linux-aarch64.so.1 (0x000000555da22000)
        libGL.so.1 => /usr/lib/aarch64-linux-gnu/tegra/libGL.so.1 (0x0000007f73e68000)
        libcudart.so.9.0 => /usr/local/cuda-9.0/lib64/libcudart.so.9.0 (0x0000007f73e03000)
        libz.so.1 => /lib/aarch64-linux-gnu/libz.so.1 (0x0000007f73ddc000)
        liblapack.so.3 => /usr/lib/liblapack.so.3 (0x0000007f73963000)
        libcblas.so.3 => /usr/lib/libcblas.so.3 (0x0000007f73938000)
        libm.so.6 => /lib/aarch64-linux-gnu/libm.so.6 (0x0000007f7388b000)
        libopencv_cudaarithm.so.4.0 => /usr/local/lib/libopencv_cudaarithm.so.4.0 (0x0000007f71fa5000)
        libopencv_imgproc.so.4.0 => /usr/local/lib/libopencv_imgproc.so.4.0 (0x0000007f71ba3000)
        libnppc.so.9.0 => /usr/local/cuda-9.0/lib64/libnppc.so.9.0 (0x0000007f71b29000)
        libnppif.so.9.0 => /usr/local/cuda-9.0/lib64/libnppif.so.9.0 (0x0000007f6dfcd000)
        libnppim.so.9.0 => /usr/local/cuda-9.0/lib64/libnppim.so.9.0 (0x0000007f6da96000)
        libnvrm_gpu.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so (0x0000007f6da61000)
        libnvrm.so => /usr/lib/aarch64-linux-gnu/tegra/libnvrm.so (0x0000007f6da28000)
        libnvidia-fatbinaryloader.so.28.2.0 => /usr/lib/aarch64-linux-gnu/tegra/libnvidia-fatbinaryloader.so.28.2.0 (0x0000007f6d9c3000)
        libGLX.so.0 => /usr/lib/aarch64-linux-gnu/tegra/libGLX.so.0 (0x0000007f6d980000)
        libGLdispatch.so.0 => /usr/lib/aarch64-linux-gnu/tegra/libGLdispatch.so.0 (0x0000007f6d854000)
        libopenblas.so.0 => /usr/lib/libopenblas.so.0 (0x0000007f6d1f5000)
        libgfortran.so.3 => /usr/lib/aarch64-linux-gnu/libgfortran.so.3 (0x0000007f6d106000)
        libatlas.so.3 => /usr/lib/libatlas.so.3 (0x0000007f6cee6000)
        libnppial.so.9.0 => /usr/local/cuda-9.0/lib64/libnppial.so.9.0 (0x0000007f6c463000)
        libnppidei.so.9.0 => /usr/local/cuda-9.0/lib64/libnppidei.so.9.0 (0x0000007f6bc9a000)
        libnppig.so.9.0 => /usr/local/cuda-9.0/lib64/libnppig.so.9.0 (0x0000007f69ff5000)
        libnppist.so.9.0 => /usr/local/cuda-9.0/lib64/libnppist.so.9.0 (0x0000007f69005000)
        libnppitc.so.9.0 => /usr/local/cuda-9.0/lib64/libnppitc.so.9.0 (0x0000007f68d0d000)
        libcublas.so.9.0 => /usr/local/cuda-9.0/lib64/libcublas.so.9.0 (0x0000007f66edc000)
        libcufft.so.9.0 => /usr/local/cuda-9.0/lib64/libcufft.so.9.0 (0x0000007f5fe02000)
        libnvos.so => /usr/lib/aarch64-linux-gnu/tegra/libnvos.so (0x0000007f5fde4000)
        libX11.so.6 => /usr/lib/aarch64-linux-gnu/libX11.so.6 (0x0000007f5fcbc000)
        libXext.so.6 => /usr/lib/aarch64-linux-gnu/libXext.so.6 (0x0000007f5fc9b000)
        libxcb.so.1 => /usr/lib/aarch64-linux-gnu/libxcb.so.1 (0x0000007f5fc70000)
        libXau.so.6 => /usr/lib/aarch64-linux-gnu/libXau.so.6 (0x0000007f5fc5c000)
        libXdmcp.so.6 => /usr/lib/aarch64-linux-gnu/libXdmcp.so.6 (0x0000007f5fc47000)

which looks like it’s loading the correct dependencies, but still not applying the sobel filter to the video pipeline. The full output to std from

gst-launch-1.0 nvcamerasrc fpsRange="30 30" sensor-id=1 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name=./lib-gst-custom-opencv_cudaprocess.so ! 'video/x-raw(memory:NVMM), format=(string)RGBA' ! nvegltransform ! nveglglessink

is

Setting pipeline to PAUSED ...

Using winsys: x11

Available Sensor modes :
4056 x 3040 FR=15.000000 CF=0x1009208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2028 x 1520 FR=60.000000 CF=0x1009208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1920 x 1080 FR=60.000000 CF=0x1009208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Setting pipeline to PLAYING ...
New clock: GstSystemClock

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 2 WxH = 1920x1080 FrameRate = 60.000000 ...

Hi,
We have verified HoneyP’s sample on r28.2.1/TX2. Please refer to the steps:
1 Do not install OpenCV 3.3.1 via Jetpack. It is installed by default. Please un-check OpenCV 3.3.1
2 Get the script https://github.com/AastaNV/JEP/blob/master/script/install_opencv3.4.0_TX2.sh
3 Execute the script

$ mkdir OpenCV
$ ./install_opencv3.4.0_TX2.sh OpenCV

4 Apply the patch and rebuild, replace libnvsample_cudaprocess.so
5 Run

$ gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM)' ! nvivafilter customer-lib-name=libnvsample_cudaprocess.so cuda-process=true ! 'video/x-raw(memory:NVMM),format=(string)RGBA' ! nvoverlaysink

Thank HoneyP’s active help always.

0001-nvsample_cudaprocess-hook-cv3-gpuMat.zip (1.29 KB)