Jetson Nano CSI Raspberry Pi Camera V2 upside down video when run an example with deepstream-app

Hi guys, I’m trying to run deepstream-app with the example “source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt” on Jetson Nano with “deepstream-4.0_4.0-1_arm64.deb”, using a Raspberry Pi V2 camera NoIr, but the image appears face-up. anybody can help me?

I modified the config file:
[source0]
enable = 1
#Type - 1 = CameraV4L2 2 = URI 3 = MultiURI 4 = RTSP 5 = Camera (CSI) (Jetson only)
type = 5
intra-decode-enable = 1
gpu-id = 0
camera-id = 0
camera-width = 1920
camera-height = 1280
camera-fps-n = 30
camera-fps-d = 1
camera-csi-sensor-id = 0
drop-frame-interval = 0

but, when i run, the image appears upside down. On the other hand, when executing the jetson inference example on the same computer
~/git/jetson-inference/build/aarch64/bin ./camera-viewer
the image appears ok and in the pipeline show
“nvvidconv flip-method = 2”. Then the question is:

How can I indicate the orientation of the camera in the configuration file to run deepstream-app?

JetPack Version: 4.2.2

Thanks.

Version JetPack 4.2.2

Hi,
It does not support ‘flip-method’ property in nvvideoconvert. Please modify create_camera_source_bin() to run

nvarguscamerasrc bufapi-version=0 ! nvvidconv flip-method=2 ! video/x-raw,forma=NV12 ! nvvideoconvert ! video/x-raw(memory:NVMM),format=NV12

Hello, I am just learning about Jetson Nano DK, I tried several ways to program as indicated to solve the problem of turning the image, but I could not move forward. Is there any way to see the string generated by the create_camera_source_bin program? That is, there is some method to query / emulate the gstreamer command generated by a program.
The attached contains the source code modified, but, when execute them the mesage is:

<li></li>
./deepstream-app -c ../../../../samples/configs/deepstream-app/source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano-csiCam-test01.txt --gst-debug=3 2>&1 | tee debug.txt 
0:00:00.242025044 10454   0x557ee9c290 WARN                     omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/mherrera/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)

(deepstream-app:10454): GLib-GObject-WARNING **: 02:31:43.653: cannot register existing type 'GstInterpolationMethod'

(deepstream-app:10454): GLib-GObject-CRITICAL **: 02:31:43.654: g_param_spec_enum: assertion 'G_TYPE_IS_ENUM (enum_type)' failed

(deepstream-app:10454): GLib-GObject-CRITICAL **: 02:31:43.654: validate_pspec_to_install: assertion 'G_IS_PARAM_SPEC (pspec)' failed
0:00:11.051427092 10454   0x557ef64850 FIXME                default gstutils.c:3981:gst_pad_create_stream_id_internal:<src_elem:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Creating LL OSD context new

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.

**PERF: FPS 0 (Avg)	
**PERF: 0.00 (0.00)	
** INFO: <bus_callback:163>: Pipeline ready

** INFO: <bus_callback:149>: Pipeline running

Creating LL OSD context new
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3264 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3264 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: PowerService: requested_clock_Hz=627200000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
nvbuf_utils: nvbuffer Payload Type not supported
NvBufferGetParams failed for src_dmabuf_fd
nvbuffer_transform Failed
gst_nvvconv_transform: NvBufferTransform Failed 
0:00:11.977186035 10454   0x557ef64850 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<src_elem> error: Internal data stream error.
0:00:11.977212963 10454   0x557ef64850 WARN                 basesrc gstbasesrc.c:3055:gst_base_src_loop:<src_elem> error: streaming stopped, reason error (-5)
ERROR from src_elem: Internal data stream error.
Debug info: gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstNvArgusCameraSrc:src_elem:
streaming stopped, reason error (-5)
Quitting
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
App run failed
GST_ARGUS: 
PowerServiceHwVic::cleanupResources
mherrera@JN01EXPL:/opt/nvidia/deepstream/deepstream-4.0/sources/apps/sample_apps/deepstream-app$ 
<li></li>

Thanks
deepstream_source_bin.c (23.7 KB)

Hi,
It should be ‘bufapi-version=FALSE’.

g_object_set (G_OBJECT (bin->src_elem), "bufapi-version", <b>FALSE</b>, NULL);

Hello Dane:
When update the code with

g_object_set (G_OBJECT (bin->src_elem), "bufapi-version", FALSE, NULL);

the message is

ERROR from src_bin_muxer: Input buffer number of surfaces (0) must be equal to mux->num_surfaces_per_frame (1)

For reference, i’m attached the output file and deepstream_source_bin.c.

Obs: I tried different code updates to the deepstream_source_bin.c without success. Finlay, i back to the original code and update only with the previous recommendation and the error message is the same.

Obs2: I restarted with a new installation of the operating system (and Deepstream SDK 4.0.1, that is, the tests I mentioned in this post are with 4.0.1

Regards,

deepstream_source_bin.c (23.8 KB)
20191004-deepstream-app-RBPV2Cam.log (2.79 KB)

Hi,
Please use NvBufSurface APIs. Below is a patch for reference:
sources\apps\apps-common\src\deepstream_source_bin.c:

@@ -22,6 +22,8 @@
 
 #include <string.h>
 
+#include "nvbufsurface.h"
+#include "nvbufsurftransform.h"
 #include "gstnvdsmeta.h"
 #include "deepstream_common.h"
 #include "deepstream_sources.h"
@@ -62,6 +64,64 @@ set_camera_v4l2_params (NvDsSourceConfig * config, NvDsSrcBin * bin)
   return TRUE;
 }
 
+static GstPadProbeReturn
+nvargus_src_pad_buffer_probe (GstPad * pad, GstPadProbeInfo * info,
+    gpointer u_data)
+{
+    GstBuffer *buf = (GstBuffer *) info->data;
+    GstMapInfo outmap = GST_MAP_INFO_INIT;
+    gst_buffer_map (buf, &outmap, GST_MAP_WRITE);
+    NvBufSurface*  surface = (NvBufSurface *)outmap.data;
+
+    NvBufSurfTransformRect src_rect, dst_rect;
+    src_rect.top   = 0;
+    src_rect.left  = 0;
+    src_rect.width = (guint) surface->surfaceList[0].width;
+    src_rect.height= (guint) surface->surfaceList[0].height;
+
+    dst_rect.top   = 0;
+    dst_rect.left  = 0;
+    dst_rect.width = (guint) surface->surfaceList[0].width;
+    dst_rect.height= (guint) surface->surfaceList[0].height;
+
+    NvBufSurface *dst_surface = NULL;
+    NvBufSurfaceCreateParams nvbufsurface_create_params;
+
+    nvbufsurface_create_params.gpuId  = surface->gpuId;
+    nvbufsurface_create_params.width  = (gint) surface->surfaceList[0].width;
+    nvbufsurface_create_params.height = (gint) surface->surfaceList[0].height;
+    nvbufsurface_create_params.size = 0;
+    nvbufsurface_create_params.colorFormat = surface->surfaceList[0].colorFormat;
+    nvbufsurface_create_params.layout = surface->surfaceList[0].layout;
+    nvbufsurface_create_params.memType = surface->memType;
+
+    NvBufSurfaceCreate(&dst_surface,1,&nvbufsurface_create_params);
+
+    NvBufSurfTransformParams nvbufsurface_params;
+    nvbufsurface_params.src_rect = &src_rect;
+    nvbufsurface_params.dst_rect = &dst_rect;
+    nvbufsurface_params.transform_flag =  0;
+    nvbufsurface_params.transform_filter = NvBufSurfTransformInter_Default;
+
+    NvBufSurfTransformConfigParams transform_config_params;
+    NvBufSurfTransform_Error err;
+
+    transform_config_params.compute_mode = NvBufSurfTransformCompute_Default;
+    transform_config_params.gpu_id = surface->gpuId;
+    transform_config_params.cuda_stream = NULL;
+    err = NvBufSurfTransformSetSessionParams (&transform_config_params);
+    // copy to dst_surface
+    err = NvBufSurfTransform (surface, dst_surface, &nvbufsurface_params);
+    // rototate 180 degree to original surface
+    nvbufsurface_params.transform_flag =  NVBUFSURF_TRANSFORM_FLIP;
+    nvbufsurface_params.transform_flip = NvBufSurfTransform_Rotate180;
+    err = NvBufSurfTransform (dst_surface, surface, &nvbufsurface_params);
+    NvBufSurfaceDestroy(dst_surface);
+
+    gst_buffer_unmap (buf, &outmap);
+    return GST_PAD_PROBE_OK;
+}
+
 static gboolean
 create_camera_source_bin (NvDsSourceConfig * config, NvDsSrcBin * bin)
 {
@@ -144,12 +204,16 @@ create_camera_source_bin (NvDsSourceConfig * config, NvDsSrcBin * bin)
     NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");
 
   } else {
+    GstPad *src_pad;
 
     g_object_set (G_OBJECT (bin->cap_filter), "caps", caps, NULL);
 
     gst_bin_add_many (GST_BIN (bin->bin), bin->src_elem, bin->cap_filter, NULL);
 
     NVGSTDS_LINK_ELEMENT (bin->src_elem, bin->cap_filter);
+    src_pad = gst_element_get_static_pad (bin->src_elem, "src");
+    gst_pad_add_probe (src_pad, GST_PAD_PROBE_TYPE_BUFFER,
+        nvargus_src_pad_buffer_probe, NULL, NULL);
 
     NVGSTDS_BIN_ADD_GHOST_PAD (bin->bin, bin->cap_filter, "src");
   }

sources\apps\sample_apps\deepstream-app\Makefile

@@ -24,6 +24,7 @@ APP:= deepstream-app
 
 TARGET_DEVICE = $(shell gcc -dumpmachine | cut -f1 -d -)
 
+CUDA_VER:=10.0
 NVDS_VERSION:=4.0
 
 LIB_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/lib/
@@ -41,10 +42,10 @@ PKGS:= gstreamer-1.0 gstreamer-video-1.0 x11
 
 OBJS:= $(SRCS:.c=.o)
 
-CFLAGS+= -I../../apps-common/includes -I../../../includes -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=4
+CFLAGS+= -I../../apps-common/includes -I../../../includes -DDS_VERSION_MINOR=0 -DDS_VERSION_MAJOR=4 -I /usr/local/cuda-$(CUDA_VER)/include
 
 LIBS+= -L$(LIB_INSTALL_DIR) -lnvdsgst_meta -lnvds_meta -lnvdsgst_helper -lnvds_utils -lm \
-       -lgstrtspserver-1.0 -lgstrtp-1.0 -Wl,-rpath,$(LIB_INSTALL_DIR)
+       -lgstrtspserver-1.0 -lgstrtp-1.0 -Wl,-rpath,$(LIB_INSTALL_DIR) -lnvbufsurface -lnvbufsurftransform
 
 CFLAGS+= `pkg-config --cflags $(PKGS)`
2 Likes