Please provide complete information as applicable to your setup.
• Hardware Platform RTX3090
• DeepStream Version 6.0.1
• JetPack Version (valid for Jetson only)
• TensorRT Version 8.4
• NVIDIA GPU Driver Version 515.43.04
• Issue Type questions / new requirements
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details nvdspreprocess
I’m trying to create a custom input tensor for my model with shape 1;9;256;512. It should be made out of concatenation of 3 frames x 3 channels, hence dimension 9. I need to have all 3 RGB/BGR channels of the first frame to be followed by the next frame, i.e:
[R1, G1, B1; R2, G2, B2; R3, G3, B3].
I’ve followed this example: DeepStream 3D Action Recognition App — DeepStream 6.1.1 Release documentation (2D and 3D action recognition). Here is my preprocess config:
[property] gpu-id=0 enable=1 target-unique-ids=1 network-input-shape=1;9;256;512 processing-width=512 processing-height=256 # 0=NVBUF_MEM_DEFAULT 1=NVBUF_MEM_CUDA_PINNED 2=NVBUF_MEM_CUDA_DEVICE 3=NVBUF_MEM_CUDA_UNIFIED 4=NVBUF_MEM_SURFACE_ARRAY scaling-pool-memory-type=0 # 0=NvBufSurfTransformCompute_Default 1=NvBufSurfTransformCompute_GPU 2=NvBufSurfTransformCompute_VIC scaling-pool-compute-hw=0 scaling-buf-pool-size=8 tensor-buf-pool-size=8 scaling-filter=0 # custom-lib-path=/opt/nvidia/deepstream/deepstream-6.0/lib/gst-plugins/libcustom2d_preprocess.so # custom-tensor-preparation-function=CustomTensorPreparation #custom-lib-path=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_custom_sequence_preprocess.so custom-lib-path=../custom_sequence_preprocess/libnvds_custom_sequence_preprocess.so custom-tensor-preparation-function=CustomSequenceTensorPreparation #0=NCHW 1=NHWC 2=CUSTOM. We have NCHW network-input-order=2 #0=RGB 1=BGR 2=GRAY network-color-format=1 tensor-data-type=0 tensor-name=input_1:0 [user-configs] # pixel-normalization-factor=0.0039215697906911373 channel-scale-factors=0.0039215697906911373;0.0039215697906911373;0.0039215697906911373 #channel-scale-factors=0.007843137;0.007843137;0.007843137 #channel-mean-offsets=127.5;127.5;127.5 # channel-mean-offsets=0;0;0 stride=3 # stride=1 subsample=0 [group-0] src-ids=0 process-on-roi=0
I’ve tried two libraries to do preprocessing as shown in the config:
The first one rejects the config with error stating that dimensions should be 3 when RGB/BGR is being used. The second library runs fine but produces input tensor with channels in a different order from desired one, i.e:
[R1, R2, R3; G1, G2, G3; B1, B2, B3]. I have tested by dumping the tensor input into pgie and visualizing it.
How can I make preprocessing plugin to produce tensor in the desired order? Perhaps I need to patch the
custom_sequence_preprocess or is there any other way? Maybe I can use some custom color format or any other settings? Some help would be appreciated.