NvBufSurfTransform failed with error -2 while converting buffer

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU) Jetson Xavier
• DeepStream Version 5.0 GA
• JetPack Version (valid for Jetson only) 4.4
• TensorRT Version 7.1.3-1
• Issue Type( questions, new requirements, bugs) Bug
• How to reproduce the issue ? Attached
**• Requirement details **

I am running deepstream-app with pgie (config-infer-primary) and sgie (pgie-config-fd-lpd) but after running for some time, its getting frozen with the following error:

$ ./deepstream-app -c /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.txt
Unknown or legacy key specified 'net-stride' for group [property]
Unknown or legacy key specified 'parse-func' for group [property]
Warn: 'threshold' parameter has been deprecated. Use 'pre-cluster-threshold' instead.

Using winsys: x11 
0:00:00.176642409 18646   0x559c671600 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 4]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1715> [UID = 4]: Trying to create engine from model files
Warning, setting batch size to 1. Update the dimension after parsing due to using explicit batch size.
INFO: [TRT]: 
INFO: [TRT]: --------------- Layers running on DLA: 
INFO: [TRT]: 
INFO: [TRT]: --------------- Layers running on GPU: 
INFO: [TRT]: conv1_branch_2 + activation_1/Relu_branch_2, conv1_branch_1 + activation_1/Relu_branch_1, block_1a_conv_1_branch_1 + activation_2/Relu_branch_1, block_1a_conv_1_branch_2 + activation_2/Relu_branch_2, block_1a_conv_2_branch_2, block_1a_conv_2_branch_1, block_1a_conv_shortcut_branch_1 + add_1_branch_1 + activation_3/Relu_branch_1, block_1a_conv_shortcut_branch_2 + add_1_branch_2 + activation_3/Relu_branch_2, block_2a_conv_1_branch_2 + activation_4/Relu_branch_2, block_2a_conv_1_branch_1 + activation_4/Relu_branch_1, block_2a_conv_2_branch_1, block_2a_conv_2_branch_2, block_2a_conv_shortcut_branch_2 + add_2_branch_2 + activation_5/Relu_branch_2, block_2a_conv_shortcut_branch_1 + add_2_branch_1 + activation_5/Relu_branch_1, block_3a_conv_1_branch_1 + activation_6/Relu_branch_1, block_3a_conv_1_branch_2 + activation_6/Relu_branch_2, block_3a_conv_2_branch_2, block_3a_conv_2_branch_1, block_3a_conv_shortcut_branch_1 + add_3_branch_1 + activation_7/Relu_branch_1, block_3a_conv_shortcut_branch_2 + add_3_branch_2 + activation_7/Relu_branch_2, block_4a_conv_1_branch_2 + activation_8/Relu_branch_2, block_4a_conv_1_branch_1 + activation_8/Relu_branch_1, block_4a_conv_2_branch_1, block_4a_conv_2_branch_2, block_4a_conv_shortcut_branch_2 + add_4_branch_2 + activation_9/Relu_branch_2, block_4a_conv_shortcut_branch_1 + add_4_branch_1 + activation_9/Relu_branch_1, conv2d_bbox_branch_1 || conv2d_cov_branch_1, conv2d_cov/Sigmoid_branch_1, output_bbox_branch_2 || output_cov_branch_2, output_cov/Sigmoid_branch_2, output_bbox_branch_2 copy, conv2d_bbox_branch_1 copy, 
INFO: [TRT]: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
INFO: [TRT]: Detected 1 inputs and 6 output network tensors.
0:00:20.718205090 18646   0x559c671600 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<secondary_gie_0> NvDsInferContext[UID 4]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1748> [UID = 4]: serialize cuda engine to file: /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/fd_lpd/fd_lpd.caffemodel_b1_gpu0_fp16.engine successfully
INFO: [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT data            3x270x480       
1   OUTPUT kFLOAT output_cov      4x17x30         
2   OUTPUT kFLOAT output_bbox     16x17x30        

0:00:20.727704506 18646   0x559c671600 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<secondary_gie_0> [UID 4]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/pgie_config_fd_lpd.txt sucessfully
gstnvtracker: Loading low-level lib at /opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
gstnvtracker: Optional NvMOT_RemoveStreams not implemented
gstnvtracker: Batch processing is OFF
gstnvtracker: Past frame output is OFF
ERROR: Deserialize engine failed because file path: /opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine open error
0:00:20.816873115 18646   0x559c671600 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::deserializeEngineAndBackend() <nvdsinfer_context_impl.cpp:1690> [UID = 1]: deserialize engine from file :/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed
0:00:20.816921981 18646   0x559c671600 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::generateBackendContext() <nvdsinfer_context_impl.cpp:1797> [UID = 1]: deserialize backend context from engine from file :/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/../../models/Primary_Detector/resnet10.caffemodel_b30_gpu0_int8.engine failed, try rebuild
0:00:20.816948670 18646   0x559c671600 INFO                 nvinfer gstnvinfer.cpp:619:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Info from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1715> [UID = 1]: Trying to create engine from model files
INFO: [TRT]: Reading Calibration Cache for calibrator: EntropyCalibration2
INFO: [TRT]: Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
INFO: [TRT]: To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
INFO: [TRT]: 
INFO: [TRT]: --------------- Layers running on DLA: 
INFO: [TRT]: 
INFO: [TRT]: --------------- Layers running on GPU: 
INFO: [TRT]: conv1 + activation_1/Relu, block_1a_conv_1 + activation_2/Relu, block_1a_conv_2, block_1a_conv_shortcut + add_1 + activation_3/Relu, block_2a_conv_1 + activation_4/Relu, block_2a_conv_2, block_2a_conv_shortcut + add_2 + activation_5/Relu, block_3a_conv_1 + activation_6/Relu, block_3a_conv_2, block_3a_conv_shortcut + add_3 + activation_7/Relu, block_4a_conv_1 + activation_8/Relu, block_4a_conv_2, block_4a_conv_shortcut + add_4 + activation_9/Relu, conv2d_cov, conv2d_cov/Sigmoid, conv2d_bbox, 
INFO: [TRT]: Detected 1 inputs and 2 output network tensors.
ERROR: Serialize engine failed because of file path: /opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine opened error
0:00:29.630376615 18646   0x559c671600 WARN                 nvinfer gstnvinfer.cpp:616:gst_nvinfer_logger:<primary_gie> NvDsInferContext[UID 1]: Warning from NvDsInferContextImpl::buildModel() <nvdsinfer_context_impl.cpp:1743> [UID = 1]: failed to serialize cude engine to file: /opt/nvidia/deepstream/deepstream-5.0/samples/models/Primary_Detector/resnet10.caffemodel_b1_gpu0_int8.engine
INFO: [Implicit Engine Info]: layers num: 3
0   INPUT  kFLOAT input_1         3x368x640       
1   OUTPUT kFLOAT conv2d_bbox     16x23x40        
2   OUTPUT kFLOAT conv2d_cov/Sigmoid 4x23x40         

0:00:29.636817651 18646   0x559c671600 INFO                 nvinfer gstnvinfer_impl.cpp:313:notifyLoadModelStatus:<primary_gie> [UID 1]: Load new model:/opt/nvidia/deepstream/deepstream-5.0/samples/configs/deepstream-app/config_infer_primary.txt sucessfully

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

NOTE: To expand a source in the 2D tiled display and view object details, left-click on the source.
      To go back to the tiled display, right-click anywhere on the window.


**PERF:  FPS 0 (Avg)	
**PERF:  0.00 (0.00)	
** INFO: <bus_callback:181>: Pipeline ready

Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
** INFO: <bus_callback:167>: Pipeline running

KLT Tracker Init
**PERF:  60.80 (60.70)	
**PERF:  59.73 (60.13)	
**PERF:  59.80 (60.08)	
/dvs/git/dirty/git-master_linux/nvutils/nvbufsurftransform/nvbufsurftransform.cpp:3115: => VIC Configuration failed image scale factor exceeds 16, use GPU for Transformation
0:00:46.023833267 18646   0x559c0570a0 WARN                 nvinfer gstnvinfer.cpp:1268:convert_batch_and_push_to_input_thread:<secondary_gie_0> error: NvBufSurfTransform failed with error -2 while converting buffer
ERROR from secondary_gie_0: NvBufSurfTransform failed with error -2 while converting buffer
Debug info: /dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvinfer/gstnvinfer.cpp(1268): convert_batch_and_push_to_input_thread (): /GstPipeline:pipeline/GstBin:secondary_gie_bin/GstNvInfer:secondary_gie_0
Quitting

My pgie file is this:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=3
uri=file:///home/nvidia/Videos/kb-back-2.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=0
type=3
#1=mp4 2=mkv
container=1
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=2000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
output-file=out.mp4
source-id=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
bitrate=4000000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

[osd]
enable=1
gpu-id=0
border-width=4
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=1
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
## If set to TRUE, system timestamp will be attached as ntp timestamp
## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
# attach-sys-ts-as-ntp=1

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
#model-engine-file=../../models/Primary_Detector/resnet10.caffemodel_b4_gpu0_int8.engine
batch-size=1
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt


[tracker]
enable=1
# For the case of NvDCF tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=640
tracker-height=384
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_iou.so
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_nvdcf.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
#ll-config-file required for DCF/IOU only
#ll-config-file=tracker_config.yml
#ll-config-file=iou_config.txt
gpu-id=0
#enable-batch-process and enable-past-frame applicable to DCF only
enable-batch-process=1
enable-past-frame=0
display-tracking-id=1

[secondary-gie0]
enable=1
#model-engine-file=../../models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine
gpu-id=0
batch-size=1
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0
#config-file=config_infer_secondary_vehicletypes.txt
config-file=pgie_config_fd_lpd.txt

[secondary-gie1]
enable=0
model-engine-file=../../models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine
batch-size=16
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;2
config-file=config_infer_secondary_carcolor.txt

[secondary-gie2]
enable=0
model-engine-file=../../models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine
batch-size=16
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carmake.txt

[tests]
file-loop=1

and sgie0 is this:

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
model-file=fd_lpd/fd_lpd.caffemodel
proto-file=fd_lpd/fd_lpd.prototxt
labelfile-path=fd_lpd/labels.txt
net-stride=16
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
#force-implicit-batch-dim=1

num-detected-classes=4
interval=0
gie-unique-id=4
parse-func=4
output-blob-names=output_bbox;output_cov
#parse-bbox-func-name=NvDsInferParseCustomResnet
#custom-lib-path=/path/to/libnvdsparsebbox.so
#enable-dbscan=1

[class-attrs-all]
threshold=0.1
group-threshold=1
## Set eps=0.7 and minBoxes for enable-dbscan=1
eps=0.1
#minBoxes=3
roi-top-offset=200
roi-bottom-offset=200
detected-min-w=0
detected-min-h=0
detected-max-w=1920
detected-max-h=1920

#parser-bbox-norm=35.0;35.0

Your pgie file means deepstream-app configuration file, right? Your sgie input size is 480x270, right?

Please add “input-object-min-width=30” and “input-object-min-height=17” to your sgie configuration. https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html#page/DeepStream%20Plugins%20Development%20Guide/deepstream_plugin_details.html#wwpID0E04DB0HA

The error has told you “VIC Configuration failed image scale factor exceeds 16”. You need to limit the input object size.

1 Like