Custom Tracker Segmentation Fault

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• T4
• DeepStream Version
• 5.0
• TensorRT Version
• 7.0
• NVIDIA GPU Driver Version (valid for GPU only)
• 450.51.05
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
• Plugin: libnvdsgst_tracker

Hello!
I’m attempting to create my own custom tracker and have ran into an issue when I populate the NvMOTTrackedObjBatch *pTrackedObjectsBatch. I can populate the data just fine and have verified that the objects persist before I return an NvMOTStatus_OK. However, the issue is when the function returns, I get a Segmentation Fault.
This is the stack trace from Valgrind:

==5103== Thread 13:
==5103== Use of uninitialised value of size 8
==5103==    at 0x1C410932: NvTrackerProc::updateFrameMeta(_NvDsFrameMeta*, _NvMOTTrackedObjList const&, NvTrackerProc::ProcParams const&) (in /opt/nvidia/deepstream/deepstream-5.0/lib/gst-plugins/libnvdsgst_tracker.so)
...
==5103== Invalid write of size 8
==5103==    at 0x1C410932: NvTrackerProc::updateFrameMeta(_NvDsFrameMeta*, _NvMOTTrackedObjList const&, NvTrackerProc::ProcParams const&) (in /opt/nvidia/deepstream/deepstream-5.0/lib/gst-plugins/libnvdsgst_tracker.so)
...
==5103== Process terminating with default action of signal 11 (SIGSEGV): dumping core
==5103==  Access not within mapped region at address 0x38
==5103==    at 0x1C410932: NvTrackerProc::updateFrameMeta(_NvDsFrameMeta*, _NvMOTTrackedObjList const&, NvTrackerProc::ProcParams const&) (in /opt/nvidia/deepstream/deepstream-5.0/lib/gst-plugins/libnvdsgst_tracker.so)

The code below is roughly what I’m doing to populate the pTrackedObjectsBatch.

NvMOTTrackedObjList* MOTobjList = &pTrackedObjectsBatch->list[0];

  for(size_t i = 0; i < objects.size; ++i)
  {
   customTrackerObject trackerObject = objects.at(i);

    MOTobjList->list[i] = NvMOTTrackedObj{};
    MOTobjList->list[i].classId = trackerObject.classId;
    MOTobjList->list[i].trackingId = trackerObject.trackingId;
    MOTobjList->list[i].bbox = {trackerObject.bbox.x, 
                                trackerObject.bbox.y,
                                trackerObject.bbox.width,
                                trackerObject.bbox.height};
    MOTobjList->list[i].confidence = trackerObject.confidence;
    MOTobjList->list[i].age = trackerObject.age;

    MOTobjList->list[i].associatedObjectIn = new NvMOTObjToTrack;
    MOTobjList->list[i].associatedObjectIn->classId = trackerObject.associatedObjectIn->classId;
    MOTobjList->list[i].associatedObjectIn->bbox = {trackerObject.associatedObjectIn->bbox.x, 
                          trackerObject.associatedObjectIn->bbox.y,
                          trackerObject.associatedObjectIn->bbox.width,
                          trackerObject.associatedObjectIn->bbox.height};
    MOTobjList->list[i].associatedObjectIn->confidence = trackerObject.associatedObjectIn->confidence;
    MOTobjList->list[i].associatedObjectIn->doTracking = true;
  }

  MOTobjList->numFilled = objects.size;
  MOTobjList->valid = true;

I should note that I’m only testing a single stream and am not concerned with multiple streams (at least not yet.)
I’m unsure as to what the issue could be since I’m able to access the data before I return from the processing function. Although, the docstring states:

 * @param [out] pTrackedObjectsBatch
 *                             A pointer to a batch of lists of tracked object
 *                             slots to be filled by the tracker. The batch is
 *                             allocated by the client. Bounding boxes are
 *                             scaled to the resolution of the first input
 *                             image transform buffer.

Thtat the lists of tracked objects are to be filled by the tracker, but the Batch is to be allocated by the client (which I assume is me). However, when I access the variable everything is already allocated for me or unless I’m misinterpreting the docstring?

3

Hey, would you mind to share your custom lib(source code and makfile) with me, so I can repro locally to look into it.

4

Howdy @bcao! Thank you for the reply!

For the sake of time, and explaining the custom tracker, I’ve created a minimum working example which produces the exact same result as I’ve seen with the custom tracker I’ve built. Below are the .cpp file and the Makefile which I’ve used to build the minimum working example. I should note that this code and the code from my tracker never get passed the first frame of the stream so regardless of what information is present in the pTrackedObjectsBatch object, the code never gets past the updateFrameMeta function from the private Nvidia tracker library.

gstnvdcustomtracker.cpp (8.0 KB)

################################################################################
# Copyright (c) 2017-2020, NVIDIA CORPORATION.  All rights reserved.
#
# Permission is hereby granted, free of charge, to any person obtaining a
# copy of this software and associated documentation files (the "Software"),
# to deal in the Software without restriction, including without limitation
# the rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Software, and to permit persons to whom the
# Software is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
# THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
#################################################################################
#enable this flag to use optimized dsexample plugin
#it can also be exported from command line
CUDA_VER?= # CUDA version ENV variable
ifeq ($(CUDA_VER),)
  $(error "CUDA_VER is not set")
endif
TARGET_DEVICE = $(shell gcc -dumpmachine | cut -f1 -d -) # command to get the machines architecture
CXX:= g++ # basic cpp macro

# file source name
SRCS:= gstnvdscustomtracker.cpp

INCS:= $(wildcard *.h) # inclusions for the header files of the source
LIB:=libnvds_customtracker.so # resulting static library name

NVDS_VERSION:=5.0# DeepStream version

CFLAGS+= -fPIC -DDS_VERSION=\"5.0.0\" \
	 -I./ \
	 -I/usr/local/cuda-$(CUDA_VER)/include \
	 -I/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/sources/includes \

LIB_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/lib/

 # Flags for library paths and the librarys in those paths
LIBS := -shared -Wl,-no-undefined \
	-L/usr/local/cuda-$(CUDA_VER)/lib64/ -lcudart -lcudadevrt -ldl \
	-lnppc -lnppig -lnpps -lnppicc -lnppidei \
	-L$(LIB_INSTALL_DIR) -lnvdsgst_helper -lnvdsgst_meta -lnvds_meta -lnvbufsurface -lnvbufsurftransform \
	-Wl,-rpath,$(LIB_INSTALL_DIR) # this line tells linker to search the LIB_INSTALL_DIR at runtime to resolve references -Wl refers to the ld command with a ',' being a delimiter which gets
	# expanded to a space-separated list when 'ld' command is ran using the -rpath command with the variable LIB_INSTALL_DIR expanded to its value.

OBJS:= $(SRCS:.cpp=.o) # substitution for the SRCS variable which replaces the .cpp with .o
# gets the proper packages for the machine architecture using the TARGET_DEVICE command

ifeq ($(TARGET_DEVICE),aarch64)
	PKGS:= gstreamer-1.0 gstreamer-base-1.0 gstreamer-video-1.0 opencv4
else
	PKGS:= gstreamer-1.0 gstreamer-base-1.0 gstreamer-video-1.0 opencv
endif

CFLAGS+=$(shell pkg-config --cflags $(PKGS)) # get the CFlags for the packages required by this plugin (-I flags (includes))
LIBS+=$(shell pkg-config --libs $(PKGS)) # get the Libraries required by the packages (-l Flags)

all: $(LIB)

# builds the .o files for the cpp files, called as part of a makefile rule (this rule also requires a Makefile in that dir)
%.o: %.cpp $(INCS) Makefile 
	@echo $(CFLAGS)
	$(CXX) $< -c -o $@ $(CFLAGS)

# Expanding the LIB variable and creating a rule which builds the .so
$(LIB): $(OBJS) Makefile
	@echo $(CFLAGS)
	$(CXX) -o $@ $(OBJS) $(LIBS)

install: $(LIB)
	cp -rv $(LIB) $(LIB_INSTALL_DIR)

clean:
	rm -rf $(OBJS) $(LIB)
5

Thanks, will try locally.

6

Thank you @bcao! I’ll be eagerly awaiting your findings!

7

Hey customer,
I commented following line in your code and it work well. You can check the plugin manual Gst-nvtracker — DeepStream 6.1.1 Release documentation, the associatedObjectIn should point to the NvMOTObjToTrack passed in the function rather than allocate by yourself.

MOTobjList->list[i].associatedObjectIn = new NvMOTObjToTrack;
MOTobjList->list[i].associatedObjectIn->classId = 0;
MOTobjList->list[i].associatedObjectIn->bbox = {170, 
                            162,
                            13,
                            10};
MOTobjList->list[i].associatedObjectIn->confidence = -0.1;
MOTobjList->list[i].associatedObjectIn->doTracking = true;
8

Thank you @bcao! This was definitely a lot of help!