How to custom tracker in deepstream

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
NVIDIA GeForce GTX 1650

• DeepStream Version

• JetPack Version (valid for Jetson only)

• TensorRT Version

• NVIDIA GPU Driver Version (valid for GPU only)
Driver Version: 525.125.06

• Issue Type( questions, new requirements, bugs)
new requirements

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
I want to custom my tracker to deepstream, like NanoTrack, but the doc is too simple that i cannot understand, etc: In NvMOTContext create code

 m_pLocalizer = LocalizerFactory::getInstance().makeLocalizer(config.customConfigFilePath);

where is “LocalizerFactory”? can you provide detailed documents or samples, thanks very much.

Those code is for understanding the API. Please implment your own tracker based on the API. Maybe your tracker don’t need LocalizerFactory. It depends on the algrithm of different tracker



I want to custom this track algo in deepstream, Now backbone and neckhead tensorrt inference has finished, with some opencv offical track code, i can run it as executable as well as shared library. but i have no idea about how to add to deepstream.

the tracker has two api:

void init(cv::Mat, cv::Rect);
cv::Rect update(cv::Mat);


  • what should do in NvMOT_Query?
  • should i allocate memory on gpu resource in NvMOT_Init?
  • should i create backbone engine and neckhead engine in NvMOT_Context?
  • should i run “init” and “update” func in NvMOT_Process?


expected some detailed doc or sample code for custom tracker in deepstream. thank you very much.


Please forgive me for poor English.

We haven’t sample code. We only have this document: Gst-nvtracker — DeepStream 6.3 Release documentation

hello, now i finish part of custom nanotrack, but got Bus error after some frames(not fixed num frame), with gdb, the backtrack is:

 #0  0x00007fffed2b8440 in NvTrackerProc::updateFrameMeta(_NvDsFrameMeta*, _NvMOTTrackedObjList const&, NvTrackerProc::ProcParams const&) () at /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/
#1  0x00007fffed2b8f67 in NvTrackerProc::processSurfaceStream(NvTrackerProc::DispatchInfo*) () at /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream/
#2  0x00007ffff70d279f in std::execute_native_thread_routine(void*) (__p=0x7fff580020f0) at /nv/src/gcc-target-12.1.0/libstdc++-v3/src/c++11/
#3  0x00007ffff7694b43 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:442
#4  0x00007ffff7726a00 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81

the context code (for SOT) is:

NvMOTStatus NvMOTContext::processFrame(const NvMOTProcessParams *params, NvMOTTrackedObjBatch *pTrackedObjectsBatch) {
    cv::Mat  cv_frame;
    cv::Rect rect;

    float confi;
    int   label;

    // std::cout << "stream num: " << pTrackedObjectsBatch->numFilled << std::endl;
    for (uint streamIdx = 0; streamIdx < pTrackedObjectsBatch->numFilled; streamIdx++){
        NvMOTTrackedObjList   *trackedObjList = &pTrackedObjectsBatch->list[streamIdx];
        NvMOTFrame            *frame          = &params->frameList[streamIdx];

        /* get tracker per stream */
        if (NanoTrackMap.find(frame->streamID) == NanoTrackMap.end()) {
            NanoTrackMap[frame->streamID] = std::make_shared<TrackerNanoTRT>(custom_config_file);
        /* get input frame */
        NvBufSurfaceParams *surfaceParams = frame->bufferList[0];
        cv_frame = cv::Mat(surfaceParams->height, surfaceParams->width, CV_8UC4, (char*)surfaceParams->dataPtr);
        cv::cvtColor(cv_frame, cv_frame, cv::COLOR_RGBA2BGR);

        /* get input rect */
        for (uint32_t numObjects = 0; numObjects < frame->objectsIn.numFilled; numObjects++) {
            NvMOTObjToTrack *objectToTrack = &frame->objectsIn.list[numObjects];

            float confi = objectToTrack->confidence;
            int   label = objectToTrack->classId;
            rect.x      = (int)(objectToTrack->bbox.x);
            rect.y      = (int)(objectToTrack->bbox.y);
            rect.width  = (int)(objectToTrack->bbox.width);
            rect.height = (int)(objectToTrack->bbox.height);

            /* single object track */

        /* tracking */>streamID)->update(cv_frame, rect);

        /* output */
        NvMOTTrackedObj *trackedObj = new NvMOTTrackedObj;

        trackedObj->confidence                     = 1;
        trackedObj->classId                        = (uint16_t)label;
        trackedObj->trackingId                     = 0;

        trackedObj->bbox.x                         = (float)rect.x;
        trackedObj->bbox.y                         = (float)rect.y;
        trackedObj->bbox.width                     = (float)rect.width;
        trackedObj->bbox.height                    = (float)rect.height;

        /* associatedObjectIn
        trackedObj->age                            = (uint32_t) sTrack.tracklet_len;
        trackedObj->associatedObjectIn             = sTrack.associatedObjectIn;
        trackedObj->associatedObjectIn->doTracking = true;

        std::cout << "frame num: " << frame->frameNum << std::endl;
        // std::cout << "numFilled: " << filled << std::endl;
        trackedObjList->streamID     = frame->streamID;
        trackedObjList->frameNum     = frame->frameNum;
        trackedObjList->valid        = true;
        trackedObjList->list         = trackedObj;
        trackedObjList->numFilled    = 1;
        trackedObjList->numAllocated = 1;

the query code is:

NvMOTStatus NvMOT_Query(uint16_t customConfigFilePathSize,
                        char *pCustomConfigFilePath,
                        NvMOTQuery *pQuery) {
     * Users can parse the low-level config file in pCustomConfigFilePath to check
     * the low-level tracker's requirements
    pQuery->computeConfig = NVMOTCOMP_CPU;       // among {NVMOTCOMP_GPU, NVMOTCOMP_CPU}
    pQuery->numTransforms = 1;                   // 0 for IOU tracker, 1 for NvDCF or DeepSORT tracker as they require the video frames

#ifdef __aarch64__
    pQuery->memType = NVBUF_MEM_DEFAULT;
    pQuery->memType = NVBUF_MEM_CUDA_UNIFIED;

    pQuery->batchMode              = NvMOTBatchMode_NonBatch;    // set NvMOTBatchMode_Batch if the low-level tracker supports batch processing mode. Otherwise, NvMOTBatchMode_NonBatch
    pQuery->supportPastFrame       = false;    // set true if the low-level tracker supports the past-frame data or not

     * return NvMOTStatus_Error if something is wrong
     * return NvMOTStatus_OK if everything went well
    std::cout << "[NanoTrack Initialized]" << std::endl;
    return NvMOTStatus_OK;

You can get the source code of nvtracker plugin on DeepStream 6.3. So you can debug it with the source code.

root@p4station:/opt/nvidia/deepstream/deepstream-6.3# ls /opt/nvidia/deepstream/deepstream-6.3/sources/gst-plugins/gst-nvtracker/
Makefile README convbufmanager.cpp convbufmanager.h gstnvtracker.cpp gstnvtracker.h invtracker_proc.h logging.h miscdatabufmanager.cpp miscdatabufmanager.h nvtracker_proc.cpp nvtracker_proc.h

thank you, the problem has solved, you can close the issue