• Hardware Platform (Jetson / GPU) - X86
• DeepStream Version - 7.1 / 8.0
• Issue Type (questions, new requirements, bugs) - Bug Fix
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) - Use NVDS-Analytics Plug-In for a long time (several hours), once it takes more than current frame rate, backpressure will build up and affect the rest of the system.
I noticed that in Deepstream 8 NVDS-Analytics internal library was open-sourced.
So, I was able to find and fix the above-mentioned bug.
The bug was also reported in this topic-
Nvdsanalytics processing time increasing over time - Intelligent Video Analytics / DeepStream SDK - NVIDIA Developer Forums
This is the problem and the solution-
Element internal data is never properly removed although the code seem to remove object not seen for the last m_timeOut ms.
This happens in the file nvds_analytics.cpp
Current code -
unordered_map <uint64_t, NvDsAnalyticInfo> m_nvanalyticInf;
...
vector <int> remove_obj;
...
for_each(begin(remove_obj),end(remove_obj),[&](int n){m_nvanalyticInf.erase(n);});
Fix-
unordered_map <uint64_t, NvDsAnalyticInfo> m_nvanalyticInf;
...
vector <uint64_t> remove_obj;
...
for_each(begin(remove_obj),end(remove_obj),[&](uint64_t n){m_nvanalyticInf.erase(n);});
- Notice
int->uint64_t
When Gst-nvtracker is configured with useUniqueID=1, tracking object ID uses all 64 bits, then the use of int instead of uint64_t causes implicit casting to 32bits - you get different keys.
In C++ erase() on std::unordered_map with key return the number of elements removed, so effectively it does nothing (returns 0).
This means internal m_nvanalyticInf keeps growing forever.
Once all types are aligned to uint64_t you can see the erase is working properly, m_nvanalyticInf.size() is stable, element processing time is great.