Can you also check the CPU loading? How about limit the maxTargetsPerStream to 50? You can also check the perf issue with nsight systems: Nsight Systems | NVIDIA Developer
Found the CPU loading is too high when maxTargetsPerStream=800. The nvtracker lib consumed most of the CPU as there are many target manager operation. Seem you need limited maxTargetsPerStream in your current environment.
There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks.