Missing detection in some frames while using deepstream yolo detection on two videos

I am using deepstream pipeline (custom deetstream-test3) for detection. I used my custom yolov4 model, converted using https://github.com/marcoslucianops/DeepStream-Yolo this repo and got “.so” file.
When I run it on one video there are no missing detections. But while running on two videos, there are missing detections in a few frames. While running on two same videos also i see same issue and also miss on different frame, there will not be detection in both videos at a same time. It seems the detection just switch from one frame to other in case of multi streaming and skipping the frame to detect.
I have attached the config file and the python deepstream file
config_infer_primary_yolov4-tiny.txt (2.5 KB)
deepstream_test_3_working.py (29.3 KB)

There was some bug in the git repo which I used to do the conversion of the yolo model. I had a few conversion with the owner of the repo, and he fixed all the issue. Now it is working perfectly fine