Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU) Xavier AGX & Xavier NX
• DeepStream Version 5.0
• JetPack Version 4.4
• TensorRT Version 7.1.3
• Issue Type Question
Good Morning,
I am using the deepstream SDK with python Bindings, I have successfully created a working deepstream pipeline with multiple CCTV camera inputs, we have tuned the parameters to ensure minimal latency and maintain a stable FPS per stream.
The problem is that the System comes to a grinding halt if any of the cameras is disconnected or unstable.
What is the recommended approach for handling dodgy cameras (some are linked with long range radio extenders that can be temperamental in bad weather)?
I have already tried by specify nvstream.batched-push-timeout to be 1/FPS, but it had minimal effect
if a Camera is disconnected each stream drops down to 0.2 FPS so it basically chokes the entire pipeline
Regards Andrew
Can you share your python script? Theoretically, the pipeline will not be impacted even there is one stream broken. We need to know more information to analysis the problem.
okay let me test first with multistream example application and see if I can reproduce it first
Hi Fiona,
I can indeed reproduce the results with deepstream_imagedata-multistream.py
I just ran the deepstream_imagedata-multistream.py application with 3 rtsp sources
python3 deepstream_imagedata-multistream.py rtsp://admin:pass@192.168.0.1:554/profile1 rtsp://admin:pass@192.168.0.2:554/profile1 rtsp://admin:pass@192.168.0.3:554/profile1 test
It works perfectly, however when i disconnect the lan cable of 1 of the cameras all the streams drop to 0.2 FPS
below is some console output (trimmed some unecessary information like obj count)
Seting bufapi_version
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
In cb_newpad
In cb_newpad
Frame Number= 0 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
*
*
*
Frame Number= 129 Number of Objects= 2 Vehicle_count= 0 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 25.8
Frame Number= 129 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 2 is 25.8
Frame Number= 129 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 25.8
Frame Number= 130 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
*
*
*
*
Frame Number= 246 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 2 is 23.2
Frame Number= 246 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 23.2
Frame Number= 246 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 23.2
Frame Number= 247 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
*
*
*
*
Vehicle_count= 0 Person_count= 0
Frame Number= 392 Number of Objects= 2 Vehicle_count= 0 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 29.0
Frame Number= 392 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 2 is 29.0
Frame Number= 392 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 29.0
Frame Number= 393 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
*
*
<------------------- Removed lan cabel------------------->
*
*
Frame Number= 405 Number of Objects= 2 Vehicle_count= 0 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 2.4
Frame Number= 405 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 2.4
Frame Number= 406 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
Frame Number= 406 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
Frame Number= 407 Number of Objects= 2 Vehicle_count= 0 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 0.2
Frame Number= 407 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 0.2
Frame Number= 408 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
Frame Number= 408 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
Frame Number= 409 Number of Objects= 3 Vehicle_count= 1 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 0.2
Frame Number= 409 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 0.2
*
*
<------------------- Reconnected lan cabel------------------->
*
*
Frame Number= 556 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
**********************FPS*****************************************
Fps of stream 0 is 29.2
Frame Number= 556 Number of Objects= 2 Vehicle_count= 0 Person_count= 2
**********************FPS*****************************************
Fps of stream 1 is 29.2
Frame Number= 557 Number of Objects= 0 Vehicle_count= 0 Person_count= 0
\
Is this not to do with the batch Inference?
Regards Andrew
@Fiona.Chen
Do you have any feedback on this topic?
Regards Andrew
We need to reproduce the problem.
Please try to modify the following code:
streammux.set_property(‘batched-push-timeout’, 4000000)
change it to
streammux.set_property(‘batched-push-timeout’, 40000)
1 Like
Hi Andrew, I have created deepstream python pipeline with one camera and now if the camera goes off for some time, app exits with EOS message. I want to handle this issue using approach you discussed. Can you give me some pointer for python apps for this case.