Deepstreamer Pipeline: Optimisation GPU Utilisation

Accelerated Computing Intelligent Video Analytics DeepStream SDK
, ,
22

Thank you for getting back to me @Fiona.Chen. I appreciate your continued support.

What I had done was regenerated the onnx & engine to support dynamic batching. I have updated the configuration file, as well as manually set the batch size in the code, and added code to print out the size of the batch for each batch using a probe. The outputs from this indicated that batches 99% of the time were processed in 10s (the number of sources), but occasionally 1 or 2.

Config Exert:

batch-size=10

Code Exert:

// Set the batch size for the primary inference engine
g_object_set(G_OBJECT(pgie), "batch-size", pgie_batch_size, nullptr);
std::cout << "Batch size (A): " << pgie_batch_size << std::endl;

// Verify that the batch size was set correctly
guint actual_batch_size = 0;
g_object_get(G_OBJECT(pgie), "batch-size", &actual_batch_size, nullptr);

std::cout << "Actual Batch size (B): " << actual_batch_size << std::endl;

if (actual_batch_size != pgie_batch_size)
{
23

The pieces of code gave not enough information. As you said, there are customized probe functions, special configurations with your own model, udpsink for network operation, …

Every component in the pipeline will impact each other when the pipeline is running, we need to check every possible factor.

A simpler way to identify the problem is to deploy your model with deepstream-app sample. Then we will be aligned by the same app.

Can you deploy your model with “deepStream-app” for the performance checking? Let’s check with the basic pipeline first.

1 Like