DS plugin nvmultiurisrcbin problems with internal_id

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Please provide complete information as applicable to your setup.

• Hardware Platform GPU
• DeepStream 7.0 and 7.1 (2 different situations)

Hello, I have two different situations with multiurisrcbin

1) Using DS 7.1

When adding cameras sequentially via the multiurisrcbin REST API, they are added in order, and as long as I don’t remove any, the internal_ids generated by DS can be automatically mapped to the source_id values sent in the JSON to the API, giving me output control.

The problem arises when removing cameras. For example, I have a max-batch-size of 32 with 32 cameras added via API. If I remove camera 2 and camera 4, then re-add the camera with source_id = 4, DS 7.1 automatically assigns it the first available internal_id, which would be 2.

This breaks the internal mapping in my projects, as I maintain additional camera information in a custom config file.

Is there any way to retrieve the internal_id from the API? The API provides a get-stream-info method, but it only returns the parameters sent by API—not the actual internal_id, so the mapping isn’t guaranteed.

This mismatch also affects the tiler output since the camera order changes. It’s even worse when not using a tiler and relying on streamdemux outputs.

Is there any practical solution? Or in the worst case, would it be necessary to modify the C++ code of one of the plugins? If so, which file and which part of the code should be adjusted?

2) Using DS 7.0

A very similar situation happens here, when adding cameras sequentially via the multiurisrcbin REST API, they are added in order, and as long as I don’t remove any, the internal_id s generated by DS can be automatically mapped to the source_id values sent in the JSON to the API, giving me output control

The problem is when a source is removed, its associated internal_id is not recycled. So when re-adding a source, the internal_id increases by 1.

For example, I have 32 cameras added to multiurisrcbin with max-batch-size = 32. I remove cameras 2 and 4. When re-adding them, DS 7.0 assigns them internal_ids 33 and 34.

This causes internal mapping issues and even code crashes (exceptions) that kill the pipeline. While this may be easier to track (e.g., using probe data to detect sources that have not sent data for N seconds and replacing them), it still presents challenges in mapping.

The tiler is also affected: rows and columns increase dynamically, and sometimes the new cells do not display anything, and in the same way it would shoot the number of tiler cells to infinity while removing and adding cameras for days, also leaving the cells of the lost internal_id inactive.

So again, is there any practical solution? Or in the worst-case scenario, would I need to dive into the C++ code of the plugin? If yes, which file and what part should be modified?

Please, someone lend me a hand. The multiurisrcbin plugin is extremely powerful as it allows us to add/remove cameras dynamically without restarting the production pipeline, but it has the limitations I’ve just described. These issues are holding us back from launching an on-demand production system.

Looking forward to your kind official support, friends at NVIDIA.

Best regards.

Feat: Camera ID Metadata in DeepStream Pipeline Downstream (nvmultiurisrcbin)
3
  1. Let’s focus only on the latest version DeepStream 7.1.
  2. Could you please elaborate on exactly what the internal_id and source_id mean in this scenario?
  3. Could you provide a simplified sample to specifically illustrate this issue?
1 Like
4

Hi @yuweiw sure:

The internal_id is the id that DS automatically gives to the source internally inside multiurisrcbin, it works like the internal_id that the sink_pad of the streammux gives to each source connected to it. In the other side the source_id is the camera_id that you have to put in the JSON that is sent to the API inside multiurisrcbin, this is an example from the official documentation:

curl -XPOST ‘http://localhost:9000/api/v1/stream/add’ -d ‘{
“key”: “sensor”,
“value”: {
“camera_id”: “uniqueSensorID1”,
“camera_name”: “front_door”,
“camera_url”: “file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4”,
“change”: “camera_add”,
“metadata”: {
“resolution”: “1920 x1080”,
“codec”: “h264”,
“framerate”: 30
}
},
“headers”: {
“source”: “vst”,
“created_at”: “2021-06-01T14:34:13.417Z”
}
}’

I’m using the camera_id with numbers like 0,1,2,… just like the internal_ids you get from frame look like, in order to have a mapping.

Think about this situation:
I sent 4 sources to multiurisrc by API and I have a multiurisrcbin max-batch-size of 5, the camera_ids in the JSON i put are 0,1,2,3 and they map the internal_ids given sequentially by DS internally. Now I remove by API the camera_id 1, now I have cameras 0,2,3 now I add again the source id with camera_id = 1 from API, but DS in multiurisrcbin automatically gives to this camera the internal_id = 4 because the plugin rest API adds sequentially like a queue, now I have a problem because the camera_id = 2 has the internal_id = 4 and this breaks my mapping and there’s no mode I can map the source information given by another point of the API and the internal_id I get from the frame of the source because the endpoint just gives me the camera_id information, and this camera_id is info related to the JSON only

5

After adding or removing the sources, we can’t ensure the consistency between the camera_id(the term you used ‘source_id’) and the source_id(the term you used ‘internal_id’). You cannot use these two variables interchangeably. What is the purpose of your need for the map of these two variables?

1 Like
6

Sure, let me get my thoughts in order so I can explain our corporate situation to you.

7

I needed the source_id, so I included it in the plugin since it’s open-source. This might solve your problem.

1 Like
9

@Levi_Pereira Excellent buddy! That’s exactly what I needed, I updated the .cpp plugin and compile the .so in order to get the URI and the source_id and it works!

1 Like
10

@yuweiw I have another question, if i get the metadata inside a probe attached to a pad, for example i want to compare the source_id from a frame with all the source_ids obtained from /api/v1/get-stream-info, is there any problem if i consume the API each time? Or it will collapse at any moment if they are so many requests like a common API?

11

No. This part is open source. You can check that from the source code below. It’s just get data without other operations.

deepstream\sources\libs\nvds_rest_server\nvds_rest_server.cpp
else if (uri.find ("/stream/get-stream-info") != std::string::npos) {
      /* GET Requests Specific */
      m_func[uri] =
          [get_request_cb, uri] (const Json::Value & req_info,
          const Json::Value & in, Json::Value & out,
          struct mg_connection * conn) {
        return handleGetRequest (req_info, in, out, conn, get_request_cb, uri);
      };
    }
1 Like
12

Regarding your decision to keep querying the API in the deepstream probe:

I think querying the API hundreds of times per second isn’t a sensible implementation approach.

You control who gets added to the pipeline, you only need to query the API once to map the source_id, since changes will only occur when something is added. For removals, you can simply perform periodic checks every X seconds to clean up any removed sources, but this doesn’t interfere with the source_id mapping because removed sources won’t appear in the probe anyway.

I implemented a similar approach and followed this strategy: every time a source is added, it generates the source_id mapping, and if the probe detects any unknown source_id, it queries the API.

While the API is quite lightweight and doesn’t have significant impact, it’s still much better to query an in-memory variable than to keep hitting the API repeatedly.

This approach provides better performance, reduces unnecessary network calls, and creates a more efficient and maintainable system overall.

1 Like
13

@yuweiw
To help us make our development process more efficient, NVIDIA could assist us by adding the metadata fields camera_id and camera_name to the NvDsFrameMeta structure, similar to how source_id was previously added.

We suggest that NVIDIA consider implementing this feature, as the community would greatly appreciate it. This enhancement would eliminate the need for external queries to identify the camera_id and camera_name, streamlining our workflow significantly.

This addition would provide:

  • Direct access to camera identification within the frame metadata

  • Reduced dependency on external database lookups

  • Improved performance by eliminating additional query overhead

  • Better consistency with the existing source_id field

We believe this would be a valuable enhancement that would benefit the entire DeepStream developer community.

Thank you for considering this feature request.

2 Likes
14

For specific scenarios, we recommend that the customer add those to the NvDsUserMeta themselves. This makes it more flexible to use.

1 Like
15

Thank you very much!

I’d like to explore this topic a bit further, as I have a slightly different perspective on using NvDsUserMeta, despite it being a functional solution.

NVIDIA has begun implementing the concept of camera_id and camera_name, which makes much more sense than using source_id - which, in my view, is an internal pipeline property despite being used externally.

With the implementation of the REST API Server, which already helps significantly with source management at the uri-bin endpoint, we currently cannot utilize the camera_id property anywhere downstream in the pipeline. It would be extremely valuable to replace the variable source_id with something static like camera_id.

Using camera_id for payload message creation in probes or even in nvdsanalytics would be much more stable than constantly trying to work with an unknown internal variable (source_id) that only exists within the pipeline and is changed after each pipeline source change.

While NvDsUserMeta does solve the immediate problem, I believe that since camera_id is not a custom implementation but rather a native DeepStream feature, it would be beneficial to maintain and expand its usage throughout the DeepStream API.

2 Likes
16

Great, that’s a nice idea, I’m going to try it!

17

Because the source_id is used to implement the core logic in many plugins like nvstreamdemux, nvdsosd, nvstreamtiler, etc… and DeepStream also support many local video scenarios, it is inappropriate to replace it with camera_id here.

Since it can now be obtained through the get-stream-info method, could you describe in detail how you plan to obtain and use this camera_id in a specific scenario? Thanks

1 Like
18

I used the word “replace” incorrectly. My intention wasn’t to remove the existing source_id variable and substitute it with camera_id.

My actual goal is to introduce camera_id as a new variable and have users start using it instead of source_id, which would remain in the system for compatibility purposes.

Since it can now be obtained through the get-stream-info method, could you describe in detail how you plan to obtain and use this camera_id in a specific scenario? Thanks

Let me provide a concrete example of why extending camera_id usage is crucial:

Current Problem with source_id:

I created an pipeline using the REST API and added 3 file sources sequentially on-demand to the pipeline with sync=false on fakesink.

Here’s what happens:

  1. Source added → source_id is generated (unknown at this point)

  2. Frames start processing immediately through probes

  3. Hundreds of frames processed before I can call get-stream-info to discover which camera_id belongs to which source_id

  4. Data loss: All those initial frames are processed without knowing their camera_id

Additional nvdsanalytics complexity: Since nvdsanalytics also depends on source_id, I have to:

  1. Start the source

  2. Wait to get the source_id

  3. Rebuild the YAML/INI configuration file with the correct source_id

  4. Reload nvdsanalytics to apply ROIs correctly

Even when done quickly, this introduces significant latency while hundreds of frames are already being processed without proper ROI configuration.

Before REST API vs Now:

  • Before: Users manually controlled source_id - they knew exactly which source_id belonged to which source before start source

  • Now: The source_id becomes available once the source is already running, making it unknown until we query for it

Solution with camera_id: If camera_id were available in metadata:

  1. Pre-configure: Load ROIs in nvdsanalytics using known camera_id values BEFORE starting sources

  2. Immediate processing: When frames reach probes, camera_id would be immediately available in metadata

  3. No external queries: No need to call REST API during processing to map source_id to camera_id

  4. Zero latency: No frames lost or processed incorrectly during startup

This would eliminate the current race condition between source startup and configuration discovery.

Thanks.

2 Likes
19

Thanks for your suggestion. There is currently no plan to add the camera_id in the metadata. For now, you can record the camera_id yourselves as a workaround.

  1. When you add a new camera, record the camera_id.
  2. Mark your recorded camera_idusing the source_id of the newly added stream.
2 Likes
20

I’m going to try to implement this feature on nvdsmultiurisrcbin using NvDsUserMeta. I’ve read some code and I think it’s feasible.

1 Like
21

Sure. If it is successful, you can release your source code here for others to refer to. Thanks

2 Likes
22

I’ll hold off on this for a little while. I’m planning to implement it in DeepStream 8.0, as it adds support for nvdsanalytics with nvmultiurisrcbin.

1 Like