DeepStream App access source camera-id

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

• Jetson Nano
• DeepStream SDK 5.0.0
• JetPack 4.3

I’m attempting to customize my deepstream-app to take inferenced objects and pass them on to an external application (a custom app I’ll develop later). Part of that is making sure I know which source the detected object came from. Thumbing through the deepstream_app_main.c it looks like the best place to get at that data is in the all_bbox_generated() callback function. I can get at the NvDsFrameMeta and NvDsObjectMeta, and although the documentation says NvDsFrameMeta::source_id holds the camera ID, it’s always zero for me no matter which source the object came from.

Is there some other way to get at the [sourceX] camera-id property? I’ve looked at this thread and this thread and still didn’t really figure it out. Any help would be greatly appreciated.

My example sources:

[source0]
enable=1
type=4
uri=rtsp://mycamerauri
gpu-id=0
cudadec-memtype=0
camera-id=10

[source1]
enable=1
type=4
uri=rtsp://myothercamerauri
gpu-id=0
cudadec-memtype=0
camera-id=11

And just to test I’m printing the source id like this in the sample code whenever a person is detected:

g_print("source_id: %d\n", frame_meta->source_id);
1 Like
3

Did you figure this out ?

4

Not yet! It seems like as per the documentation it should work!

5

I haven’t gotten around to checking this @cclaunch but based on your description it does seem as if it’s a bug. Likely it’ll be fixed by the 5.0 GA release. Tagging: @DaneLLL @mchi

1 Like
6

If there are any additional debug friendly steps I can do please let me know. Also if there’s a public facing bug tracker I wouldn’t mind following along.

7

It may be worth taking a look at the deepstream-test3 sample. This is a test program that takes n sources as input and is a nice basic example.
This test app definitely has the source-id available in the frame meta. I use it in my apps.

1 Like
8

@cclaunch

If you’re looking for a GStreamer debug “cheat sheet”, in addition to the usual tools like gdb and it’s frontends, the closest i’ve found is here:
https://gstreamer.freedesktop.org/documentation/tutorials/basic/debugging-tools.html

RidgeRun also has a wiki page on gstreamer debuggings with some additional tips and tools:

As far as a bug tracker, you’d have to ask Nvidia. I just use the forum here and stuff gets fixed. If it’s working for @jasonpgf2a, however, it may be worth comparing deespstream-test3 to your own code as he suggests to see if something source-id related is missing. TBH I don’t use the source-id in my own pipelines currently but I will be needing it soon.

9

@cclaunch
So, reading up a bit, it looks like you may need to assign that based on something like a pad id (a pad probe callback is suggested for this), but there seems to be no example of how to do that (at least from a grep of the source) and you’re right, it seems to contradict the docs. Also, if a pad id is used, the source id would be assigned by when it’s connected, which in the case of sometimes pads cannot be counted on to be identical every time.

I will investigate this on monday since it crosses purpose with what I’m doing right now. If I find all ids are 0 in my own pipeline, I’ll post here (and likewise will share my pipeline design if it works).

1 Like
10

@jasonpgf2a thanks for that suggestion! That app definitely does show me the source IDs (at least indexed by input parameter anyways). So it sounds like it could be a bug specifically with the deepstream-app. I’m digging in too to be absolutely sure.

@mdegans thanks for checking into it. It’d be nice to know I’m not misreading the documentation.

11

@mdegans looking through the code, the only place I see camera_id referred to is in deepstream_app_config_parser.c in the set_source_all_configs() function. And in there it’s setting camera_id to the index of the source. This would also be fine with me (if it worked) but I don’t see where it’s taking the camera-id=X input as the documentation says.

12

Have a look at the source for deepstream-test5. This is just a copy of deepstream-app with some modifications to add smart record…
The sources are available and work properly in this app as well as I copied it to change the way smart record works so that it starts recording when a certain type of detection is made… I use the source id to send a message to AWS IoT with the details…

1 Like
13

@jasonpgf2a awesome, this works. I’m baffled as to why the main deepstream-app is not working though, the code does look identical. That smart-record stuff is pretty cool too. Ideally in the end I’d like to send important detections to my Blue Iris setup as triggers.

14

No worries. The deepstream-app might be fixed up when DS5.0 is GA soon… I’ve noticed a couple of issues with it in this developer preview release. Streamdemuxing doesn’t work properly either - yet it did in DS4.02.

1 Like
15

@jasonpgf2a I have a couple of random add on questions since you seem to be using deepstream very much in the same vein as how I would like to. I’m still brand new to this SDK so please excuse any ignorance.

  1. I’d like it to do the object detections but also general motion detection of said objects. That way I can hopefully weed out false positives that aren’t moving. E.g. one camera I have faces the street so it just sits there detecting parked cars all day across street. Is there functionality built into the detector that allows for this?
  2. I’d like to use the display hooked up as the actual monitor we use to see the cameras, but be able to change which cameras appear on the display based on various inputs (motion, objects, user input, etc). Does deepstream allow you to change which sources are shown in the tiled output on the fly or does it all have to be up front when you start?
16

  1. There is nothing built-in to do this but you can code the logic yourself by post processing the bboxes for example. Or even creating your own motion detection gstreamer element which could add user metadata so that your downstream probe can take into account both the detections from nvinfer and the motion element…

  2. I don’t know of any standard capability to do this but it would be cool… The nvmultistreamtiler does have a show-source control property. So by default it shows all sources - tiled. But you can tell it to show just one source fullscreen. So maybe when a specific event occurs that you are interested in you can set the tilesrs show source property to zoom into that source… Then after a delay go back to tiled…?!? To see this (sort of) in action you can test with the sample deepstream-app. It is coded such that if you click on a tile it zooms into that tile…

1 Like
17

Good deal, lots of stuff to look at. Thanks again for your help!

18

@cclaunch
I got a chance to implement a source id for my metadata broker element and frame_meta->source_id is always 0 as you report. The documentation says it “holds the source ID of the frame in the batch”, but what is supposed to set it is unclear. I think I remember reading something else about it but I can’t recall exactly where.

I had always assumed(?) that was supposed to be set by nvstreammux (closed source). It does appear appears that’s either broken or it is expected to be set in another way. It would be nice to have feedback from Nvidia on this (either confirmation of a bug, or a pointer to some example code). Deepstream_app itself not working makes me thing bug.

@DaneLLL @Amycao @mchi pls help us! edit: think we’re good now

19

Think I solved it, @cclaunch

tl;dr: a nvtiler element will set source_id to 0 so if you use a tiler, you’ll have to access source_id beforehand.

I found a note in apps/sample_apps/deepstream-app/deepstream_app.c that says:

 *         NOTE: source_id key and this API is valid only when we
 *         disable [tiler] and thus use demuxer for individual
 *         stream out

So I asked @jasonpgf2a about whether he had a tiler in his pipeline. He did, but it was branched off in a tee. So i swapped the order, of things a bit and it works.

My link code (Genie) was:

			// link all elements
			if not self.pie.link_many( \
					self.tracker, self.tiler, self.distance, \
					self.broker, self.osdconv, self.osd)
				error(@"$(self.name) failed to link elements")

I changed that to:

			// link all elements
			if not self.pie.link_many( \
					self.tracker, self.distance, self.broker, self.tiler, \
					self.osdconv, self.osd)
				error(@"$(self.name) failed to link elements")

And now source_id is populated and my broker element writes the source_id.

Hope this helps!

re: where source_id is set: seems to be by the stream muxer based on the pad id, so you’ll still need to make some mapping between the id and the source itself. I think that’s what @mchi was saying in the other thread, rereading it. Why it’s broken in deepstream_app, I can’t say, but that may give you a clue.

3 Likes
20

Very nice find! Thanks @mdegans!