Feed second classifier with two input bounding box


I am trying to create a person re-identification model, therefore, i have developed a Siamese Neural Network (on a RTX 2080 ti) that compare two bounding boxes and give a True, False output with the confidence score.

Right now, I am trying to develop the same model on the jetson nano. But I do not know how to feed the second classifier (my custom one) with two bounding boxes instead of having the pipeline feeding the sgie with one bbox of the pgie. (may I can try to have one image input and the use tf.split to create the pipeline?)

  1. It is possible to have a siamese sgie? I have the uff model but I am not able to try it on deepstream

  2. It is possible to feed the sgie with two bbox? How?


This is currently not possible in nvinfer, There are some suggestions u can attempt. You can first check if the both the boxes need to be fed to the network simultaneously or it can be done sequentially. Assuming both the boxes are being fed to the network sequentially, you can -
Order the two bboxes in gst_nvinfer_process_objects so both images required for one inference are sequential in order.
U should also define parse-classifier-func-name where the output buffers of both the images are parsed and corresponding metadata should be generated.

If the network has two inputs then the changes required may not be trivial and will need further investigation. Our internal team will discuss and support this feature in next release.