Running Deepstream on a cluster of jetson nanos

Hi,

Is it possible to run Deepstream (with a custom application that uses yolov3-tiny) on a cluster of jetson nanos? if yes, How should I do this?

You can run one DeepStream process on each nano.

Thank You8 for Your reply. I am seeing that in config files You can select gpu_id for each nvinfer. meaning that, it seems each primary-gie or secondary-gie can run on a separate gpu with a unique gpu-id. I am developing a license plate recognition software. I one primary-gie (which is for car’s grill detection). one secondary-gie1 that operates on the output of primary-gie (the grill detection network) and finds license plate within it. finally another secondary-gie (secondary-gie2) does inference on the output of the previous stage (secondary-gie1) to detect characters inside the detected plates.

to sum up, I have three detectors that are finding grill, plate inside the detected grill, and characters inside the detected plate. and, they are cascaded one after another.

are You telling me there is no way to run each of these three classifiers independently on one of these three Jetson Nanos?

are You telling me there is no way to run each of these three classifiers independently on one of these three Jetson Nanos?

I misunderstood your question. Yes, you can run these three classifiers independently on one of these three Jetson Nanos, but you need to have your own implemenation to pass the result from one Nano to another Nano.

thank you for your response.

If I make a kubernetes cluster of jetson nanos, as described here:

assuming that I have compiled and successfully tested deepstream yolov3-tiny application on a docker image (the docker image provided by nvidia), is it possible to simply run deepstream on the whole cluster, or I need to write huge amount of code to specify which gpu should do what?

in other words, is it possible to run deepstream (without any changes to deepstream’s source code) on the whole cluster (i.e. the whole cluster functioning exactly like a single unified GPU, without me having to distribute different tasks like primary detection, secondary detection, etc. across several nodes)?

if not, can you please give me a brief overall picture of what should be done and what the difficulties I should be expecting are?

This is not supported by DS SDK - you will have to try it out. you can have their own IPC mechanism to have inferencing on a cluster of nanos. Or can you use Jetson NX or Xavier based on their compute requirement - else it will be too much application complexity