When will the MTMC(Multi-Camera Tracking) microservice be available for Jetson, or will it be released?
1 Like
I will check internally and feedback here. Thanks.
1 Like
Thank You for your interest in MTMC. MTMC is on the roadmap for Jetson but at this point we are focused on improving accuracy for MTMC which requires several large models which wouldn’t be practical on Jetson. Once we have the accuracy where we desire, we will look into optimizing with smaller models which should run on Jetson
2 Likes
@kesong thank you for the reply. I had the same question. Could you please guide me on steps on how can i breakdown the MTMC app into a hybrid mode like VST and Deepstream running on jetson and other MTMC services on cloud? We are trying to solve a usecase of Intrusion detection on multiple camera, if a intrusion is detected we want to keep tracking that person through multiple cameras. Since we have a custom pipeline created in pure python, we are thinking of scaling and MTMC looks like a great tool to test this out quickly. Please let me know if we can solve this. We are open to collaborating as well.
Thank You
Subham Agrawal
Drishti Works
Thanks for your intresting MTMC. Why you need run VST and DeepStream on Jetson? MTMC is for dGPU currently.
Thank you for the quick response. I want to run VST and Deepstream on Jetson because of local camera inference and tracking and then run rest of the pipeline on cloud for ReID, global tracking etc.
We may need transformer based model for more accuracy. Maybe Jetson isn’t the best option for MTMC currently.
In my opinion, MTMC can work on Jetson Orin AGX with proper models (detection, person embedding, clustering/matching) and nvtracker, and without EKL stack, milvus, etc.
Please consider put MTMC in the JPS roadmap.
Can you share more details of your use case?
We have many use cases using a base pipeline that is MCMT, including but not limited to Retail/Transportation Hubs, etc: tracking people in all cams (MCMT).
Base on MCMT, coordinates of peoples in pixel coordinates can mapped to store/metro layout (retail) if cameras are calibrated and there is a transform from pixel coordinates to world (store/metro layout) coordinates. Then there are maybe a analytic service, webAPI and web UI.
Some of the specs/requirements are
The system can process N 5-FPS RTSP/Video streams in near real-time (30 ~ 90 seconds) fashion.
The flow is quite similar to that of MCMT in dGPU.
- DeepStream/Perception service (stream decoding, person detection, person tracking)
- Results of the first step (tracking results of all cameras, time series data) are sent to MCMT microservice via a msgbroker/bus.
- MCMT processes time series in batch (time/frame window)
- Visualize MCMT results on original streams. If store/metro layout is available, visualize footprints on the layout also.
- Analytic service: count #people passing/staying an RoI between t1 and t2, …
- WebAPI, Web UI
The step to assure ReID/MCMT accuracy is step 3. Assume a session timeout for a person is 30 mins, and a time window is 30 secs, so MCMT service must maintain a memory base db/gallery for quick searches in addition to a clustering process. The larger the time window the higher the accuracy and the larger the delay, so there are many factors to tune here. We hope that we can bring our custom models, MCMT algorithms to your MCMT services. For example, you can allow users to provide/inject a callback that is a clustering function, …
Also please consider disaster recovery (app restarted, Jetson device restarted, Network unstable, …), in case of processing video stream, say we have a video of 1000 frames, the pipeline have processed 900 frames and restarted how all the services can handle (start at 901)?
Why MTMC on dGPU is not enough for your project? What is the benifit of Jetson?
Sometimes, we cannot persuade clients to use a dGPU in case they have many small retail store (say each store just has 5 cams), so a Jetson device is enough?