Hi All,
Our automation project required a large number of cameras to be connected to a single Jetson. We could have gone the one-camera-per-Jetson route, but there would have the issue of network connectivity. There is no guarantee that WiFi would have been appropriate in our operational environment, and space limitations precluded wired Ethernet, etc. Not to mention: We would had to write software to coordinate all those Jetson’s.
I am sure that others have faced the same problems, so I would like to get some feedback before embarking on completion of a 24-camera multiplexer. Specifically, I would like to know the “magic number” of cameras for a mux that people would prefer. Because our situation required so many cameras, I figure that, if I am going to do the design, might as well anticipate the need for others. I would like to avoid a situation where we do 24-cameras-per mux, and discover later that someone else would have been happy with 25, for example.
Before I start the journey for this mux, my question is, how many cameras does everyone else want attached to a single Jetson?
I do not plan to have all cameras running at once, only a few at a time of the 24. Take a snap-shot, switch cameras, take snap-shot, …
I am aware of the products over at Arducam, Auvidea, E-ConSystems, etc.'s. I have to make a custom board anyway because of other constraints.
Cheers,
-John
hello RareHare,
Assume you would like to have 24 video nodes, (i.e. /dev/video0, …/dev/video23), and accessing each of MIPI camera sensor individually.
Truth is,
- you’ll need above with Virtual Channel support. but we only enable VC support for TX2 and Xavier platforms.
- in addition, if each camera sensor occupy 1 MIPI lane. there’re only 12-lanes on TX2, 16-lanes on Xavier. you might also check Port Index session for details.
An alternative way would be reduce the physical number connected to tegra platform.
for example,
according to Jetson Nano Product Design Guide, please referring to [Table 37. CSI Configurations], Nano only able to support four camera sources.
if you combine 6 camera sensor frames as one single input source, your request (24 MIPI sensor) would become four different cameras connected to tegra platform.
you’ll need to have implementation from user-space to crop the necessary region for your usage.
thanks
Jerry,
Thanks for this insight.
That was my strategy: Have 24 cameras by attaching 6 cameras to my multiplexer, then connecting 4 multiplexers to Jetson Nano (for example) for a total of 24 cameras. I would do something similar for the higher-end Jetson platforms. My original plan was to switch between each of the 6 cameras using combination of hardware, software, so that only 1 of 6 cameras is “streaming” at once (snap-shot, switch, snap-shot, switch, etc.), but after reading some of the documentation that you provided, it appears that it might be possible to fuse 6 images into one with combination of hardware, software, without too much headache. Of course, this has been done before in other contexts. The trade-off is the complexity between fast switching to achieve smooth interleaving, and complexity of hardware on the mux. But in the case of snapshot/switch/snapshot/switch/etc., there are a lot of scenarios (automated inspection, etc.) where companies would be happy with, say, a 1-second (or less) delay between each of 24 images of an object under inspection, then running those images through AI pipeline to make determinations. The alternative to a 24-camera mux would be to have 4 full Jetsons with 6 cameras each. But the Jetsons would have to be networked, software written, etc. Not that that is bad… there are use cases for widely-dispersed Jetsons, but in some cases, a single powerful Jetson with many cameras is preferable.
Very nice documentation, BTW. It was my first time reading documentation from nVidia. It is clear that a lot of effort went into bringing Jetson to the market.
Cheers,
-John
Hi John and Jerry,
I am keen to try that approach: 6 cameras per multiplexer (4). Which multiplexer could I use?
Cheers
hello gtalckmin,
please note that, we only enable VC support for TX2 and Xavier platforms.
you might also contact with Jetson Preferred Partners for camera solutions.
thanks
Hi gtalckmin,
A multiplexer that can support 10 cameras per MIPI CSI-2 (1 lane) port has been built. It was tested to work with Raspberry Pi, as there seemed to be more interest from the Raspberry Pi people than from NVidia. If enough people show sufficient interest for Jestson, I would not mind issuing a board for the Jetson also.
The Raspberry Pi model is a prototype and has not been brought to market yet. I was hoping that there would be an easy way to do interleaving, where multiple imagines could be combined (panoramic views, etc.) It turns out that it is possible, but to do it, I would need access to code that is buried in the bowels of the RPi’s image processor.
In any case, the simple task of taking images or video sequences, one camera at a time, is possible, and has been tested.
Hi RareHare,
10 cameras per lane is exceptional. I only found two multiplexers (Arducam and IVPort) available for this task.
As I understand, Jetson has the upperhand over RPis when it comes to image processing/tasks. I am looking to build either a 3D scanning system (photogrammetry) or a multi-band camera. If you could give some pointers where to start, I would appreciate it a lot.
Cheers
Hi gtalckmin,
I am familiar with both of those vendors. They were our first options before developing the 10-camera multiplexer.
Just to be clear: You do realize that our multiplexer cycles through the cameras in a round-robing fashion that is too slow for real-time frame-interleaving, right? Also, I have not yet tested on Jetson, though I cannot see a reason why it would not work with Jetson.
Cheers,
-John
Hi John,
Thanks for the reply. I get the point that the cameras are taking photos in serial-fashion (thus, the multiplexing). For the applications I mentions, it should not be an issue. High frame rate is not necessary. If you have multiplexers which you are keen to propose, please let me know (pvt or post).
Cheers