tegra k1 multiple simultaneous video decoding/encoding possible?

Anybody try if nvidia k1 support multiple simultaneous video decoding/encoding with openmax API ?
from gstreamer for example

Hi Miskol,
Yes, it is possible.
Please let me know if you need more help.

Can describe how many PAL/FULLHD streams nvidia decoder can handle?
And same with PAL/FULLHD video encoder

I am looking for something better than raspberry pi
raspbery pi handle 4xPAL(h264) decoding and encoding in same time with OpenMAX

Hi Miskol and Rpaliwal,

I’m kind of on the same track and like to know how to connect 6 (FullHD) cameras to the TK1 board.

I’ve read that it’s possible to connect 4 of them via CSI 4x1 interface.
Another one could be connected via USB3.0 and the last one could be connected to MiniPCIe slot.

I’m really new to this stuff and this is what my foundings are so far.

Is this a proper solution, or are there better options?


EDIT: My excuse, I thought this was a discussion about the TK1 dev board, but I see it is a TK1 chip related discussion. I’ll just leave this here in case you happen to know. :)

IMO an easy way right now to stream multiple cameras with the Jetson is with GigEVision or some other gigabit ethernet camera, combined up to a gigabit per second aggregate on the Jetson (the K1 chip itself can be connected to more NICs over PCIe and support streaming multiple gigabits).

USB 3.0 is stated to work in R19.2, if you enable it in the odmdata, so a PointGrey USB3 camera would be good to test.

I’m curious how a camera could be connected through the PCIe half mini slot? WiFi?

Best regards,

Decoding and viewing two or three 720P videos simultaneous via gstreamer works:
Go to a terminal:
gst-launch-0.10 playbin2 uri=file:///home/ubuntu/Videos/video1.mp4&
gst-launch-0.10 playbin2 uri=file:///home/ubuntu/Videos/video1.mp4&
gst-launch-0.10 playbin2 uri=file:///home/ubuntu/Videos/video1.mp4&

The GigE vision camera’s seem pretty expensive… How would they be connected through PCIe Mini, would there be a switch or router involved? I thought the max speed is 1 Gbit/s, is this enough for 6 GigE Vision FullHD signals?

I was thinking of using a capture card on the PCIe Mini slot to get the stream in.

Not sure which way I want to go…

A major difference between a cheap camera device and an expensive camera device is whether the camera does onboard video compression. If it doesn’t do video compression (eg: most USB webcams), then it is sending full video frames as fast as it can, and so the limit of 480Mbps in USB 2.0 isn’t fast enough for 1080p (FullHD). Gigabit Ethernet and USB 3.0 both have enough bandwidth to allow sending uncompressed 1080p video streams. But neither of them have reached the mass consumer market like USB 2.0 has, so Gigabit or USB3 cameras are still expensive, and thus possibly include video compression too. If you use Gigabit Ethernet or USB 3.0 cameras that do video compression, then perhaps you can fit many FullHD streams on a single bus. (I’m not sure, I’ve never tried!).

The Jetson TK1 Wiki (http://elinux.org/Jetson_TK1) lists many different miniPCIE cards you could put into the Jetson TK1 for extra ports, such as a Dual-Gigabit Ethernet card (therefore potentially giving you 3 Gigabits of Ethernet bandwidth when combined with the existing Ethernet!) or another USB 3.0 card, or a Firewire card.

Firewire has been used for professional video capture for a long time, since Firewire cameras do a lot of both the compression and the data transmission in hardware and thus don’t put nearly as much stress on the host CPU compared to USB. I’m not sure how many 1080p streams you can fit on a Firewire card, but if it handles multiple 1080p streams then I’d say that is your safest bet.

That is some really useful info Shervin, thanks. I’ve been googling around to find some prices for FireWire cameras. For example I’ve found this PointGrey list of FW cams: http://www.ptgreystore.com/firewire

The prices are only shown when logged in but the Flea2 2.0MP for example costs 1395 USD and still isn’t able to shoot 1080p @ 30fps. So this also is a very expensive solution.

I’ve also found an interesting USB3.0 camera, which I guess delivers a raw stream, at a price of 150 USD:

If only the compression could somehow be done in a different way…

I guess catching 6 uncompressed streams on the board is not an option…

Another camera just came to my attention:

It does compression and transmits data over 10M/100Mbps.
Because the Realtek-port on the Jetson does a 1000Mbps it would be possible to connect up to 10 of these modules, correct? Seems like a solution to me, or am I missing something…?

I’ve sent an e-mail for more information about the pricing of these modules, I’ll let you guys know…!

Another question that raises my mind is the SPI connection… Does SPI 4x1 mean that it is capable of handling 4 SPI devices? If so, the camera’s listed here are also a good way to get 4 of them hooked to the board:

By the way, I’ve created a Cameras wiki page at http://elinux.org/Jetson/Cameras. Feel free to contribute to it if you have found any cameras that do in fact work on Jetson TK1, so we can start removing some of the “untested” marks on that page!

That’s great, very useful list. I am going to get a bunch of these modules (particularly the dual ethernet and WiFi) and see what drivers work.

The GigEVision cameras typically do Bayer encoding.

In theory six 1080p30 streams could fit down three GigE ports on the Jetson - one provided onboard and the other two on miniPCIe module.

I’m confused, what exactly is the difference between GigE Vision camera’s and IP Camera’s? They both do hardware compression and stream over Ethernet… IP Camera’s are much cheaper though… Is it all in the speed difference?

Only thing I can think of is standardization.

I can recycle the same image acquisition code as long I use a camera that follows the GigE Vision Standard.

Plus some guarantees as long I meet requirements.

I’m using multiple GigE Vision cameras on the TK1.

timbaas, older GigE cameras typically do not apply any compression, compression is feature of the newer gige vision 2 standard. Most industrial gige cameras for computer vision don’t have compression. As far as I know, all IP cameras have some form of compression.

I guess you’re right falco, but I still don’t understand why the costs would be that much higher… Probably just because it’s new technology…

Nico, good to know… Do you know of any drawbacks for using an uncompressed stream, especially for the TK1?

Currently I doubt between using one of the following:

Option 1:
6 (or 8 if possible) USB3.0 uncompressed YUV422 streams over 2 USB3.0 hubs on 2 USB3 interfaces.
One YUV uncompressed 1080p30fps, 8 bit streams at 995,33 Mbps
The USB 3.0 interface is 5 Gbit/s so it should be able to handle 5 streams tops.

Option 2:
6 (or more if possible) 100Mbit/s compressed ethernet/ip cameras over 1 switch/router on the 1Gbit/s interface.
1 Gbit/s could handle 10 x 100Mbit/s streams tops.

A few questions remains:

  • Can the PCIe mini handle the 5Gbit/s USB3 stream? (from what I’ve read this shouldn’t be a problem)
  • Can the TK1 handle 6 uncompressed USB3 streams?
  • Can the TK1 handle 10 compressed ethernet streams?
  • All camera’s in above options do have linux drivers, so I don’t need to do low-level programming, right?

I’ve been in contact with someone at LeopardImaging. They provide 1080p30fps USB 3.0 camera’s which I want to use on the Jetson board (options 1 in my reply above). (https://www.leopardimaging.com/LI-USB30-AR0330ICP.html)

He told me he thinks it’s not possible, I quote:
“I don’t think that could work,since for 1080p@30fps YUV422,each camera almost needs 1Gbps bandwidth(plus uvc header)…”

I don’t understand, if USB3 provides 5Gbps and I have 2 of these interfaces (1 via PCIe mini), I should be able to use 6 streams (6Gbps) via 2 hubs, right?

Has anyone tried running multiple (4 or more) simultaneous SD or HD encoders on the K1?

I’ve tryed using a PointGrey Flea3 (USB3) with Jetson.
The provided ARM application compiles and runs, but it completely hangs for resolutions close to FullHD.

I am running FullHD @ 50 fps with Basler acA2000-50g GigE camera with h/w encoder (via gst-launch-1.0), consuming about 150% of 400% CPU load. As far as I remmeber, tried Basler USB3 camera simultaneously, worked fine at 10 fps on 3840x2748 resolution and over 300% cpu was loaded.