Hi,
In the new Jetson ORin platform there is a chllamge to integrate image sensors and to bridge them from HDMI/SDI, etc to PCIe, MIPI, etc. I want to use the HoloScan as follows:
Connect MIPI/HDMI/SDI image sensor (any) and use LAttice FPGA as the ISP and bridging to PCIe, ethernet. and through the LAttie Certus trasfer it to ethernet or PCIe
Questions:
HoloScan is only workin on SFP ports? can I connect a PHY to make it work over copper? can I use standard GigE ports?
can I use to Holoscan to transfer it over PCIe as well? NVIDIA provide relevant drivers for PCIe? Or NVIDIA is agnostic for her PCIe lanes on the JEtson platform?
What is the HoloScan Lattancy with LAttice reference design from camera ouput to the Orin platform DRAM?
Holoscan SDK can be used to ingest data via different I/O interfaces, with much of the network stack abstracted away.
Holoscan is a software package and is independent of the equipment used to acquire data.
You can create HSDK operators that acquire data from any source you like.
A few helpful links: Holoscan sensor bridge software architecture - NVIDIA Docs Adapting new sensors - NVIDIA Docs Dataplane - NVIDIA Docs
From a hardware perspective, Holoscan sensor bridge has SFP ports, which can be used with an RJ45 copper to SFP adapter;
we’ve used 10GBASE-T adapters for this purpose. If your host system already has an RJ45 interface, you need to make sure that the interface supports the signalling generated by the 10GBASE-T adapter.
Yes, if you can get the data from your sensor into GPU memory, HoloscanSDK software can work with it.
To get the sensor data into GPU memory, you may choose to follow the Holoscan sensor bridge IP design. Overview - NVIDIA Docs
Note, that “The Sensor Interface and Host (Ethernet) Interface blocks are FPGA vendor specific logic”, and hence need to be worked out with the specific vendor.
On a IGX devkit with DGPU, transmitting 4k video at 60 fps takes
avg=0.99, min=0.5, max=2.6 (ms).
The measured time is from EOF received at fpga till full frame received at IGX.
For the time From EOF Received at fpga till Profiler Op after ROCE receiver on IGX, the values are
avg=1.81, min=0.9, max=3.5 (ms)
The 2.0 GA will include instructions on reproducing these numbers.
Also, if I want to connect Ethernet camera with low latency as MIPI direct Injection, the Holoscan is a good idea? usually Ethernet cameras has very high latency? and reference for that?