I have setup a HSB (with dual imx274 cameras) connected to a DGX Spark.
I can run the imx274_latency.py example (with --headless and a frame-limit) and get a SW Pipeline latency of 19.5 ms (avg)
** Complete report: **
Metric Min Max Avg
Frame Time (in sec):
Frame Time 0.0158 0.0158 0.0158
FGPA frame transfer latency (in sec):
Frame Transfer Latency 0.0001 0.0015 0.0005289972144846797
FGPA to Operator after network operator latency (in sec):
Operator Latency 0.0007 0.005 0.002523175487465181
Processing of frame latency (in sec):
Processing Latency 0.0003 0.0013 0.0006472144846796658
Frame start till end of SW pipeline latency (in sec):
SW Pipeline Latency 0.0172 0.0229 0.019564317548746518
If I then run the linux_imx274_latency.py example I get a SW Pipeline latency of 17.6 ms (avg)
** Complete report: **
Metric Min Max Avg
Frame Time (in sec):
Frame Time 0.0158 0.0158 0.0158
FGPA frame transfer latency (in sec):
Frame Transfer Latency 0.0 0.0013 0.00022211699164345405
FGPA to Operator after network operator latency (in sec):
Operator Latency 0.0005 0.0021 0.0009005849582172701
Processing of frame latency (in sec):
Processing Latency 0.0004 0.0018 0.0006279665738161559
Frame start till end of SW pipeline latency (in sec):
SW Pipeline Latency 0.0168 0.0194 0.017597799442896935
Why does the accelerated case have longer latency than the non-accelerated case? Does running the non-accelerated example make sense in this case?
This is due to a combination of the DGX Spark CPU configuration and the way that the underlying RoceReceiver needs to wake up the CPU to manage receipt of RDMA buffers for use in a holoscan pipeline.
The RoceReceiver monitors a CQ completion channel to be notified when a full camera frame has arrived via RDMA at the CX-7 NIC. Each frame comes in only every 16-33 ms for 60 fps to 30 fps camera modes. Checking sudo cpupower -c 2 idle-info on DGX Spark, you can see this is well over the residency time of the deepest idle state (~2.5 ms) so it is very likely that the linux cpuidle management puts the CPU this monitoring thread runs on into a deep sleep to save power and the nominal wake-up latency of this state is in excess of 400 microseconds. This is almost the latency budget that is normally seen on RoceReceiver under normal operation on devices that don’t have this deep sleep state enabled, e.g. the IGX platform devices.
This additional latency is not observed in the linux_* unaccelerated examples on DGX Spark because being a simple socket-based receiver, the buffer management thread wakes up every packet ( O(1-10 us) depending on camera mode and MTU) so the CPU is in constant use/never deep sleep idle.
You can see the effect by disabling the deep sleep in an A/B test:
Baseline: python3 examples/imx274_latency.py --frame-limit 300
Disable deep sleep: sudo cpupower idle-set -D 10 (note, this will disable power saving idle states on all CPUs, but since the accelerated examples don’t pin to a CPU, this is done for simple demonstration) python3 examples/imx274_latency.py --frame-limit 300
where the FPGA to Operator after network operator latency is less than 1/3 the baseline and the Processing latency and whole HSDK pipeline is also reduced because the CPUs are no longer entering deep sleep idle states.
Return to baseline: sudo cpupower idle-set -E (re-enables all idle states)
You should see performance return to what you initially observed.
