A TOP monitor program specific for the DGX SPARK

Accelerated Computing DGX Spark / GB10 User Forum DGX Spark / GB10 Projects
1

Hey all I have put together a TOP like program to monitor your Disk, and network IO performance, as well as your CPU and GPU load and temperature real time values, in a nice SSH supported TUI,

Check it out at GitHub - GigCoder-ai/dgxtop: An Enhanced TOP program to monitor your Nvidia DGX SPARK's Hardware

This was written in Python, with GLM-4.5 Air 235b running in a Ray Cluster across 2 sparks.

Please provide feedback, and let me know what you think!

Max

2

Wow! this is super nice!!!

Can it display the status of two sparks on one top instance? That’s probably a hard edge case since it has to communicate via ethernet…

3

Nice work @maxvamp !

May I suggest to get the information from procfs and sysfs instead of relying on other apps, or use nvidia-nvml-dev Python wheel. See Package Index

Like the GPU stats, with the default update interval nvidia-smi is being run ever second:

 cmd = [
         "nvidia-smi",
          f"--query-gpu={self.QUERY_FIELDS}",
          "--format=csv,noheader,nounits",
]

Details at dgxtop/dgxtop/gpu_monitor.py at b443dc63d2beeda075ef8b47325673665d06958c · GigCoder-ai/dgxtop · GitHub

4

It be cool if it would show processes and gpu usage like in nvidia-smi

(edit: I did a fork and quick implementation of this idea GitHub - sonusflow/dgxtop: An Enhanced TOP program to monitor your Nvidia DGX SPARK's Hardware )

dgx_top
dgx_top1920×1195 462 KB

5

This is amazing :)

6

Thats cool,

quick question - does anyone know how to get the System Memory and GPU Utilisation stats from the DGX Dashboard into the System Tray or in the bar at the top of the screen.

So far I am using

But would be nice to have a graph instead of just a plain number too.

7

I also developed one written in Rust, with easier installation and more detailed information. Everyone is welcome to give it a try.

image
image2512×1256 140 KB

image
image2536×379 19.1 KB

Regular computers can use it too.

8

I saw this GitHub - antheas/spark_hwmon: Linux hwmon driver for the NVIDIA DGX Spark (GB10 SoC) that exposes full system power telemetry via standard sensors / sysfs interfaces. · GitHub the other day.

9

image
image1143×883 155 KB

I am using Vitals now

10

I use sparkrun cluster monitor. It monitors the whole cluster, I can monitor it from anywhere, is part of sparkrun which belongs to our community, and is well maintained by Drew @dbsci :

IMG_0803
IMG_0803831×607 66.7 KB

Kind of a no-brainer