GLM 4.7 Flash 30B FP8 on DGX, claude code /init and some architectural analysing tasks later…
The distribution is not chaotic, but it is also not strongly clustered — it’s a typical MoE pattern with soft routing:
-
Max/Min = 11× – the strongest pair (50↔61) fires only ~11× more often than the weakest (21↔54). That’s a relatively moderate spread.
-
81% of pairs are at 10–30% of the max – the bulk is “lukewarm.” There’s no sharp separation into “always together” vs. “never together.”
-
Only 3 pairs above 90%: 50↔61, 14↔57, 4↔6 – true “buddy pairs.”
-
Experts 60, 21, and 54 appear frequently among the weakest pairs → more like “lone wolves.”
Conclusion: GLM-4.7-Flash has a fairly well-balanced router. Most experts are combined with many different partners. There are a few preferred pairs, but no strong isolated clusters. For EPLB (Expert Load Balancing), this means that buddy pairs like 50↔61 should be placed on different nodes, but overall the load is well distributed.
It’s much more about splitting the hot buddies than about identifying hot vs. cold experts only.
Thesis: All to all, but well placed:
| Nodes | Cross-Node Score | Top 10 split | Top 20 split | Top 50 split |
|-------|------------------|--------------|--------------|--------------|
| 2 | 53.9% | 4 / 10 | 12 / 20 | 33 / 50 |
| 4 | 80.5% | 10 / 10 | 20 / 20 | 48 / 50 |
With 4 nodes, all Top-20 buddy pairs are placed on different nodes — ideal for parallelization.
With 2 nodes, it is mathematically impossible to separate all of them due to too many dependencies.
hmm …