First of all - thank you for all the great work that has been put into CLARA.
We’ve been able to do many interesting AI experiments with it, particularly with CLARA FL.
We are wondering if there is any existing feature in CLARA that would help us easily record & visualize FL performance with respect to round number?
In other words, we would like to have a way to easily save the performance of the global model (as validated on all the client data) for each round & have something like the Tensorboard to conveniently visualize it.
If something like this exists already, could you kindly let us know what feature this is & which part of the documentation should we dig into?
(our understanding is that Tensorboard is only for “CLARA TRAIN” and can only be used to visualize the performance at the “client level” but not really the global server model level performance with respect to FL rounds)
If not, we would like to propose this as a potential feature in the next release as we can imagine many FL researchers wanting to inspect the convergence of their model with respect to rounds.
To be more specific, it would be useful to be able to
Do cross site validation after each round, and record the aggregated model performance as validated on each individual client dataset + weighted averaged performance for each round
Have something like Tensorboard to easily visualize the result as recorded in 1