Homomorphic Encryption on Decathlon Dataset

Hello,

I am trying to train a model under Homomorphic Encryption on the recommended Task09_Spleen dataset, following this notebook: clara-train-examples/Homomorphic_Encryption.ipynb at master · NVIDIA/clara-train-examples · GitHub.

I am able to train a model on the toy dataset (consisting of a single image) suggested in that notebook successfully, but if I try to change this to the Task09_Spleen dataset I get the following error on the client side:

Traceback (most recent call last):
File “<nvflare-0.1.4>/nvflare/apis/fl_component.py”, line 39, in fire_event
File “apps/fed_learn/trainers/client_trainer.py”, line 45, in handle_event
File “apps/fed_learn/trainers/client_trainer.py”, line 62, in initialize
File “<nvflare-0.1.4>/dlmed/utils/wfconf.py”, line 172, in configure
File “<nvflare-0.1.4>/dlmed/utils/wfconf.py”, line 167, in configure
File “<nvflare-0.1.4>/dlmed/utils/wfconf.py”, line 163, in _do_configure
File “apps/train_configer.py”, line 549, in finalize_config
File “/opt/monai/monai/data/decathlon_datalist.py”, line 128, in load_decathlon_datalist
return _append_paths(base_dir, is_segmentation, expected_data)
File “/opt/monai/monai/data/decathlon_datalist.py”, line 72, in _append_paths
raise TypeError(f"Every item in items must be a dict but got {type(item).name}.")
TypeError: Every item in items must be a dict but got str.
Traceback (most recent call last):
File “<nvflare-0.1.4>/nvflare/private/fed/client/fed_client.py”, line 127, in federated_step
File “apps/fed_learn/trainers/client_trainer.py”, line 90, in train
AttributeError: ‘ClientTrainer’ object has no attribute ‘trainer’
Traceback (most recent call last):
File “<nvflare-0.1.4>/nvflare/private/fed/client/fed_client.py”, line 229, in admin_run
File “<nvflare-0.1.4>/nvflare/private/fed/client/fed_client.py”, line 178, in run_federated_steps
File “<nvflare-0.1.4>/nvflare/private/fed/client/fed_client.py”, line 135, in federated_step
File “<nvflare-0.1.4>/nvflare/private/fed/client/fed_client_base.py”, line 217, in push_models
File “/opt/conda/lib/python3.8/multiprocessing/pool.py”, line 364, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File “/opt/conda/lib/python3.8/multiprocessing/pool.py”, line 771, in get
raise self._value
File “/opt/conda/lib/python3.8/multiprocessing/pool.py”, line 125, in worker
result = (True, func(*args, **kwds))
File “/opt/conda/lib/python3.8/multiprocessing/pool.py”, line 48, in mapstar
return list(map(*args))
File “<nvflare-0.1.4>/nvflare/private/fed/client/fed_client_base.py”, line 161, in push_remote_model
File “<nvflare-0.1.4>/nvflare/private/fed/client/communicator.py”, line 292, in submitUpdate
File “<nvflare-0.1.4>/nvflare/private/fed/client/data_assembler.py”, line 33, in get_contribution_data
File “<nvflare-0.1.4>/nvflare/private/fed/client/client_model_manager.py”, line 103, in read_current_model
TypeError: argument of type ‘NoneType’ is not iterable

This is my environment.json config file:

  • DATA_ROOT: “/data_4FL_spleen/”
  • DATASET_JSON: “/data_4FL_spleen/dataset.json”
  • PROCESSING_TASK: “segmentation”
  • MMAR_EVAL_OUTPUT_PATH: “eval”
  • MMAR_CKPT_DIR: “models”

The dataset.json file is the file generated when running the download for Task09_Spleen inside the DownloadDecathlonDataSet.ipynb notebook. This config file is the same as the default except I have updated which datast.json file to point towards.

What do I need to do to get the model training under HE on this dataset? Do I need to edit the dataset.json file after it has been created? This error possibly suggests I do - if so what changes need to be made?

File “/opt/monai/monai/data/decathlon_datalist.py”, line 72, in _append_paths
raise TypeError(f"Every item in items must be a dict but got {type(item).name}.")

Thank you.

Hi, it looks like your data list is following a different format to what load_decathlon_datalist expects. Please compare your format with the one in given in the notebook. FYI, nvflare has been updated since the last Clara train release and there are other examples using homomorphic encryption, like here, that might be of interest.