How to train an Annotation model, and question on clinical workflow

I am using Clara as a tool for my thesis and have some questions about the integration of Clara in a clinical setting.

First of all, I cannot seem to find how to train an Annotation model? I see how to train classification and segmentation models, but annotation seems to be missing. As there are few annotation models in the NVIDIA archives for Clara 2.0 I assume you have to create many from scratch, and I see how you can do that with segmentation, but not how to create an annotation model from a segmentation model. I’ve seen the “AddExtremePointsChannel” function for simulating the extreme points inputs, but I don’t really know where to go from there. I saw this thread but it was a bit vague in how you actually go about training Annotation models.

Secondly, I am trying to describe a scenario where Clara could be used to continuously generate new training data in hospitals. For example, after every relevant CT/MRI scan, the radiologist would use both Clara Deploy for it’s inference capabilities and simultaneously create new labeled training data which could be used to further improve models. My understanding is that Clara Train and Clara Deploy are separate, so is there any way in which you could do both of these at the same time?

Thank you!

Basically, what I wondering is this:

When using Clara Deploy and performing inference in a clinical setting, is it possible to export the result of the inference model and use that as part of a labeled dataset to keep training new models?

I looked further into the Clara Deploy documentation and found this:

Segmentation results in the output directory:

  • One file of the same name as your input file, with extension .mhd
  • One file of the same name, with extension .raw

Can either of these files be used for annotation in Clara Train, or is there any reason for why this would not work/not be a good idea?

Thanks for you interest in both Clara train and deploy SDK.

I guess you question is mainly towards Clara Train AIAA. Short answer is yes you can use these files.
Long Answer: AIAA is a client server architecture so you can use any file format that the client supports. We have clients integration with MITK and 3DSlicer. Both can open nifti, mhd and dicom.

For you second question on if you can use the result from inference as another set in for an ongoing continues learning. This is mainly the goal, however the inference should be verified and corrected by clinicians.
In general the network benefits more from new samples it gets wrong than samples it got right. So manually fixing/correcting the predictions is key

Hope that helps.

Hi there. I even havent known that we can train an Annotation model. So as I understood we can use these files to try do it? Thanks.

If you download an Annotation model from NGC, for example this one, it contains a script detailing how it was trained. The download button is found in the “Version History” tab, or you can view the files directly on the website.