I have access to the Clara AGX Developer kit, however I am a bit stumped on how to develop for it or develop on it. Our group typically trains/develops deep learning models on pytorch. How do we train and use models on the AGX developer kit?
It seems quite difficult to install and use applications on the AGX developer kit - for instance something like VS code doesn’t install/open correctly and a bunch of other applications do not work correctly. I followed the set up guide, and was able to get some L4T docker images installed, but they also usually had some errors (e.g. error import open cv in python). Is it only possible to use clara holoscan on the agx developer kit, am I missing something?
Hello, if you want to validate something in PyTorch on the Clara AGX Developer Kit, you could use the container PyTorch | NVIDIA NGC which supports the devkit with multi arch support. Just out of curiosity, what training workloads do you plan to run on the devkit? Since it has one RTX 6000 GPU.
For VS code, if you’re not able to install it locally on the devkit, perhaps you could use the Remote -SSH or Remote Development extension to connect to the devkit from another machine?
About the L4T docker images, on NGC there are several essential docker images that support multi-arch which includes the devkit, such as TensorRT | NVIDIA NGC and TensorFlow | NVIDIA NGC in addition to the PyTorch one linked above.
For the question “Is it only possible to use clara holoscan on the agx developer kit, am I missing something?” I’m not sure I understand, could you elaborate a little more?
Thanks for links - I’ll try the other NGC containers to see if I can reproduce the PyTorch workflow. Some of our training loads are quite minimal - e.g. under 1000 256x256 2D medical images, and it would be great to be able to train and deploy on the same machine for convenience.
To rephrase my question - what are the steps to achieve real-time inference once you have, say, a trained pytorch model? Does this require clara holoscan? We would like to keep the training–deploy cycle as simple as possible.
Oh I see, yes that description of a small training workload makes sense.
For deployment, I’d suggest the Clara Holoscan SDK. I would recommend to convert the PyTorch model into TensorRT for deployement. Our SDK has reference pipelines in GXF running models in the TensorRT format for streaming use cases.