How to move to CPU+GPU based processing for existing Python ML Models

[surfinride]

We have some Python ML algorithms in the form of .pyc files that we have tested on the Jetson Nano. The ML algos make use of Python libraries such as Numpy, Pandas, Scipy, Scikit-Learn, etc. We are using the Yocto build system to build a custom image. Currently, it seems like the algorithms run only on the CPU. We would like to be able to use the GPU to speed up the execution of these ML algorithms.

I have some questions regarding the approaches we can take to achieve this:

1. Is it possible to run these Python ML algorithms as is, without any modification in a way that they get executed on both the CPU+GPU?
2. If the method in question 1 is not possible, what is quick way with minimal modifications to the python code, that would help us use the CPU+GPU and see performance benefits?
3. What is the ideal / recommended method for moving to CPU+GPU based processing that will help us see the most performance benefits?

I am new to Python, ML and GPU based processing. A couple things I know are that Tensorflow allows for distribution of workload between CPU and GPU. There is PyCuda, which allows for GPU acceleration using CUDA. And there TensorRT as well allows using the GPU.

I would be really grateful if someone with knowledge about this can point me in the right direction, so that I can speed up my learning process.

P.S.: Originally I had posted about this on the Data Processing section of this forum, but a Moderator recommended posting here to get better help.

[AastaLLL]

Hi,

Please note that it’s required to use CUDA to run an implementation on GPU.
Lots of libraries have done this and use a very similar API for simplicity.

So please check your library and find if they have GPU support first.
Thanks.

[ankithsingh]

Hi @surfinride

Scikit-learn does not provide any GPU support as of now, you can check for Rapids Docs if you are working with ML algos, According to the Pre-requisites we need Pascal GPU architecture to use Rapids and Jetson Nano has Maxwell architecture so if you have access to high end version of Jetson Family products, except nano then you may run Rapids onto it.

Open GPU Data Science | RAPIDS

You can also once check scikit-cuda.
https://scikit-cuda.readthedocs.io/en/latest/index.html

Thank you