I have a sensor → processing using our own cuda kernels → ethernet pipeline I’d like to optimize a lot. Since we are loading a lot of calibration data used in the processing which does not change I’d like to reuse it in the next kernel without reloading from global memory. Since this is not possible I was thinking about using persistent threads, which seem to be the right tool, but I cannot find a lot of sample implementation for that.
Also it seems to be that samples I have found are using CUDA features, that are not avaiable on Jetson Nano. Is there any way to use persistent threads on Jetson Nano?
The kernel would look like this:
Load calibration data
Loop
– Wait for new image from V4L2
– Process Image
– Signal Image Ready for Ethernet access
If you move the data to the GPU memory, the buffer can be used between the CUDA kernel.
Do you define the kernel in different processes or the same process but different threads?
Yes, but I will have to load it again from global GPU memory, right? Which also takes a lot of time compared to the calculations I am doing.
Or is it cached in a way that reading the same memory location from the same kernel but different iteration does not have to be read again?
Jetson cannot share memory between processes due to GPU context limitations.
But if you implement it in a multi-thread way, the data can be reused once it doesn’t be destroyed.
I do not think you are getting my point.
You have to load data from global memory gpu memory in a cuda kernel. This takes time. I want to avoid to do this every iteration so I want to use a never ending persistent kernel.
Is my question better suited for a general CUDA forum than Jetson forum?
Sorry, I thought you were asking something related to memory access.
Memory data is held by the CUDA context.
So do you want to share the CUDA context between kernels?
Or could you share a sample so we can know more about your use case?