VPI 3.2 scheduling CUDA kernels/API calls on a VPI stream

Robotics & Edge Computing Jetson Systems Jetson AGX Orin
1

I am trying to execute an asynchronous CUDA memcpy after some VPI calls in VPI 3.2. Right now I’ve just wrapped a VPI stream around a CUDA stream, submit my VPI work on the wrapper stream, and submit the memcpy on the underlying CUDA stream after flushing on the VPI stream. However, this does not appear to be effective at enforcing execution order. It seems like only a synchronize achieves the execution order we want, but we would prefer our application to be asynchronous. I noticed in the VPI4 docs the function vpiSubmitCUDAHostFunction which seems to do what we want, and would allow us to submit CUDA work asynchronously along with VPI work. I am wondering if there is any intention to implement this function in VPI3 (for AGX Orin), or if an explicit VPI stream synchronize is the only way to achieve what we want right now. I am assuming VPI4 is only compatible with Thor right now, though I may have interpreted the docs incorrectly.

6

No. There is no plan to add functionalities in older release.

7

@siraajssandhu ,
How about using vpiSubmitHostFunction() after VPI calls? Does that help?

static void cb(void *data) {
    MyArgs *a = data;
    cudaMemcpyAsync(a->dst, a->src, a->n, a->kind, a->cudaStream);
}
// submit VPI work, then:
vpiSubmitHostFunction(stream, cb, &args)