Cuda Engine with VPI

Hello, we have two components in our app. One uses Cuda to load and run a neural network on an image. The other uses optical flow using VPI.
We are interested to run both components on the GPU, but when we do this our app crashes.
We saw that there is a way to synchronize the Cuda and VPI streams with vpiStreamCreateCUDAStreamWrapper but we can’t run both streams simultaneously. Is there no way for both these components to work on the GPU at the same time if there are enough resources?

We are using an AGX.