Questions about Plugin 'enqueue' function

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi,

I have some questions about the ‘enqueue’ function that needs to confirm.

int enqueue(int batchSize, const void * const * inputs, void ** outputs, void * workspace, cudaStream_t stream)

  • Where are the inputs and outputs stored? Inside the Device memory, right?
  • If I have a CPU based op, I need to copy the input data from Device to Host. Do computation on CPU and then, copy result back to device memory pointer 'outputs'. Am I right?

    • Thanks!

    2

    Hi,

    Enqueue uses device memory for all the operations (input, output).
    If I have a CPU based op, you need to copy data between host buffers to device memory buffers and vice-versa.
    Input formats:

    • (N, C, H, W) for 2D Convolution.
    • (N, C, D, H, W) for 3D convolution.

    Thanks