About inputs and outputs of a custom layer

I’m trying to add a custom layer and have to implement virtual member function of IPluginExt, enqueue, whose parameters are like below.

virtual int enqueue(int batchSize, const void* const* inputs, void** outputs, void* workspace, cudaStream_t stream)

Foe example in the case where

  • the input and output ITensor have kFLOAT type
  • and this custom layer also have kFLOAT type,

does casting the inputs[0] to some pointer type which is not consistent with kFLOAT (like int*) violate
the strict-aliasing rules ?
In other words, does this fall under the following ?

If a program attempts to access the stored value of an object through a glvalue whose type is not similar ([conv.qual]) to one of the following types the behavior is undefined
http://eel.is/c++draft/basic.lval#11

From my understanding, this is legal as long as inputs and outputs are just allocated storage with no dynamic type.

Thank you!

As long as you follow the rules, using void* pointers is safe. But it is your responsibility to check that you follow the rules, otherwise the behavior is undefined.

See for example:



https://en.cppreference.com/w/cpp/language/reinterpret_cast

Yes, that’s right, but that’s not what I meant.

To be more precisely, if TensorRT internally has the pointer to the storage, say, as float*, and casts it to void* and passes it to the enqueu function, it is illegal to convert inputs/outputs to any pointer type except for float* (, char*, unsigned char* or std::byte*).

I just want to make sure that this is not the case so that I can freely use inputs/outputs (,of course, as long as I stay within the rules as you said) !

I’ll show you a simple example below.

  1. This violates the rules.
// Inside the framework
float *data = allocate(); // If DataType is kFLOAT
enqueue((void *)data);
// My code
void enqueue(void *data) { int *input = (int *)data; }
  1. This follows the rules.
// Inside the framework
void *data = allocate(); // No matter what type DataType is
enqueue(data);
// My code
void enqueue(void *data) { int *input = (int *)data; }