The given information aren’t sufficient to tell what goes wrong, so I’m taking some guesses here:
I’ve recently seen this in a reproducer using OptiX 4.1.0 where the same Acceleration object was used for nodes of different types, in that case the root Group node and the GeometryGroup holding the scene’s geometry data underneath. That would be wrong and you’d need to use different Acceleration objects for these nodes.
Mind that OptiX allows you to share Acceleration structures among GeometryGroups when the underlying Geometry is exactly the same.
If that’s not it, there could still be something wrong with how you built the OptiX scene graph which could lead to such an error. Please verify all your OptiX scene graph nodes for consistency and correctness of the sizes and number of children, etc.
Other than that, make sure that all your own data structures adhere to the CUDA alignment requirements for all data types.
Complete table here: http://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#vector-types