OptiX is just a ray casting SDK with a flexible programming model which allows implementing various developer defined algorithms.
Calculating vertex attributes is fully under the developer’s control, means it’s your responsibility.
If your height-map is just a regular grid and you want to calculate the smooth vertex normal at the grid coordinates which are also the 2D vertex coordinates, that should be pretty straightforward.
You could calculate that by smoothing the face normals of the adjacent triangles at each vertex.
With a regular grid you have the necessary topology information about the adjacent triangle/vertex data and could do that either as a pre-process or at runtime.
If at all, I would do that inside a native CUDA kernel as preprocess. Then the vertex attribute data is already on the device for your OptiX 7 ray tracer to use and doesn’t need to be recalculated on closest hit program invocation.
(OptiX doesn’t care about normals. Acceleration structures are only build over vertex positions.)
The simpler approach would be to directly calculate the derivatives from the height map by doing central differencing over the height map values resp. forward or backward differencing at the height map edges.
That would be faster to do on the device by uploading the height map as a texture.
With the proper bilinear filtering that would automatically interpolate the necessary values for that differencing to work per fragment (surface hit point).
It depends on for what purpose you require the normals in the end. Only for visualization or actually per vertex data inside some final mesh result.