Transforming Optixcamera from OpenGl Camera

Hello,

The following picture shows a scene of my current project. This scene is an combination of an OpenGl scene and an OptiX scene. The scene in the quad is rendered by OptiX. The skybox around the quad is in OpenGL.

If I move the camera eye of the OpenGl scene, the quad moves as well. Therefore, the OptiX scene moves as well. This means I have to move the camera eye of the OptiX scene as well to keep both camera eyes corresponding.

Now the main point of this scene is, no matter where the camera eye of the OpenGl scene is moved, the sphere should by kept in focus, thus not move out the quad. My problem is, if I move the camera eye in the OptiX scene and change the lookat parameter like it is done for the OptiX pinhole camera:

void PinholeCamera::translate(float3 t)
{
  float3 trans = camera_u*t.x + camera_v*t.y + lookdir*t.z;//camera parameters as described in pinholecamera.h

  eye = assignWithCheck( eye, eye + trans );
  lookat = assignWithCheck( lookat, lookat + trans );
}

the scenes don’t fit into each other and the sphere still slightly moves out the quad a in the following picture.

Moreover I tried to use a rotationmatrix computed from the quaternion of the changing vectors between camera eye and center of sphere which led to a camera movement, where the sphere was completely out of focus, but the scenes nearly fit into each other.

Now my question is, is there a way to keep the sphere in focus? How and which camera parameters have to be transformed?

I hope I was able to describe my problem in a way, it is understandable.
I could attach a video, if necessary.

Thank you.