Hello,
I am trying to implement the pin hole camera model from Opencv (https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html).
Looking at the examples from the optix SDK and the advanced samnples, I am getting a little bit confused about the way the camera is implemented.
- How is the camera coordinate system defined relative to the world soordinate system?
- Is every single vector, that describes the camera represented in world coordinates?
- What is the meaning of the "camera_up" variable in the advanced samples and the tutorials of the SDK?
- Why is the pixel "d" for each traced ray normalized, so that it lies between -1 and 1?
Thanks for you help.
Please have a look my GTC 2018 presentation “An Introduction to NVIDIA OptiX” which explains how the pinhole camera inside the OptiX examples work (on slide 18 and 34). All necessary links are in this post:
https://devtalk.nvidia.com/default/topic/998546/optix/optix-advanced-samples-on-github/
In contrast to the pinhole camera picture on your linked site where the Yc vector points down, the corresponding V vector in the OptiX pinhole camera goes up. Means different handedness of the camera coordinate system.
My examples also implement a fisheye and a spherical projection:
https://github.com/nvpro-samples/optix_advanced_samples/blob/master/src/optixIntroduction/optixIntro_10/shaders/lens_shader.cu
Search for “sysLensShader” inside the samples’ source code to find how they are setup and used.
Yes. The resulting ray origin and ray directions are in world coordinates.
World coordinates inside the OptiX samples are all right-handed and y-axis up.
(The pinhole camera UVW coordinate system itself is actually left-handed and not an ortho-normal basis. )
That’s the up-vector of the camera which is used to orient the camera UVW vectors inside the sutil::calculateCameraVariables() function. It’s similar to the OpenGL helper function gluLookAt().
Because d is the normalized device coordinate of the sample on the full camera plane (sensor, or window into the world if you want) which is used to calculate the ray direction from ray origin through that sample via the UVW vectors of the pinhole camera. The UV vectors define the upper right quadrant of that full camera plane.
World coordinates inside the OptiX samples are all right-handed and y-axis up.
(The pinhole camera UVW coordinate system itself is actually left-handed and not an ortho-normal basis. )
That seems pretty confusing. Why would you use two types of conventions and not just for example the right hand system?
The UV vectors define the upper right quadrant of that full camera plane.
When I am ploting the absolute value of the V or U vector, their length is not equal to half of the corresponding width or height of the screen.
Does’nt the length of the U vector have to be half of the image width and the length of the V vector half of the image height?
I am still a little stuck. When implementing the Opencv Pinhole camera, I inserted the following code for testing in my pinhole camera .cu file:
RT_PROGRAM void pinhole_camera() {
optix::size_t2 screen = output_buffer.size();
unsigned int seed = tea<16>(screen.x * launch_index.y + launch_index.x, frame);
//Subpixel jitter: send the ray through a different position inside the pixel each time,
// to provide antialiasing.
float2 subpixel_jitter = frame == 0 ? make_float2(0.0f) : make_float2(rnd(seed) - 0.5f, rnd(seed) - 0.5f);
// d is pixel for a casted ray
float2 d = (make_float2(launch_index) + subpixel_jitter) / make_float2(screen) * 2.f - 1.f;
float3 ray_origin = eye;
float f_x = 1139.637922f;
float f_y = 1140.183577f;
float c_x = 477.1662541f;
float c_y = 653.0763205f;
float3 ray_direction = normalize(make_float3((1.0f/f_x) * (d.x - c_x), ((1.0f/f_y) * (d.y - c_y)), 1.0f));
...
...
...
My rendered Geometry renders fine (which means that my general implementation of the model should be right?).
One thing that is still broken, is the position of the camera, and the movement of the camera vie the GUI.
Could you help me with that?
It doesn’t really matter if you know your Linear Algebra. The difference is only a negation on the W vector when calculating the directions.
Woah, no!
This is all relative in some arbitrary units. This has nothing to do with absolute “screen size”. Screen means client window size in pixels (or render resolution in cells) in that case irrespective of their size in the physical world.
None of the OptiX examples knows what monitor size you’re running on or what the size of a pixel is on it. (Just the aspect ratio calculations assume pixels are square.)
Again, the UVW coordinate system is not an ortho-normal basis, means none of the three vectors is necessarily normalized, nor do they need to be orthogonal. (If it is you have a camera with 90 degree field of view on both axes. Perfect for rendering cubemap faces. I digress…)
Means the camera plane can become a parallelogram and the projection can be sheared if you set the UVW vectors accordingly.
(Sheared view frusta can be beneficial for stereo setups to reduce ghosting due to foreshortening between left and right image.)
Picture this: The UV coordinates are spanning a window into the virtual world. The ratio of U/V is normally the same as the aspect ratio of the render resolution width/height.
The vector W points from the camera position, which locks that coordinate system into space, to the center of this virtual window to the world.
Since W is not normalized, you can change the field of view by changing the length of W while keeping UV fixed. Or you can change the field of view by keeping W fixed and changing UV.
(Means you can make the UV vectors half your render resolution size and adjust the field of view by the length of the W vector if you want.)
Well, you removed the UVW vectors which orient the provided pinhole camera in space and can only translate the ray origins via the eye (camera position) now.
But because you’re implementing some fixed camera projection, the simplest approach to orient that would be to add a 3x3 matrix which transforms ray_direction by that orientation matrix of your camera.
The optix::Matrix3x3 class in optixu\optixu_matrix_namespace.h contains the necessary container and operators.
Thank you for your detailed answer. That helped a lot.