Hello!
My goal
My final goal is to get rig file which contains sensors (cameras) calibration data using ocam camera model. I am trying to calibrate setup with 4 cameras (camera-type=ar0231-rccb-bae-sf3324). I’m following steps from “Static Camera Calibration” tutorial.
Current state
I have collected all the calibration data needed and I’m executing extrinsic calibration routine (calibration-graph-cli executable). I have two sets of intrinsics calibration files produced by using calibration-intrinsics-constraints tool. One set is generated using --camera-model=ocam parameter, the other one is using the default - ftheta. I’m trying the calibration-graph-cli tool with both intrinsic calibration outputs separately.
The problem
When executing extrinsic calibration using intrinsic calibration files generated for ocam model, the tool hangs, which doesn’t happen with ftheta calibrated intrinsics files. Altough graph.json is generated, cameras [camera-name]-refined-intrinsics.json files are not. Looking top, the calibration-gra process is occupying 100% of corresponding CPU core and the calibration program window has “Loading…” on its status bar.
Full output of calibration-graph-cli execution:
Arguments:
--data=
--dir=calib-data
--graph=
--offscreen=0
--profiling=1
--showDebug=1
--targetDB=
[05-02-2020 14:52:30] Platform: Detected Generic x86 Platform
[05-02-2020 14:52:30] TimeSource: monotonic epoch time offset is 1580900636627901
[05-02-2020 14:52:30] Platform: number of GPU devices detected 1
[05-02-2020 14:52:30] Platform: currently selected GPU device discrete ID 0
[05-02-2020 14:52:30] SDK: Resources mounted from ../../data/
[05-02-2020 14:52:30] TimeSource: monotonic epoch time offset is 1580900636627901
[05-02-2020 14:52:30] Initialize DriveWorks SDK v2.2.3136
[05-02-2020 14:52:30] Release build with GNU 7.4.0 from heads/buildbrain-branch-0-gca7b4b26e65
[05-02-2020 14:52:30] Initialize DriveWorks VisualizationSDK v2.2.3136
[05-02-2020 14:52:30] Initialize DriveWorksGL SDK v2.2.3136
[05-02-2020 14:52:30] GL-SDK: initialize OpenGL
[05-02-2020 14:52:30] Loading calibration data from directory structure, dir=calib-data
[05-02-2020 14:52:30] Target database loaded
Cameras found: camera0 camera2 camera1 camera3
Debug: intrinsics json of external camera not found (intrinsics/external.json). Falling back to calibrate these from images in external/ subfolder
[05-02-2020 14:52:30] Saving calibration data description to calib-data/calib-data.json
Parsing intrinsics
Camera camera0
Camera camera2
Camera camera1
Camera camera3
Camera external
Image size=[ 6000, 4000 ]
calib-data/external/_DSC0441.JPG: targets 95(35 tags) 92(35 tags) 90(31 tags)
calib-data/external/_DSC0453.JPG: targets 96(35 tags) 94(35 tags), 10 invalid tags
calib-data/external/_DSC0455.JPG: targets 181(9 tags) 91(35 tags) 89(35 tags), 14 invalid tags
calib-data/external/_DSC0456.JPG: targets 183(9 tags) 93(35 tags)
calib-data/external/_DSC0450.JPG: targets 182(9 tags) 180(8 tags) 96(33 tags) 91(35 tags) 89(35 tags), 25 invalid tags
calib-data/external/_DSC0435.JPG: targets 180(9 tags) 93(35 tags), 3 invalid tags
calib-data/external/_DSC0442.JPG: targets 181(9 tags) 91(35 tags) 90(34 tags) 89(35 tags), 1 invalid tags
calib-data/external/_DSC0448.JPG: targets 180(9 tags) 96(35 tags) 94(35 tags) 89(35 tags), 5 invalid tags
calib-data/external/_DSC0437.JPG: targets 182(9 tags) 91(20 tags) 90(35 tags), 9 invalid tags
calib-data/external/_DSC0451.JPG: targets 180(9 tags) 93(35 tags), 8 invalid tags
calib-data/external/_DSC0449.JPG: targets 182(9 tags) 180(9 tags) 96(35 tags) 91(30 tags) 89(35 tags), 9 invalid tags
calib-data/external/_DSC0431.JPG: targets 183(9 tags) 181(9 tags) 95(23 tags) 91(35 tags) 90(35 tags), 7 invalid tags
calib-data/external/_DSC0447.JPG: targets 183(9 tags) 181(5 tags) 95(35 tags) 91(22 tags) 90(35 tags), 2 invalid tags
calib-data/external/_DSC0436.JPG: targets 182(9 tags) 180(9 tags) 96(30 tags) 93(25 tags), 15 invalid tags
calib-data/external/_DSC0438.JPG: targets 91(35 tags) 90(35 tags) 89(35 tags), 1 invalid tags
calib-data/external/_DSC0446.JPG: targets 95(35 tags) 92(35 tags) 90(35 tags), 8 invalid tags
calib-data/external/_DSC0445.JPG: targets 183(9 tags) 181(9 tags) 95(35 tags) 91(31 tags) 90(35 tags), 5 invalid tags
Detections don't agree.
calib-data/external/_DSC0454.JPG: targets 95(35 tags) 92(35 tags) 90(16 tags)
calib-data/external/_DSC0440.JPG: targets 94(35 tags) 89(35 tags)
calib-data/external/_DSC0434.JPG: targets 182(9 tags) 180(9 tags) 96(35 tags) 91(30 tags) 89(35 tags), 7 invalid tags
calib-data/external/_DSC0443.JPG: targets 95(35 tags), 23 invalid tags
calib-data/external/_DSC0444.JPG: targets 181(9 tags) 91(35 tags) 89(28 tags), 12 invalid tags
calib-data/external/_DSC0433.JPG: targets 180(8 tags) 96(35 tags) 94(25 tags) 89(35 tags), 7 invalid tags
calib-data/external/_DSC0452.JPG: targets 180(9 tags) 96(35 tags), 12 invalid tags
calib-data/external/_DSC0432.JPG: targets 183(9 tags) 95(35 tags) 94(16 tags) 90(35 tags), 12 invalid tags
calib-data/external/_DSC0457.JPG: targets 183(8 tags) 95(35 tags) 93(21 tags)
calib-data/external/_DSC0439.JPG: targets 96(35 tags) 94(35 tags), 5 invalid tags
Parsing extrinsics' constraints
New pose for camera camera0
Found constraint between [camera0, target-96]: 140 points
Found constraint between [camera0, target-94]: 140 points
Found constraint between [camera0, target-89]: 80 points
New pose for camera camera2
Found constraint between [camera2, target-91]: 140 points
Found constraint between [camera2, target-90]: 140 points
Found constraint between [camera2, target-89]: 140 points
New pose for camera camera1
Found constraint between [camera1, target-95]: 140 points
Found constraint between [camera1, target-92]: 140 points
New pose for camera camera3
Found constraint between [camera3, target-93]: 140 points
New pose for camera external
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-92]: 140 points
Found constraint between [external, target-90]: 124 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-94]: 140 points
Found constraint between [external, target-181]: 36 points
Found constraint between [external, target-91]: 140 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-183]: 36 points
Found constraint between [external, target-93]: 140 points
Found constraint between [external, target-182]: 36 points
Found constraint between [external, target-180]: 32 points
Found constraint between [external, target-96]: 132 points
Found constraint between [external, target-91]: 140 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-93]: 140 points
Found constraint between [external, target-181]: 36 points
Found constraint between [external, target-91]: 140 points
Found constraint between [external, target-90]: 136 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-94]: 140 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-182]: 36 points
Found constraint between [external, target-91]: 80 points
Found constraint between [external, target-90]: 140 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-93]: 140 points
Found constraint between [external, target-182]: 36 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-91]: 120 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-183]: 36 points
Found constraint between [external, target-181]: 36 points
Found constraint between [external, target-95]: 92 points
Found constraint between [external, target-91]: 140 points
Found constraint between [external, target-90]: 140 points
Found constraint between [external, target-183]: 36 points
Found constraint between [external, target-181]: 20 points
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-91]: 88 points
Found constraint between [external, target-90]: 140 points
Found constraint between [external, target-182]: 36 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-96]: 120 points
Found constraint between [external, target-93]: 100 points
Found constraint between [external, target-91]: 140 points
Found constraint between [external, target-90]: 140 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-92]: 140 points
Found constraint between [external, target-90]: 140 points
Found constraint between [external, target-183]: 36 points
Found constraint between [external, target-181]: 36 points
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-91]: 124 points
Found constraint between [external, target-90]: 140 points
Detections don't agree.
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-92]: 140 points
Found constraint between [external, target-90]: 64 points
Found constraint between [external, target-94]: 140 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-182]: 36 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-91]: 120 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-181]: 36 points
Found constraint between [external, target-91]: 140 points
Found constraint between [external, target-89]: 112 points
Found constraint between [external, target-180]: 32 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-94]: 100 points
Found constraint between [external, target-89]: 140 points
Found constraint between [external, target-180]: 36 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-183]: 36 points
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-94]: 64 points
Found constraint between [external, target-90]: 140 points
Found constraint between [external, target-183]: 32 points
Found constraint between [external, target-95]: 140 points
Found constraint between [external, target-93]: 84 points
Found constraint between [external, target-96]: 140 points
Found constraint between [external, target-94]: 140 points
[05-02-2020 14:54:09] Saved graph to calib-data/graph.json
Initializing camera intrinsics
Initializing intrinsics for camera external
FOV: 76.6475
Initializing poses for intrinsic constraints for camera external
Stats: Median=nan, mean=5.14218, var=79.9195, std dev=8.93977, sample count=9148, min=0.0175034, max=189.403
Initializing intrinsics for camera camera3
OcamCalib: initial calib
Closed form result: 960,604,978.544,0,0.000450655,-1.44354e-07,1.59029e-10, error: 7.04952e+08
OcamCalib: refine center
OcamCalib: new calib
OcamCalib: non-linear
OcamCalib: find inv poly
Final result: 999.438,882.524,1.09042,-0.0312137,0.00356704
: ,1187.87,0,0.000155169,-2.28898e-08,-3.07516e-10
: ,59417.9,-350291,902255,-1.29494e+06,1.12829e+06,-610473,199679,-35851,2667.18
Closed-form camera initialization:
center=999.438,882.524
affine=1.09042,-0.0312137,0.00356704
poly=[1187.87,0,0.000155169,-2.28898e-08,-3.07516e-10,]
inv poly=59417.9,-350291,902255,-1.29494e+06,1.12829e+06,-610473,199679,-35851,2667.18,]
reprojection error (libintrcalib)=4.92601
FOV: 108.064
Initializing poses for intrinsic constraints for camera camera3
Stats: Median=nan, mean=2.91506, var=6.95974, std dev=2.63813, sample count=51220, min=0.00112915, max=38.8925
Initializing intrinsics for camera camera1
OcamCalib: initial calib
Closed form result: 960,604,958.588,0,0.000451352,-1.21809e-07,1.32997e-10, error: 9.54736e+08
OcamCalib: refine center
OcamCalib: new calib
OcamCalib: non-linear
OcamCalib: find inv poly
Final result: 925.51,809.438,1.03895,-0.0109369,-0.00204915
: ,1187.19,0,-3.35177e-05,-5.45944e-07,5.17881e-10
: ,226035,-1.39466e+06,3.80358e+06,-5.94392e+06,5.80909e+06,-3.63232e+06,1.41804e+06,-315835,30713.4
Closed-form camera initialization:
center=925.51,809.438
affine=1.03895,-0.0109369,-0.00204915
poly=[1187.19,0,-3.35177e-05,-5.45944e-07,5.17881e-10,]
inv poly=226035,-1.39466e+06,3.80358e+06,-5.94392e+06,5.80909e+06,-3.63232e+06,1.41804e+06,-315835,30713.4,]
reprojection error (libintrcalib)=4.07966
FOV: 83.0537
Initializing poses for intrinsic constraints for camera camera1
Stats: Median=nan, mean=1.9924, var=3.39573, std dev=1.84275, sample count=57648, min=0.0037879, max=39.4414
Initializing intrinsics for camera camera0
OcamCalib: initial calib
Closed form result: 960,604,1004.64,0,0.000460394,-1.09848e-07,9.97693e-11, error: 3.79994e+08
OcamCalib: refine center
OcamCalib: new calib
OcamCalib: non-linear
OcamCalib: find inv poly
Final result: 811.223,1016.83,1.04736,-0.0381014,-0.0312305
: ,996.653,0,-0.000832751,3.59157e-06,-2.56629e-09
: ,1.08551e+07,-7.56436e+07,2.28335e+08,-3.90216e+08,4.13237e+08,-2.77867e+08,1.15921e+08,-2.74456e+07,2.82463e+06
Closed-form camera initialization:
center=811.223,1016.83
affine=1.04736,-0.0381014,-0.0312305
poly=[996.653,0,-0.000832751,3.59157e-06,-2.56629e-09,]
inv poly=1.08551e+07,-7.56436e+07,2.28335e+08,-3.90216e+08,4.13237e+08,-2.77867e+08,1.15921e+08,-2.74456e+07,2.82463e+06,]
reprojection error (libintrcalib)=12.4403
FOV: 94.2833
Initializing poses for intrinsic constraints for camera camera0
Stats: Median=nan, mean=6.59299, var=108.387, std dev=10.4109, sample count=49756, min=0.0100475, max=376.417
Initializing intrinsics for camera camera2
OcamCalib: initial calib
Closed form result: 960,604,945.493,0,0.000283503,2.53993e-07,-1.68049e-10, error: 8.19257e+07
OcamCalib: refine center
OcamCalib: new calib
OcamCalib: non-linear
OcamCalib: find inv poly
Final result: 850.182,876.54,0.994052,-0.00745689,-0.00300687
: ,1220.76,0,0.000485356,-1.91977e-06,1.46054e-09
: ,1.26303e+06,-7.95296e+06,2.19197e+07,-3.44851e+07,3.38526e+07,-2.12258e+07,8.29887e+06,-1.84942e+06,179823
Closed-form camera initialization:
center=850.182,876.54
affine=0.994052,-0.00745689,-0.00300687
poly=[1220.76,0,0.000485356,-1.91977e-06,1.46054e-09,]
inv poly=1.26303e+06,-7.95296e+06,2.19197e+07,-3.44851e+07,3.38526e+07,-2.12258e+07,8.29887e+06,-1.84942e+06,179823,]
reprojection error (libintrcalib)=1.77897
FOV: 62.5276
Initializing poses for intrinsic constraints for camera camera2
Stats: Median=nan, mean=1.53439, var=1.06613, std dev=1.03253, sample count=28420, min=0.00848059, max=9.48672
Refining camera intrinsics
Refining intriniscs for camera external
Error before: Median=nan, mean=5.14218, var=79.9195, std dev=8.93977, sample count=9148, min=0.0175034, max=189.403
iter cost cost_change |gradient| |step| tr_ratio tr_radius ls_iter iter_time total_time
0 2.672848e+04 0.00e+00 2.96e+15 0.00e+00 0.00e+00 1.00e+04 0 1.14e-02 1.46e-02
1 2.853256e+04 -1.80e+03 0.00e+00 1.08e+03 -5.26e-01 5.00e+03 1 9.36e-03 2.40e-02
2 2.456842e+04 2.16e+03 3.06e+20 6.90e+02 8.05e-01 6.47e+03 1 1.86e-02 4.26e-02
3 2.132344e+04 3.24e+03 7.86e+19 6.72e+02 1.08e+00 1.94e+04 1 1.94e-02 6.20e-02
4 1.961670e+04 1.71e+03 2.96e+19 6.15e+02 1.08e+00 5.82e+04 1 1.91e-02 8.11e-02
5 2.297316e+04 -3.36e+03 0.00e+00 2.96e+02 -1.37e+00 2.91e+04 1 9.05e-03 9.02e-02
6 1.831047e+04 1.31e+03 2.38e+19 2.38e+02 8.53e-01 4.49e+04 1 1.86e-02 1.09e-01
7 1.903173e+04 -7.21e+02 0.00e+00 1.99e+02 -3.21e-01 2.25e+04 1 9.02e-03 1.18e-01
8 1.694881e+04 1.36e+03 4.31e+19 1.24e+02 9.29e-01 6.10e+04 1 1.88e-02 1.37e-01
9 2.656222e+04 -9.61e+03 0.00e+00 3.06e+02 -3.88e+00 3.05e+04 1 8.67e-03 1.45e-01
10 1.619285e+04 7.56e+02 1.51e+20 1.43e+02 4.85e-01 3.05e+04 1 2.15e-02 1.67e-01
11 1.501040e+04 1.18e+03 2.01e+20 1.55e+02 5.12e-01 3.05e+04 1 1.94e-02 1.86e-01
12 1.373842e+04 1.27e+03 2.38e+20 1.73e+02 4.76e-01 3.05e+04 1 1.90e-02 2.05e-01
13 1.232739e+04 1.41e+03 2.61e+20 1.87e+02 4.73e-01 3.05e+04 1 1.96e-02 2.25e-01
14 1.074556e+04 1.58e+03 2.69e+20 1.94e+02 5.01e-01 3.05e+04 1 1.94e-02 2.44e-01
15 9.016715e+03 1.73e+03 2.58e+20 1.96e+02 5.56e-01 3.05e+04 1 1.91e-02 2.63e-01
16 7.262116e+03 1.75e+03 2.29e+20 1.90e+02 6.35e-01 3.11e+04 1 1.94e-02 2.83e-01
17 5.726107e+03 1.54e+03 1.85e+20 1.80e+02 7.19e-01 3.40e+04 1 1.91e-02 3.02e-01
18 4.622418e+03 1.10e+03 1.38e+20 1.65e+02 7.95e-01 4.27e+04 1 1.96e-02 3.22e-01
19 3.951153e+03 6.71e+02 9.50e+19 1.46e+02 8.60e-01 6.81e+04 1 1.94e-02 3.41e-01
20 3.584422e+03 3.67e+02 6.60e+19 1.23e+02 9.01e-01 1.41e+05 1 1.92e-02 3.60e-01
21 3.373511e+03 2.11e+02 4.43e+19 8.78e+01 9.23e-01 3.55e+05 1 1.96e-02 3.80e-01
22 3.245395e+03 1.28e+02 2.90e+19 1.65e+02 9.17e-01 8.46e+05 1 1.94e-02 3.99e-01
23 3.169129e+03 7.63e+01 1.16e+19 2.64e+02 1.03e+00 2.54e+06 1 2.19e-02 4.21e-01
24 3.144759e+03 2.44e+01 2.69e+18 2.26e+02 1.16e+00 7.62e+06 1 2.44e-02 4.46e-01
25 3.140450e+03 4.31e+00 4.01e+17 8.59e+01 1.36e+00 2.28e+07 1 2.06e-02 4.66e-01
26 3.139341e+03 1.11e+00 5.34e+16 2.05e+01 1.61e+00 6.85e+07 1 2.05e-02 4.87e-01
27 3.138887e+03 4.54e-01 8.31e+15 5.81e+00 1.67e+00 2.06e+08 1 2.13e-02 5.08e-01
28 3.138683e+03 2.04e-01 1.66e+15 2.67e+00 1.68e+00 6.17e+08 1 1.98e-02 5.28e-01
29 3.138589e+03 9.36e-02 3.85e+14 1.57e+00 1.68e+00 1.85e+09 1 1.98e-02 5.48e-01
30 3.138546e+03 4.32e-02 7.35e+13 9.96e-01 1.68e+00 5.55e+09 1 2.08e-02 5.68e-01
31 3.138526e+03 2.01e-02 1.09e+13 6.54e-01 1.68e+00 1.67e+10 1 1.99e-02 5.88e-01
32 3.138517e+03 9.40e-03 3.08e+13 4.38e-01 1.69e+00 5.00e+10 1 1.91e-02 6.08e-01
33 3.138512e+03 4.44e-03 3.08e+13 2.97e-01 1.69e+00 1.50e+11 1 1.98e-02 6.27e-01
Intrinsics refinement for camera external:
Solver Summary (v 1.13.0-eigen-(3.2.92)-no_lapack-eigensparse-openmp)
Original Reduced
Parameter blocks 90 90
Parameters 546 546
Effective parameters 545 545
Residual blocks 18296 18296
Residual 36592 36592
Minimizer TRUST_REGION
Sparse linear algebra library EIGEN_SPARSE
Trust region strategy LEVENBERG_MARQUARDT
Given Used
Linear solver SPARSE_SCHUR SPARSE_SCHUR
Threads 8 8
Linear solver threads 1 1
Linear solver ordering AUTOMATIC 89,1
Schur structure 2,6,11 2,d,d
Cost:
Initial 2.672848e+04
Final 3.138512e+03
Change 2.358996e+04
Minimizer iterations 34
Successful steps 30
Unsuccessful steps 4
Time (in seconds):
Preprocessor 0.0032
Residual evaluation 0.0202
Jacobian evaluation 0.3089
Linear solver 0.2480
Minimizer 0.6328
Postprocessor 0.0002
Total 0.6362
Termination: CONVERGENCE (Function tolerance reached. |cost_change|/cost: 6.749904e-07 <= 1.000000e-06)
Before camera: intrinsics=[ 3000, 2000, 0, 0.000185962, 0, 0, 0, 0, 5377.45, 0.0131268, -0.0296414, 0.0205994 ]
After camera: intrinsics=[ 2992.63, 2014.02, 0, 0.000248426, 1.7433e-09, -5.79674e-12, 5.94458e-16, 1.6757, 3982.54, 230.722, 326.109, 1158.12 ]
Error after: Median=nan, mean=0.812647, var=1.1868, std dev=1.08941, sample count=9148, min=0.0104613, max=38.0737
Max fov: 74.7741
Refining intriniscs for camera camera3
Error before: Median=nan, mean=2.91506, var=6.95974, std dev=2.63813, sample count=51220, min=0.00112915, max=38.8925
Note: fixing principal point, D, and E for ocam camera