VRWorks 360 Rig Calibration Issue

Hello,
We want to use VRWorks 360 for stitching frames from four fisheye cameras in mono mode, but can’t pass calibration routine. We used nvcalib_sample application with ours .xml rig description and frames. But calibration process fails with NVCALIB_ERROR_FAILED_CONVERGENCE.
Description of lenses we are using is here Optical Devices | Fujifilm [United States]
Xml description is the following:

<?xml version="1.0" encoding="utf-8"?>
<camera_rig coord_axes="z-up" rig_diameter_cm="35.000000">
	<camera width="1248" height="1248" layout="equatorial">
		<input_calib_file name="1.jpg"/>
		<pose>
			<rotation yaw_deg="0.000000" pitch_deg="0.000000" roll_deg="0.000000"/>
			<translation x_cm="-16.5" y_cm="0.0" z_cm="0.000000"/>
		</pose>
		<optics>
			<focal_length focal_pixels="386.204524"/>
			<principal_point center_offset_x="0" center_offset_y="0"/>
			<lens type="fisheye" k1="1" k2="0" k3="0" k4="0"/>
			<fisheye_radius radius_pixels="624"/>
		</optics>
	</camera>
	<camera width="1024" height="1024" layout="equatorial">
		<input_calib_file name="2.jpg"/>
		<pose>
			<rotation yaw_deg="90.000000" pitch_deg="0.000000" roll_deg="0.000000"/>
			<translation x_cm="6.0" y_cm="-23.5" z_cm="0.000000"/>
		</pose>
		<optics>
			<focal_length focal_pixels="316.830175"/>
			<principal_point center_offset_x="0" center_offset_y="0"/>
			<lens type="fisheye" k1="0" k2="0" k3="0" k4="0"/>
			<fisheye_radius radius_pixels="512"/>
		</optics>
	</camera>
	<camera width="1024" height="1024" layout="equatorial">
		<input_calib_file name="3.jpg"/>
		<pose>
			<rotation yaw_deg="180.000000" pitch_deg="0.000000" roll_deg="0.000000"/>
			<translation x_cm="21.0" y_cm="1.0" z_cm="0.000000"/>
		</pose>
		<optics>
			<focal_length focal_pixels="316.830175"/>
			<principal_point center_offset_x="0" center_offset_y="0"/>
			<lens type="fisheye" k1="0" k2="0" k3="0" k4="0"/>
			<fisheye_radius radius_pixels="512"/>
		</optics>
	</camera>
	<camera width="1024" height="1024" layout="equatorial">
		<input_calib_file name="4.jpg"/>
		<pose>
			<rotation yaw_deg="270.000000" pitch_deg="0.000000" roll_deg="0.000000"/>
			<translation x_cm="4.5" y_cm="23.0" z_cm="0.000000"/>
		</pose>
		<optics>
			<focal_length focal_pixels="316.830175"/>
			<principal_point center_offset_x="0" center_offset_y="0"/>
			<lens type="fisheye" k1="0" k2="0" k3="0" k4="0"/>
			<fisheye_radius radius_pixels="512"/>
		</optics>
	</camera>
</camera_rig>

Can somebody explain which parameter can affect to calibration convergence?

I have a similar issue , but using a iZugar 4xZ-Cam E1 rig :

<?xml version="1.0" encoding="utf-8"?>
<camera_rig coord_axes="y-up" rig_diameter_cm="9.5">
	
    <camera width="2432" height="2434" layout="equatorial">
        <input_calib_file name="Camera4.jpg"/>
        <pose>
            <rotation yaw_deg="270.0" pitch_deg="0.0" roll_deg="-90.0" />
            <translation x_cm="1.5" y_cm="0.0" z_cm="0.0"/>
        </pose>
        <optics>
            <focal_length focal_pixels="961"/>
            <principal_point center_offset_x="0.0" center_offset_y="0.0"/>
            <lens type="fisheye" k1="-0.0" k2="-0.0" k3="0.0" k4="0.0"/>
            <fisheye_radius radius_pixels="1524"/>
        </optics>
<!--        <input_media_feed>
            <input_media_form form="file" />
            <input_media_file name="Camera4.mov"  fps="30000/1001" width="2432" height="2434"/>
        </input_media_feed> -->
    </camera>
    <camera width="2432" height="2434" layout="equatorial">
        <input_calib_file name="Camera3.jpg"/>
        <pose>
            <rotation yaw_deg="180.0" pitch_deg="0.0" roll_deg="-90.0" />
            <translation x_cm="1.5" y_cm="0.0" z_cm="0.0"/>
        </pose>
        <optics>
            <focal_length focal_pixels="961"/>
            <principal_point center_offset_x="0.0" center_offset_y="0.0"/>
            <lens type="fisheye" k1="-0.0" k2="-0.0" k3="0.0" k4="0.0"/>
            <fisheye_radius radius_pixels="1524"/>
        </optics>
 <!--       <input_media_feed>
            <input_media_form form="file" />
            <input_media_file name="Camera3.mov"  fps="30000/1001" width="2432" height="2434"/>
        </input_media_feed> -->
    </camera>
    <camera width="2432" height="2434" layout="equatorial">
        <input_calib_file name="Camera2.jpg"/>
        <pose>
            <rotation yaw_deg="90.0" pitch_deg="0.0" roll_deg="-90.0" />
            <translation x_cm="1.5" y_cm="0.0" z_cm="0.0"/>
        </pose>
        <optics>
            <focal_length focal_pixels="961"/>
            <principal_point center_offset_x="0.0" center_offset_y="0.0"/>
            <lens type="fisheye" k1="-0.0" k2="-0.0" k3="0.0" k4="0.0"/>
            <fisheye_radius radius_pixels="1524"/>
        </optics>
  <!--      <input_media_feed>
            <input_media_form form="file" />
            <input_media_file name="Camera2.mov"  fps="30000/1001" width="2432" height="2434"/>
        </input_media_feed> -->
    </camera>
    <camera width="2432" height="2434" layout="equatorial">
        <input_calib_file name="Camera1.jpg"/>
        <pose>
            <rotation yaw_deg="0.0" pitch_deg="0.0" roll_deg="-90.0" />
            <translation x_cm="1.5" y_cm="0.0" z_cm="0.0"/>
        </pose>
        <optics>
            <focal_length focal_pixels="961"/>
            <principal_point center_offset_x="0.0" center_offset_y="0.0"/>
            <lens type="fisheye" k1="-0.0" k2="-0.0" k3="0.0" k4="0.0"/>
            <fisheye_radius radius_pixels="1524"/>
        </optics>
 <!--       <input_media_feed>
            <input_media_form form="file" />
            <input_media_file name="Camera1.mov"  fps="30000/1001" width="2432" height="2434"/>
        </input_media_feed> -->
    </camera>
</camera_rig>

These values when used in the sample_calib_rig_spec.xml for the nvstitch_sample give roughly okay results, but I cannot calibrate (to fine tune the stitch) - Error 12 ?

Can somebody explain what’s failing or how I can log what is happening during calibration fail .?

In my experience, the parameters that make the convergence fail mostly are either yaw/pitch/roll values or the focal length.

I was able to make the solution converge but the output calibration file had an inexplicably high focal length value so I’ve resorted to manual calibration.

I’m using the same lens YuryAlk. I have pm’d you my input xml file that was able to generate the calibrated xml file.

If anyone was able to successfully output meaningful calibration xml file using the SDK, please let me know.

I resolved my calibration problem by removing the fisheye k values, the translation value (!) and using the original square image captures, 2432*2432… even if vlc says that the video is 2432x2434.

oh, and calculating the correct fisheye radius in pixels (key is to consider FOV against sensor dimensions)… should be 1524

Is the output xml file sensible? My output xml file contains a focal length of about 3000px whereas according to my calculations it should be 300-500px. When I use the output xml file, my stitched output is an all black image because the focal length is so high. For now I’ve manually edited the focal length back to 317px, with this value I get much better stitching results.

UPDATE: I checked the value for computed focal length in the code and it is 706827136. Also the principal points are way off.

The figures for Focal_pixels on each camera are (roughly) close to estimate - they do differ a little (10s of pixels) but seem Ok …

I figured it out, my yaw_deg setting was incorrect.