Auto-magic-calib No valid tracklet matches found

Accelerated Computing Intelligent Video Analytics DeepStream SDK
,
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
NVIDIA GeForce RTX™ 3050 Ti Laptop GPU
• DeepStream Version
8.0
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
575.64.03
• Issue Type( questions, new requirements, bugs)
Application doesn’t work as intended
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
This is about GitHub - NVIDIA-AI-IOT/auto-magic-calib: Sample application repository for AutoMagicCalib tool. We are trying it with our own videos.

This was the input command:

DEEPSTREAM_IMAGE="``nvcr.io/nvidia/deepstream:8.0-gc-triton-devel``" bash launch_EndToEndCalib.sh -v "${PWD}/../assets/NAME" -o "${PWD}/../assets/NAME/output" -l "${PWD}/../assets/NAME/manual_adjustment/layout.png" > ../log.txt 2>&1

Single camera calibration works but it launch_EndToEndCalib.sh fails at “Step 2: Multi-view calibration and bundle adjustment” with error “ERROR - No valid tracklet matches found”. See log in attached file

log.txt (7.4 KB)

We tried reducing the values for tracklet filtering in configs/config_AutoMagicCalib/mv_amc_config.yaml but we get the same error. We tried only keeping the two cameras with higher overlap and we get the same error.

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

3

Can you change the log level to “DEBUG” and run from the beginning? We need more log to identify the issue. auto-magic-calib/configs/config_AutoMagicCalib/mv_amc_config.yaml at main · NVIDIA-AI-IOT/auto-magic-calib

4

In assets/NAME/output/multi_view_results/Logs, I have these logs:

INFO - ----------------------------------------------------------------------------------------------------
INFO - Step 1: read single view calib output
INFO - ----------------------------------------------------------------------------------------------------
DEBUG - Track ids: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28}
DEBUG - Track ids: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29}
INFO - ----------------------------------------------------------------------------------------------------
INFO - Step 2: tracklet matching
INFO - ----------------------------------------------------------------------------------------------------
INFO - Processing camera pair: [0, 1]
DEBUG - Filtered tracklet1: [0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28]
DEBUG - Filtered tracklet2: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, 26, 27, 28, 29]
DEBUG - Initial matching. sorted_score_dict: []
ERROR - No valid tracklet matches found

multi_view_calib_20251024_094900.log

Let me know if you need more.

5

From your log, you may need to check the content of the following single view tracking output:
/home/mar/Documents/auto-magic-calib/assets/epg-warehouse-23102025/output/single_view_results/cam_00/trajDump_Stream_0_3d.txt
/home/mar/Documents/auto-magic-calib/assets/epg-warehouse-23102025/output/single_view_results/cam_00/camInfo_hyper_00_opencv.yaml
/home/mar/Documents/auto-magic-calib/assets/epg-warehouse-23102025/output/single_view_results/cam_01/trajDump_Stream_0_3d.txt
/home/mar/Documents/auto-magic-calib/assets/epg-warehouse-23102025/output/single_view_results/cam_01/camInfo_hyper_00_opencv.yaml

6

It seems that we had to

  • include distortion for GeoCalib
  • not skip rectification

after doing these changes, i get another error (so it goes a bit further):

================================================================
Step 1: Multi-view calibration started
================================================================
Executing: docker exec multiview_2025-10-28_10-58-28 pyarmor_python "/auto-magic-calib/core/camera_estimation/multi_view_calib.py" -c /auto-magic-calib/configs/config_AutoMagicCalib/mv_amc_config.yaml -i /auto-magic-calib/input -o /auto-magic-calib/output
INFO - ----------------------------------------------------------------------------------------------------
INFO - Step 1: read single view calib output
INFO - ----------------------------------------------------------------------------------------------------
DEBUG - Track ids: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}
DEBUG - Track ids: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17}
INFO - ----------------------------------------------------------------------------------------------------
INFO - Step 2: tracklet matching
INFO - ----------------------------------------------------------------------------------------------------
INFO - Processing camera pair: [0, 1]
DEBUG - Filtered tracklet1: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
DEBUG - Filtered tracklet2: [0, 1, 2, 4, 5, 6, 7, 8, 9, 10, 12, 13, 14, 15, 16, 17]
<frozen camera_estimation.math_utils>:57: RuntimeWarning: invalid value encountered in scalar divide
DEBUG - Initial matching. sorted_score_dict: [((9, 4), 6)]
DEBUG - Hungarian algorithm matched pairs: [(1, 0), (5, 1), (9, 4), (12, 5), (14, 6), (18, 9), (19, 10), (21, 14), (22, 15), (23, 17)]
DEBUG - sorted_score_dict_hungarian: [((22, 15), 0.36332383467300217), ((21, 14), 0.4116160513941688), ((19, 10), 0.4643004537374812)]
DEBUG - Sorted tracklet matches: [[((22, 15), 0.36332383467300217), ((21, 14), 0.4116160513941688), ((19, 10), 0.4643004537374812)]]
INFO - tracklet_matches: [((0, 22), (1, 15)), ((0, 21), (1, 14)), ((0, 19), (1, 10))]
DEBUG - Tracklet to global mapping: {(0, 22): 0, (1, 15): 0, (0, 21): 1, (1, 14): 1, (0, 19): 2, (1, 10): 2}
INFO - Saved tracklet to global ID mapping to /auto-magic-calib/output/tracklet_to_global.json
INFO - ----------------------------------------------------------------------------------------------------
INFO - Step 3: multi-view calibration
INFO - ----------------------------------------------------------------------------------------------------
INFO - ====================================================================================================
INFO - Processing camera pair: [0, 1]
DEBUG - Track ids: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26}
DEBUG - Track ids: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17}
DEBUG - sorted_tracklet_matches: [[[22, 15], 0.36332383467300217], [[21, 14], 0.4116160513941688], [[19, 10], 0.4643004537374812]]
DEBUG - matches: [[22, 15], [21, 14], [19, 10]]
INFO - --- Method 3: estimate R, t using 2D matching points (robust method) ---
INFO - calibrate_2_cams_robust method
DEBUG - best_in_out_ratio: 0.4144927536231884, best_n_inliers: 143, best_n_outliers: 202
DEBUG - best_reproj_err1: 21.935595372066416, best_reproj_err2: 5.732716638038623
ERROR - Failed to estimate R and t after 100 iterations
Auto-generated camera_pairs from cam_dir: [[0, 1]]
Camera directories: ['cam_00', 'cam_01'] -> Camera numbers: [0, 1]
Traceback (most recent call last):
  File "/usr/local/bin/pyarmor_python", line 16, in <module>
    runpy.run_path(script_to_run, run_name="__main__")
  File "/usr/lib/python3.10/runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "</auto-magic-calib/core/camera_estimation/multi_view_calib.py>", line 1, in <module>
  File "<frozen camera_estimation.multi_view_calib>", line -1, in <module>
  File "<frozen camera_estimation.multi_view_calib>", line 2477, in main
  File "<frozen camera_estimation.multi_view_calib>", line 2127, in two_view_calib
TypeError: tuple indices must be integers or slices, not str
Error: Multi-view calibration failed
7

After some chatgpt to modify mv_amc_config.yaml, the mv calibration successfully finished. now the conversion to MV3DT format doesn’t work. 1st error is `[error] /auto-magic-calib/input/multi_view_results/manual_adjustment/layout.png not found`. After I add it manually, the error is `FileNotFoundError: [Errno 2] No such file or directory: ‘/auto-magic-calib/input/multi_view_results/BA_output/results_ba_scaled_world’` and that is true.

this is the mv_amc_config.yaml:

# camera directories (will be joined with base_dir from -i argument)
# The ordering defines the pairwise relationships: consecutive cameras will be paired automatically
# Camera numbers are extracted from directory names, so:
# cam_dir: ['cam_00', 'cam_01', 'cam_02', 'cam_03', 'cam_06']
# will generate camera_pairs: [[0,1], [1,2], [2,3], [3,6]]  <- Note: uses actual camera numbers!
# cam_dir: ['cam_00', 'cam_01', 'cam_02', 'cam_03']
cam_dir: ['cam_00', 'cam_01']

# camera_pairs will be auto-generated from cam_dir ordering
# Manual specification is no longer needed but can be provided to override auto-generation

# output directory is provided via -o argument to the script

# video resolution
video_resolution: [1920, 1080]

# image boundary to be ignored in tracklet data(0~1.0)
image_side_boundary: 0.05 # 0.1 means the side 10% of the image is ignored
image_bottom_boundary: 0.9 # 0.9 means the bottom 10% of the image is ignored

# tracklet filtering
tracklet_filter:
  min_length: 60 # was 90 # minimum length of the tracklet
  min_moving_dist: 1.0 # was 2.0 # minimum moving distance of the tracklet
  min_mean_moving_dist: 0.005 # was 0.01 # mean distance threshold to remove long-standing objects

# use smoothed tracklets for tracklet matching
use_smoothed_tracklets: true     # was false

# filtering for tracklet matching
# minimum number of matching points for tracklet matching
min_matching_points: 30          # was 100,60,40

# minimum and maximum scale for tracklet matching
min_scale: 0.8
max_scale: 1.3
# dtw score threshold for tracklet matching
min_dtw_score_threshold: 20 # was 25

# distance score threshold for tracklet matching
min_distance_score_threshold: 1.2

# maximum distance score for hungarian algorithm
max_distance_score_hungarian_threshold: 1 # was 0.8

# minimum number of matched tracklets for multi-view calibration
min_matched_tracklets: 2         # was 6, 3

# maximum distance between matched tracklets for multi-view calibration
max_matched_tracklets_dist: 2.0

# minimum distance between the last position and current position of the tracklet
min_tracklet_dist: 3.0 # 5 pixels

# pose estimation
ransac_threshold: 15.0           # was 10.0

# pairwise calibration
robust_two_view_calib:
  enabled: true
  min_matched_points: 40         # was 100,60
  min_matched_tracklets: 2       # keep consistent with global, was 3
  min_samples: 3 # 3 matches for RANSAC <-- fixes the RandomState.choice error
  max_iterations: 2000           # was 100, 1000
  inlier_threshold: 18.0         # px, was 10.0 pixels, 15
  min_inliers: 0.40              # was 0.60; your best ≈ 0.414, so this lets it pass # 60% of points must be inliers
  reproj_err_threshold: 50.0     # px, was 25.0, 35, 40, 50 # 25 pixels

# bundle adjustment
# skip the number of frames
skip_frame: 5 # was 10
# weight for the distance between the 3D point
distance_weight: 10.0

# ba method
ba_method: iterative # standard/iterative

# experiment
input_cam_params: ba # ba or pairwise_calib, was ba
enable_manual_adjustment: false
weight_manual_constraint: 10.0
manual_adjustment_data_source: file

# Layout image path is provided via -l argument to the script

# logging level
logging_level: DEBUG

# robust scale estimation (Umeyama)
robust_scale_estimation:
  enabled: false  # Set to false for original method
  max_iterations: 1000
  inlier_threshold: 0.5  # meters (adjust based on your scene scale)
  min_inliers: 0.6  # 60% of points must be inliers
  min_samples: 3  # minimum points for 3D similarity transform
  random_seed: 42  # for reproducibility
8

Hi,
I’m also encountering the issue: “No valid tracklet matches found” even though the cameras have overlapping views. Has your issue been resolved? If yes, could you please share the workaround? It would be really helpful for me. Thank you!

9

@ysantoshofficial Can you create your own topic for your issue?

10

Hi @Fiona.Chen , @mbru

To resolve the issue we need to capture the input videos with minimum 30% camera overlapping, which we can confirm from boundary box sampling output while running AMC. we need to get grid coverage for at least 30% in boundary box sampling. With enough people detected and tracked across cameras, we will resolve the “No valid tracklet matches found” issue.

As my issue got resolved, I will no longer create a new topic.

Thank you.