MLPerf inference on Orin

virtual.ramblings · October 10, 2022, 9:29am

I’m trying to reproduce MLPerf inference 2.1 results on AGX Orin. The page for retinanet does not have a link from where the OpenImages dataset can be downloaded. (inference_results_v2.1/closed/NVIDIA/code/retinanet/tensorrt at master · mlcommons/inference_results_v2.1 · GitHub)

The official site for OpenImages (Open Images V6 - Download) shows 6 versions available. Which is the right version to download?

dusty_nv · October 11, 2022, 1:38pm

Hi @virtual.ramblings, we’re checking with our team who works on MLPerf, and will get back to you about it.

virtual.ramblings · October 17, 2022, 8:46am

Any updates on this?

dusty_nv · October 17, 2022, 3:08pm

Sorry for the delay - still waiting on confirmation, but due to some references in the code I’m inclined to think it’s OpenImages v6:

github.com

mlcommons/inference_results_v2.1/blob/8b68e7b6168ae4fcab44d8d26e21da73dc2d06a8/closed/NVIDIA/code/retinanet/tensorrt/preprocess_data.py#L61


      
                  # Calculated by: 
                  # np.uint32(int("3caa54fc", base=16)).view(np.dtype('float32')).item() * 127.0
                  max_abs = 2.64064
                  image_int8 = image.clip(-max_abs, max_abs) / max_abs * 127.0
                  return image_int8.astype(dtype=np.int8, order='C')
          
          

          
    preprocessor = ImagePreprocessor(loader, quantizer)
              if not val_only:
                  # Preprocess calibration set. FP32 only because calibrator always takes FP32 input.
                  preprocessor.run(os.path.join(data_dir, "open-images-v6-mlperf", "calibration", "train", "data"),
                                   os.path.join(preprocessed_data_dir, "open-images-v6-mlperf", "calibration", "Retinanet"),
                                   "data_maps/open-images-v6-mlperf/cal_map.txt", ["fp32"], overwrite)
              if not cal_only:
                  # Preprocess validation set.
                  preprocessor.run(os.path.join(data_dir, "open-images-v6-mlperf", "validation", "data"),
                                   os.path.join(preprocessed_data_dir, "open-images-v6-mlperf", "validation", "Retinanet"),
                                   "data_maps/open-images-v6-mlperf/val_map.txt", formats, overwrite)
          
          

          
def copy_openimage_annotations(data_dir, preprocessed_data_dir):

dusty_nv · October 17, 2022, 5:18pm

Hi @virtual.ramblings, there’s a link to a downloader script for the OpenImages mlperf validation set here: https://github.com/mlcommons/inference/tree/master/vision/classification_and_detection#datasets

virtual.ramblings · October 18, 2022, 10:33am

Thanks so much! Will try this out.

system · November 9, 2022, 4:24am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.