How to work with explicit batches in python

spolisetty · February 3, 2022, 4:47pm

Hi,

Looks like there is some problem in allocating buffer.
Please refer the following similar issue, which may help you.

Also, please refer the following sample.

NVIDIA/TensorRT/blob/master/samples/python/efficientnet/build_engine.py#L161


      
              inputs = [self.network.get_input(i) for i in range(self.network.num_inputs)]
              outputs = [self.network.get_output(i) for i in range(self.network.num_outputs)]
          
              log.info("Network Description")
              for input in inputs:
                  self.batch_size = input.shape[0]
                  log.info("Input '{}' with shape {} and dtype {}".format(input.name, input.shape, input.dtype))
              for output in outputs:
                  log.info("Output '{}' with shape {} and dtype {}".format(output.name, output.shape, output.dtype))
              assert self.batch_size > 0
              self.builder.max_batch_size = self.batch_size
          
          def create_engine(self, engine_path, precision, calib_input=None, calib_cache=None, calib_num_images=25000,
                            calib_batch_size=8, calib_preprocessor=None):
              """
              Build the TensorRT engine and serialize it to disk.
              :param engine_path: The path where to serialize the engine to.
              :param precision: The datatype to use for the engine, either 'fp32', 'fp16' or 'int8'.
              :param calib_input: The path to a directory holding the calibration images.
              :param calib_cache: The path where to write the calibration cache to, or if it already exists, load it from.
              :param calib_num_images: The maximum number of images to use for calibration.

Thank you.