VPIWarpGrid and vpiWarpMapAllocData alignment restrictions

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Xavier
,
1

Hi, I’m trying to use the VPI library to perform remapping on the VIC. I’m specifying a dense grid as follows:

VPIWarpMap map;
memset(&map, 0, sizeof(map));
map.grid.numHorizRegions = 1;
map.grid.numVertRegions = 1;
map.grid.regionWidth[0] = dst_width;
map.grid.regionHeight[0] = dst_height;
map.grid.horizInterval[0] = 1;
map.grid.vertInterval[0] = 1;

However, after calling vpiWarpMapAllocData(), map.numHorizPoints and map.numVertPoints are not equal to dst_width and dst_height, but are rounded up to the nearest multiple of 64 and 16 respectively. Looking through the documentation, I can’t find a restriction on resolution to be a multiple. I only see restrictions on the minimum width and height. Is it required that width and height be multiples of 64 and 16? I’m running JetPack 4.6 and vpi 1.1. Also, I would expect vpiWarpMapAllocData() to generate an error instead of succeeding with rounded values. I’m also curious what happens when vpiSubmitRemap is called and the output image dimensions don’t match the width/height of the warp. Is that behavior defined? It also doesn’t seem to generate an error.

3

Hi,

Would you mind sharing a complete source so we can give it a check?
Thanks.

4

It’s part of a much larger program. I’d need to spend some time creating a standalone example, but the relevant code is this function:

bool VPIRemap::initialize(const int dst_width, const int dst_height,
                          const int src_width, const int src_height,
                          const float* xloc, const float* yloc)
{
  _dst_height = dst_height;
  _dst_width = dst_width;
  _src_height = src_height;
  _src_width = src_width;

  VPIWarpMap map;
  memset(&map, 0, sizeof(map));
  map.grid.numHorizRegions = 1;
  map.grid.numVertRegions = 1;
  map.grid.regionWidth[0] = dst_width;
  map.grid.regionHeight[0] = dst_height;
  map.grid.horizInterval[0] = 1;
  map.grid.vertInterval[0] = 1;

  if (vpiWarpMapAllocData(&map) != VPIStatus::VPI_SUCCESS)
  {
    std::cout << "vpiWarpMapAllocData failed!" << std::endl;
    return false;
  }

  if (vpiWarpMapGenerateIdentity(&map) != VPIStatus::VPI_SUCCESS)
  {
    vpiWarpMapFreeData(&map);
    std::cout << "vpiWarpMapGenerateIdentity failed!" << std::endl;
    return false;
  }

  if (map.numHorizPoints != dst_width || map.numVertPoints != dst_height)
  {
    printf("map has incorrect number of points: %d %d %d %d\n", dst_width,
           dst_height, map.numHorizPoints, map.numVertPoints);
    vpiWarpMapFreeData(&map);
    return false;
  }

  for (int i = 0; i < map.numVertPoints; ++i)
  {
    VPIKeypoint* row =
        (VPIKeypoint*)((uint8_t*)map.keypoints + map.pitchBytes * i);
    for (int j = 0; j < map.numHorizPoints; ++j)
    {
      row[j].x = xloc[dst_width * i + j];
      row[j].y = yloc[dst_width * i + j];
    }
  }

  if (vpiCreateRemap(VPI_BACKEND_VIC, &map, &_warp) != VPIStatus::VPI_SUCCESS)
  {
    vpiWarpMapFreeData(&map);
    _warp = nullptr;
    std::cout << "vpiCreateRemap failed!" << std::endl;
    return false;
  }

  vpiWarpMapFreeData(&map);

  return true;
}

The following inputs:

vpi_remap->initialize(912, 472, 912, 472, x, x);
vpi_remap->initialize(456, 236, 456, 236, x, x);

Result in these outputs:

map has incorrect number of points: 912 472 960 480
map has incorrect number of points: 456 236 512 240
5

I ended up working around this by allocating extra memory for the output image (so it could be treated as having the rounded up width and height when using vpi remap), and then ignoring the extra rows on the bottom and columns on the side in later processing. I’m wondering if this is fixed in later JetPack versions? The vpi 2.1 release notes under bug fixes says:

vpiWarpMapAllocData now allocates less memory.
6

Hi,

Sorry for the late update.
Confirmed that we can reproduce this behavior in our environment.

We are checking if there are any underlying restrictions for the VPIWarpMap with our internal team.
Will share more information with you later.

Thanks.

8

Hi,

Thanks for your patience.

Our internal team has confirmed that this is expected behavior.
We will add this information to our document.

Thanks.

9

Hi,

Got more information.
This is a hardware limitation.

Thanks.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

12

Hi,

Just for your reference.
The restrictions info can be found in JetPack5.0.2.

/opt/nvidia/vpi2/include/vpi/WarpGrid.h

#define VPI_WARPGRID_MIN_REGION_WIDTH 64       /**< Minimum warp grid region width. */
#define VPI_WARPGRID_MIN_REGION_HEIGHT 16      /**< Minimum warp grid region height. */
...
 * ### Restrictions
 *
 * * numHorizRegions cannot exceed \ref VPI_WARPGRID_MAX_HORIZ_REGIONS_COUNT.
 * * numVertRegions cannot exceed \ref VPI_WARPGRID_MAX_VERT_REGIONS_COUNT.
 * * Intervals must be power-of-two.
 * * Alignment restrictions:
 *   + `regionWidth[0]` to `regionWidth[numHorizRegions-2]` must be
 *     aligned to \ref VPI_WARPGRID_MIN_REGION_WIDTH and at least \ref VPI_WARPGRID_MIN_REGION_WIDTH.
 *   + `regionWidth[numHorizRegions-1]` must be at least \ref VPI_WARPGRID_MIN_REGION_WIDTH.
 *   + `regionHeight[0]` to `regionHeight[numVertRegions-2]` must be
 *     aligned to \ref VPI_WARPGRID_MIN_REGION_HEIGHT and at least \ref VPI_WARPGRID_MIN_REGION_HEIGHT.
 *   + `regionHeight[numVertRegions-1]` must be at least \ref VPI_WARPGRID_MIN_REGION_HEIGHT.
 *
 */

Thanks.