Xavier motion vectors, pixel location

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
1

Hi Everyone, Sorry if this has been asked before but I’m looking at the Xavier motion vector struct info:

https://docs.nvidia.com/jetson/archives/l4t-multimedia-archived/l4t-multimedia-281/structMVInfo__.html

and I don’t see where on the image each motion vector is located. How do I get the pixel location of each vector as well as its components?

Thanks,
Dave

2

Hi,
Please refer to tegra_multimedia_api sample 01_video_encode

/usr/src/tegra_multimedia_api/samples/01_video_encode/
3

Hi, Thanks for your answer. Can you maybe give me a clue as to where to look in those files? I have the impression that each motion vector is associated with a 32X32 pixel block of the image and I need to know within that block where the origin (tail) of the vector is located. For example, to draw the vector on an image.

Thanks,
Dave

4

Hi,
The information is being printed out. Please check the code in 01_video_encode:

if (ctx->dump_mv)
{
    v4l2_ctrl_videoenc_outputbuf_metadata_MV enc_mv_metadata;
    if (ctx->enc->getMotionVectors(v4l2_buf->index, enc_mv_metadata) == 0)
    {
        uint32_t numMVs = enc_mv_metadata.bufSize / sizeof(MVInfo);
        MVInfo *pInfo = enc_mv_metadata.pMVInfo;

        cout << "Frame " << frame_num << ": Num MVs=" << numMVs << endl;

        for (uint32_t i = 0; i < numMVs; i++, pInfo++)
        {
            cout << i << ": mv_x=" << pInfo->mv_x <<
                " mv_y=" << pInfo->mv_y <<
                " weight=" << pInfo->weight <<
                endl;
        }
    }
}
5

Perhaps I’m missing something. What is being printed above are the motion vector x and y components and a weight. What I need given the index i above into the motion vector array are the coordinates on the image of the origin of each vector. For instance, my image has dimensions 1080 rows by 1280 columns. I’m receiving an array of 1350 motion vectors, which is (1080 / 32) * (1280 / 32). How do I compute the location in the image of each vector? From what I can tell MVInfo does not contain this (or does it?)

6

Hi,
We’re checking to share more information on this function.

7

Thanks! Much appreciated.

8

Hi,
You can run following commands to get the log:

$ /usr/src/tegra_multimedia_api/samples/00_video_decode/video_decode H264 -o a.yuv -f 2 /usr/src/tegra_multimedia_api/data/Video/sample_outdoor_car_1080p_10fps.h264
$ /usr/src/tegra_multimedia_api/samples/01_video_encode/video_encode a.yuv 1920 1080 H264 a.h264 --mvdump > ~/mvdump.log 2>&1

It reports:

Frame 0: Num MVs=8100

8100 is (1920/16)*(1080/16).

For H264, it is (width/16)(height/16). For H265, it is (width/32)(height/32).

9

Hi Thanks for that. I understand where the total number of vectors comes from. What I don’t see is how to assign an origin on the image to each vector in the array. For instance, where is the tail of the first vector in the array located on the image? In my case, H265, the answer seems to be somewhere inside the upper left 32X32 block of pixels in the image. Is it the center of that block? The top left corner? Somewhere else? Any info you could give me on that would be appreciated.

Thanks,
Dave

10

Hi,
You may try Elecard Tool, which uses center of block to represent the MVs. It shall offer information about representation of vector.

11

Really not sure why you would refer me to a 3rd party tool. If I find an answer to this question I’ll post it here but just bear in mind that the motion vectors are not useful to a user if we can’t locate them on an image.

1 Like
12

Hi,
This is the way we check the motion vectors. If you have other tools/methods, please try to set center of block.