Motion Estimation Only

Dear all,

I would like to use the motion estimation only capabilities on the TX2. We have this working with an Nvidia GPU, using the NvEnc library but this seems not portable. What is the right approach on getting this to work with the TX2; and is this possible at all ?

Best regards!

Hi Martijn_S,

I’m not sure the details of your requirement, but suppose the MMAPI could help, please get it from below links:
https://developer.nvidia.com/embedded/dlc/l4t-multimedia-api-28-2
https://developer.nvidia.com/embedded/dlc/l4t-multimedia-api-reference-28-2

And you could also refer to below topic to see if help resolving your doubts:
https://devtalk.nvidia.com/default/topic/1024771/jetson-tx2/tx2-with-jetpack-3-1-gpu-utilization-issue-for-video-encoding-msenc-nvenc/

Thanks

Dear Kaycc,

Thanks for your reply.

The motion estimation only mode I am referring to is described in:


What I need (and have achieved with the NvEnc library) is the motion estimation between consecutive frames ad infinitum.

I had already amended sample 10_camera_recording to display the motion data using the NvVideoEncoder::enableMotionVectorReporting. But in that situation I am still stuck with the frames structure of mpeg encoding. Also I have amended the NvVideoEncoder class to set V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM to 1 but this has no effect on the reported motion vectors that only use 16x16 blocks. I am also still searching how to enable quarter-pixel motion estimation.

All that has been achieved using the NvEnc library on a Maxwell graphics board.

Best regards,

Hi Martijn_S,
Now motion estimation is not supported on Tegras(TX1/TX2)

The interface is different and you need to check v4l2_ctrl_videoenc_outputbuf_metadata_MV and MVInfo defined in v4l2_nv_extensions.h
It reports information of motion vectors, but you cannot do motion estimation.

Dear DaneLLL,

I take hope in the word Now. Are there any plans to have this implemented ?

Hi Martijn S,
We don’t have plan to support motion estimation on Tegra chips.

So is there a reason you consider to run it on Tegra instead of desktop GPUs?

Hi DaneLLL,
We need the mobility of the Tegra platform. The desktop GPU test was mostly a proof of concept that (wrongly) assumed code could be ported, but also proved the hardware is capable.

Meanwhile doing some more testing on Tegra, setting I frame and IDR frame interval to 2, and using the motion reporting, it seems we can achieve similar results.

However, as stated earlier, setting V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM seems to have no effect on the reported motion vectors (nor does it generate any errors). In the same way setting V4L2_CID_MPEG_VIDEO_MPEG4_QPEL to 1 also seems to have no effect on the reported motion vectors. Am I missing something ?

Kind regards,

Hi Martijn_S,
It is as expected because we don’t have the implementation.

Hi DaneLLL,

That is a pity. These features have been implemented in the NvEnc driver so the hardware seems to support them. Are there plans for implementing them ?

No, we don’t have plan to support it on Tegra series