nvxFindHomographyNode low performance

I run video stabilizier sample from visionworks and I see that on “big” shakings performance of this node is very low, f. ex:
Find Homography time : 47.0594 ms or Find Homography time : 43.8856 ms
But on “small” shakings it’s < 2 ms.

How fix it?

Hi,

Please remember to maximize the CPU/GPU frequency to have a stable computing power:

sudo ~/jetson_clocks.sh

Thanks.

Yes, I use jetson_clocks.sh. I tested video stabilizier on JTX1, JTX2 and host PC. On all devices I have similar results: in some cases nvxFindHomographyNode is too slow.

Hi,

Do you use our video stabilizer sample?
And what are your input source? Video or camera?

We have checked our video stabilizer sample and didn’t find the issue you mentioned.
Could you help us reproduce this issue?

Thanks.

Ok, I upload to gdrive short video from webcam Logitech C270:
https://drive.google.com/open?id=0B0L0yxSRGe5vRVoxcXhQUVJDZ0U

I run video stabilizier with

./nvx_demo_video_stabilizer -s ~/test.avi

Sometimes I see next output from console:
Graph Time : 57.3297 ms <…> Find Homography time : 56.1456 ms
Graph Time : 46.7193 ms <…> Find Homography time : 45.5733 ms
Graph Time : 30.0361 ms <…> Find Homography time : 29.0394 ms

Hi,

We will reproduce this issue.
Could you share how long you meet a long execution time? In minutes, hours or days?

Thanks.

I see long execution time on some frames from video.

It’s full stabilizier log output on video from previous post:
Jetson TX2: https://pastebin.com/SKG5Jd9T
Host (MacBook Pro GT750M): https://pastebin.com/SzKzT9V5

Hi,

Thanks for your video. We can reproduce this issue now.
We find it takes a long time to calculate the homography matrix.
Will update more information to you later.

Graph Time : 51.4923 ms
         RGB to gray time : 0.079839 ms
         Copy time : 0.108319 ms
         Pyramid time : 0.143071 ms
         Optical Flow time : 0.466239 ms
<b>         Find Homography time : 49.8289 ms</b>
         Homography Filter time : 0.02672 ms
         Matrices Smoothing time : 0.064863 ms
         Truncate Stab Transform time : 0.012544 ms
         Warp Perspective time: 0.221822 ms
         Feature Track time : 0.407677 ms
Display Time : 62.78 ms

Thanks.

Hi,

Please based on your requirement, modify the parameter of homography algorithm.

Due to the textless input source, it’s not easy to find a robust homography matrix.
The algorithm will keep finding a suitable matrix (with RANSAC) until reach iteration maximal.
You can modify these parameters to balance accuracy and performance:

vx_node nvxFindHomographyNode 	( 	vx_graph  	graph,
		vx_array  	srcPoints,
		vx_array  	dstPoints,
		vx_matrix  	homography,
		vx_enum  	method,
		vx_float32  	threshold,
		vx_int32  	maxEstimateIters,
		vx_int32  	maxRefineIters,
		vx_float32  	confidence,
		vx_float32  	outlierRatio,
		vx_array  	inliers 
	) 		


[Graph] Computes homography matrix.

Parameters
    [in]	graph	Specifies the graph.
    [in]	srcPoints	Specifies the coordinates of the points in the original plane (array of vx_keypoint_t, nvx_keypointf_t or nvx_point2f_t structs, number of the points must be >= 4). For vx_keypoint_t and nvx_keypointf_t types, the method uses points only with non-zero tracking_status.
    [in]	dstPoints	Specifies the coordinates of the points in the target plane (it must have the same number of elements and type as srcPoints). For vx_keypoint_t and nvx_keypointf_t types, the method uses points only with non-zero tracking_status. The correspondence is between points with the same indexes in arrays.
    [out]	homography	Specifies the output homography matrix (VX_TYPE_FLOAT32 3x3 matrix).
    [in]	method	Specifies the method used to compute a homography matrix (see nvx_find_homography_method_e).
    [in]	threshold	Specifies the maximum allowed reprojection error to treat a point pair as an inlier, if RANSAC method is chosen. If srcPoints and dstPoints are measured in pixels, it usually makes sense to set this parameter somewhere in the range of 1 to 10. Default value is 3.
    [in]	maxEstimateIters	Specifies the maximum allowed number of estimation iterations for robust methods (RANSAC and LMeDS). Default value is 2000.
    [in]	maxRefineIters	Specifies the maximum allowed number of refinement iterations for robust methods (RANSAC and LMeDS). Default value is 10.
    [in]	confidence	Specifies the confidence level for robust methods (RANSAC and LMeDS). It must be a value between 0 and 1. Default value is 0.995.
    [in]	outlierRatio	Specifies the outlier ratio if LMeDS method is chosen. Default value is 0.45.
    [out]	inliers	Specifies the optional inliers/outliers output mask. The inliers[i] value is set to 0 if corresponding points pair srcPoints[i] and dstPoints[i] is marked as outlier by the method. It must be a VX_TYPE_UINT8 array with capacity greater than or equal to the capacity of srcPoints.

Returns
    vx_node

For example, we fix maximal iteration to 100, and no over 10ms execution time is observed.

diff --git a/demos/video_stabilizer/stabilizer.cpp b/demos/video_stabilizer/stabilizer.cpp
index 0b6333a..5e7d602 100644
--- a/demos/video_stabilizer/stabilizer.cpp
+++ b/demos/video_stabilizer/stabilizer.cpp
@@ -267,7 +267,7 @@ namespace
                                                       kp_curr_list,
                                                       homography,
                                                       NVX_FIND_HOMOGRAPHY_METHOD_RANSAC, 3.0f,
-                                                      2000, 10,
+                                                      100, 10,
                                                       0.995f, 0.45f,
                                                       mask);
         NVXIO_CHECK_REFERENCE(find_homography_node_);

Thanks.

1 Like

Thanks