My current project is to port an application I wrote in python to C++. Part of the code was to take an image captured using a videoSource object and rotating it in 90 degree increments.
In the python code I did this by accessing the image as a numpy array and using OpenCV to do the rotation. I’ve asked, in the past, if there was a way to do the rotation in the jetson-inference/jetson-util library and I was told that there wasn’t a python binding for it, but in C++ I could use cudaWarpAffine to do the rotation. Now that I’m working in C++ I wanted to look into using the cudaWarpAffine, but I could find any examples of how to use it or any documentation on using it. Can anyone please point me towards how to use this function to rotate an image in 90 degree increments?
Additionally, what’s the easiest way to take a uchar3* and get it into a format that can be used in OpenCV?
For rotating an image 90-degree using C++, cudaWarpAffine is definitely an option, but it’s not the most efficient approach. The fastest and most efficient method, especially if you’re capturing with GStreamer, is using nvvidconv flip-method=1 for rotation. Even if you’re not currently using GStreamer for capture, OpenCV’s VideoCapture supports GStreamer pipelines, where you can insert the flip element.
Thanks. I think for now just getting the uchar3* into a cv::Mat will be enough.
As for the rotation, I don’t want to rotate the image during the input, I want to be able to rotate it later on, to an arbitrary multiple of 90 degrees, and composite the rotations together for a specific use case. I need to do it programmatically after the image has already been captured from the input.
What you could do is wrap your OpenCV logic into a GStreamer element. By integrating it into a GStreamer pipeline, you could use all the built-in GStreamer elements for efficient processing and potentially optimize the rotation and compositing operations.
If it has to be pregrammatically there are a few options that all start with wrapping an NVMM buffer using NvBufSurfaceMapEglImage from the Buffer Surface Management API. Then you can use a library to do the rotation:
OpenGL: We actually have a GStreamer element that performs rotation, zoom, pan, and tilt using shaders and EGL for the transformations. It’s a bit complicated, but incredibly efficient. Essentially, you define a transformation matrix and use it as a shader, which is applied with EGL. The transformation matrix for rotation is defined as:
/* 2-D Affine Transformations (3x3 matrix):
* Rotation:
* mat[0][0] = cos(q)
* mat[0][1] = sin(q)
* mat[1][0] = -sin(q)
* mat[1][1] = cos(q)
* q specifies the angle of rotation about the origin.
*/
4. CUDA: If you want to code something from scratch
If efficiency isn’t a top priority, you can always map a CPU buffer, which is simpler, and then perform a memory copy to the GPU. Anyways, processing on the GPU is generally the best approach.
For this application it cannot be part of gstreamer. The input image needs to be normal, then it gets passed to multiple workers, one of which needs to work with rotations of the original image.
Is cudaWarpAffine not the way to do rotations?
Would it be better to read the image via openCV into a cv:Mat and do the rotates in openCV (but i’d still have to upload to GPU and download) and then convert to uchar3* for processing in jetson-inference? That seems like it would be slower.