MMAPI R28.2.1 createNvBuffer slow processing time

Hi,

We are currently developing custom use case with MMAPI based on 10_camera_recording.
FYI, we use L4T R28.2.1.

We have successfully implemented an single encoding pipeline. We use createNvBuffer (and copyToNvBuffer) to resize frame (from 1920x1080 to 640x480 for example).

.
+----------+   +----------------------------+   +------------+
|  Camera  |   | Output ! NvVideo ! Capture |   |  File      |
|  Capture |-->| plane  ! Encoder ! plane   |-->|  Recording |
|  Argus   |   |  DMA   !         !  MMAP   |   |            |
+----------+   +----------------------------+   +------------+

           ^^^^^
       createNvBuffer
        for resizing

Now, we are trying to implement an encoding pipeline with capture duplication like this:

.
                   +----------+
                   |  Video   |
               +-->|  Encoder |
               |   +----------+
               |               
               |   +----------+
               |   |  Video   |
               +-->|  Encoder |
               |   +----------+
               |               
 +----------+  |   +----------+
 |  Camera  |  |   |  Video   |
 |  Capture |--+-->|  Encoder |
 |  Argus   |  |   |          |
 +----------+  |   +----------+
               |               
               |   +----------+
               |   |  Video   |
               +-->|  Encoder |
               |   +----------+
               |               
               |   +----------+
               |   |  Video   |
               +-->|  Encoder |
                   +----------+

For achieving this, we call createNvBuffer (or copyToNvBuffer) several times for one captured frame (acquireFrame > getImage > …).

We have measured the following processing time for:

  • No resizing (from 1920x1080 to 1920x1080) : average 15.3 ms [img]http://image.noelshack.com/fichiers/2018/44/3/1540984625-single-no-resize.png[/img]
  • Resizing (from 1920x1080 to 640x480) : average 1.9 ms [img]http://image.noelshack.com/fichiers/2018/44/3/1540984627-single-resize.png[/img]

These cases are under the max frame interval for 30 and 60Hz, respectively under 16.7ms and 33.3ms, so it’s great to respect encoding requirements.

However, when calling these functions multiple times for one frame, processing time increase and couldn’t fit to max frame interval constraint.
For example, if we duplicate 5 times the captured frame, the complete processings shouldn’t exceed 33.3ms (for 30Hz) so each duplication (each createNvBuffer) shouldn’t exceed 6.7ms (33.3 / 5).
Furthermore, processing time isn’t constant:

  • No Resizing : average 12.5ms / min 6.9ms / max 20.9ms [img]http://image.noelshack.com/fichiers/2018/44/3/1540991272-multiple-no-resize.png[/img]
  • Resizing (from 1920x1080 to 640x480) : average 9.3ms / min 5.9ms / max 18.1ms [img]http://image.noelshack.com/fichiers/2018/44/3/1540991273-multiple-resize.png[/img]

These processing times are too high to achieve 25 or 30Hz encoding.
It’s not an encoder problem, it’s a copy/create problem from EGLStream/Nv/ImageNativeBuffer method.

(We have tried to use createNvBuffer only and NvBufferDestroy for each frame. We also tried using createNvBuffer (for the first buffers) then copyToNvBuffer, but we have the same results.)

Is there other methods to duplicate stream?
Why processing times are not constant? and why are they different between single resize (avg 1.9ms) and multiple resize (avg 9.3ms)?

Thanks

AurelienV,
Can you please share a patch on 10_camera_recording so that we can reproduce the issue?

Also have you tried with jetson_clocks.sh executed and ‘nvpmodel -m 0’?