We have a project where we are encoding multiple streams at different resolutions and frame rates.
The resolution is detected in the input circuitry, and the encoding starts and runs
properly.
What we are seeing is that even though we can encode 3x 1080p streams, if we have
2x 1080p and 1x Analog input (720 x 525) after a while the analog input video gets
very delayed. If we shut down the 1080p streams it will jump and catch up.
If I run 2x 720p and 1x analog, the same thing happens but less severely.
It looks like the contention for resources is with the omxh264enc step.
I have tried assigning each gstreamer pipeline to a separate cpu, that didn’t help
I have tried setting the Analog input gstreamer pipeline to a higher priority (low nice),
that didn’t seem to help.
Is there any control over how the omxh264enc encoder would operate if multiple instances
are active at the same time?
I investigated this by capturing an encoded video stream of fast traffic
moving across the field of view. I decoded this into individual jpeg frames
and examined each one in sequence. It was clear that some adjacent frames were
identical, and others showed the motion of the cars from frame to frame.
Investigating the code, I did find that the frame rates were incorrect in the v4l2 driver, and
hence the gst encode pipeline. It was a 60 fps video input, but the v4l2 driver was only
configured for 30 fps, so the gst pipeline thought it was 30 fps. Setting this
correctly to 60 fps has partially solved the problem, although running 3x 1080p-60
streams still jerky. Once the values in the driver and gst pipeline were correct
(60 fps) the encoded video had smooth motion. In addition, running 2x 1080p-60
and 1x composite 30 fps, the original problem, the halts in the composite video went
away.
Interestingly, I found the HDMI camera we were using only captured 30 fps even though
it was outputting 1080p-60, i.e. every other frame was a duplicate.
I did not yet attempt modifying libgstomx.so to run with max clocks. We are still seeing
some occasional jerks in motion when encoding 3x 1080p-60, so I am going to try that.
I’m going to mark this as closed because the original problem with composite video halting has been
fixed.