Serializing with H.265 encoding corrupts video files

Autonomous Vehicles DRIVE AGX Orin DRIVE AGX Orin General
1

Please provide the following info (tick the boxes after creating this topic):
Software Version
[ *] DRIVE OS 6.0.10.0
[ *] DRIVE OS 6.0.8.1
DRIVE OS 6.0.6
DRIVE OS 6.0.5
DRIVE OS 6.0.4 (rev. 1)
DRIVE OS 6.0.4 SDK
other

Target Operating System
[*] Linux
QNX
other

Hardware Platform
DRIVE AGX Orin Developer Kit (940-63710-0010-300)
DRIVE AGX Orin Developer Kit (940-63710-0010-200)
DRIVE AGX Orin Developer Kit (940-63710-0010-100)
DRIVE AGX Orin Developer Kit (940-63710-0010-D00)
DRIVE AGX Orin Developer Kit (940-63710-0010-C00)
[*] DRIVE AGX Orin Developer Kit (not sure its number)
other

SDK Manager Version
[*] 2.1.0
other

Host Machine Version
native Ubuntu Linux 20.04 Host installed with SDK Manager
[*] native Ubuntu Linux 20.04 Host installed with DRIVE OS Docker Containers
native Ubuntu Linux 18.04 Host installed with DRIVE OS Docker Containers
other

Issue Description
I am using Driveworks SDK, but enabling ultrafast mode for h265 encoding causes video corruption.
Furthermore, disabling ultrafast mode causes encoding to take more than 200 ms, making it impossible to encode at 10 FPS.
I use six cameras simultaneously, but even with just one camera, file corruption and processing delays occur.

Neither of these issues occurred in version 6.0.6.

Camera: Sony IMX728EVB-MLH-SMM3 *6

In our implementation, similar to the code example in Sensors Serialization,
the frames obtained by dwSensorCamera_readFrame() are
output to a file using dwSensorSerializer_serializeCameraFrameAsync().

Error String
no errors reported.

Logs

3

Do you notice it with other camera modules as well?

Are you looking to perform h265 encoding of 6 * 8M@30fps?

4

We have also confirmed that files are corrupted when ultrafastmode is enabled in Sony IMX623EVB-MLH-SMM13.

Ideally, it would be desirable to save at 30 FPS, but considering the current requirements, saving at 10 FPS is sufficient.

5

Dear @atsushi_01_iwai ,
I believe the write BW of storage must be low. Could you check the write BW of the storage ?
I notice DW APIs don’t use UHP preset for H265 encoding. Can you use H264/mp4 in your use case? Could you check if 6 cams * 2160p@10fps works for H264? you can use frame-mod=3 in rig config to get 10FPs with recorder tool. Please see DriveWorks SDK Reference: Configuration Reference

6

Using h265 encoding is a hard requirement for your use case? If not, please test with h264 and update us.

7

Thank you.
I tried the recorder tool with 6 cameras at 10 FPS using H.264 encoding, but the results showed that more than 25% of cases exceeded 130 ms between frames.
My settings are as follows.
camera-name=IMX728_RGGB,interface=csi-gh,link=0,output-format=processed,format=h264,bitrate=210000000,CPHY-mode=1,frame-mod=3

A bitrate of 210 Mbps was the value that allowed recording with a frame interval of 100 milliseconds in version 6.0.6.
However, even when set to less than 128 Mbps in version 6.0.8.1, the issue was not resolved.

Additionally, NOTIF_WARN_ICP_FRAME_DROP was continuously notified during the execution of the recorder.

8

Could you update on this. I believe storage write BW is route cause of frame drops. Increase camera count from 1 to 6 with 10 FPS with h264 encoding to identify when you see frame drops notifications to map to storage writing BW.

9

The dd command check showed a bandwidth of 700 MB/s at multiplicity 1 and 135 MB/s at multiplicity 6.

10

Could you provide any update on this?
Also, can you test using mp4 format as well.

  1. Please share dd command output on storage device.
  2. Please share recorder logs and used config json for review.
12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.