[MMAPI R28.1/R28.1] To reduce DPB delay of NvVideoDecoder

Hi DaneLLL,

Thanks a lot.

Only the first frame is corrupted after the resolution change?

No, unfortunately.

I saw one corrupted picture every GOP, not only the first picture after the resolution change.
In my environment, a few minutes after started it is reproduced 100%.


Today I’m testing memory leak and segmentation fault issue of streaming server side,
(the resolution change every 3 seconds)
for 25 minutes it was not reproduced, (25-32 min the green image was reproduced)
it was only once with re-open v4l2 device (converter/encoder).

Because the test script uses sample/10_camera_recording,
each resolution sequences are independent from converter/encoder internal state.

I have not yet found the conditions.

My environment:

  • 6 core enabled
  • max clock
  • tmpfs for logging
  • change irq smp_affinity
#!/bin/bash

# Set to 6 Core mode
nvpmodel -m 0

# Set interrupt procedures to certain CPU core

# ether_qos.common_irq
sh -c "echo 38 > /proc/irq/42/smp_affinity"
# 2490000.ether_qos.rx0
sh -c "echo 38 > /proc/irq/44/smp_affinity"
# 2490000.ether_qos.tx0
sh -c "echo 30 > /proc/irq/45/smp_affinity"
# xhci-hcd:usb1
sh -c "echo 20 > /proc/irq/59/smp_affinity"

# Set clocks to the highest fix mode
~/jetson_clocks.sh

Best Regards

Hi mynaemi,
We generate a stream but do not reproduce the issue. Can you also try it?

Please download from:

Username: XHNVWPBDFH
Password: GL/ac3"K
                  
Download URL: 
https://ftpservices.nvidia.com 

The login above will expire on 5/24/2018 12:00:00 AM

Hi DaneLLL,

I appreciate your effort very much.

I was downloading your test data, but our PC trouble occured and downloading was interrupted.
Now I’m retrying again.

I was uploading my test data and executable binary built in my environment concurrently with it.

Today in my environment, this issue was reproduced 4 times per 4 trial during the downloading.

Best Regards

Hi DaneLLL,

I have downloaded your test data, and tested with it.
Unfortunately, the issue (green images inserted) was reproduced.

At first, I was using “–input-nalu” option, but an error occured.

Set governor to performance before enabling profiler
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
Starting decoder capture loop thread
Could not read nal unit from file. EOF or file corrupted
1526630142.904832: [DEC] qbuf                [ 0]          0.000000 : Size=0
Input file read complete
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
Segmentation fault (core dumped)

Then I removed “–input-nalu” option, and it was reproduced.
Now I’m uploading 2 mp4 files (videos recorded by my smartphone).
Please see them.

  • MOV_0418.mp4
  • MOV_0419.mp4 (See the last 10 seconds)

Below are videos with my test data.

  • MOV_0415.mp4
  • MOV_0417.mp4 (See 0’22")

Best Regards

Hi mynaemi,
We don’t reproduce the issue in running default 00_video_decode on r28.2/TX2. Do you install all via Jetpack 3.2? Maybe you applied some patches on default tegra_multimedia_api package, making the issue?

Hi DaneLLL,

This issue is on the R28.2 only.

I have installed JetPack 3.2 normally, MMAPI was installed in the JetPack.
All the libtegrav4l2.so file, include files and C++ source files are original.
(Timestamp is 2nd March 2018)
00_video_decode sample source is modified for this analysis:

  • fix capture plane resolution of Converter
  • fix display resolution of Renderer
  • add logging output by “cout <<”
    It have been sent souce files on this #9 comment as an attached file “20180418_DecConvIssue.zip”.
    You can “diff” them.
    Today I have uploaded an executable binary file on the ftp server.

I don’t know how to get patches for MMAPI.

For the encoder timestamp issue, I replaced .so file of R28.1.
For other workaround, I modified MMAPI R28.2 of only server (encoder) side.

Now in my test environment, there are 3 JetsonTx2s.

  • R28.1 for encoder and decoder (comparison between R28.1 and R28.2)
  • R28.2 for encoder (modified for dqBuffer, denitPlane workaround)
  • R28.2 for decoder

What else can I do?
Should I re-install JetPack 3.2 again ?
(remove JetPack on the UbuntuPC, re-download, and erase JetsonTX2 flash memory ?)

Best Regards

Hi mynaemi,
We try your stream on clean r28.2 and don’t reproduce the issue.

Do you mean you replace certain .so of r28.1 to r28.2? The step seems not required.

Hi DaneLLL,

Thanks for your trial.

I replaced .so file for only R28.1 JetsonTX2, not for R28.2.

Today I borrowed a new one JetsonTX2 from another division in our company,
flashed Linux for Tegra, and installed JetPack 3.2.
(I borrowed another JetsonTX2, and installed JetPack 3.1 for comparison.)

Then I have tried it with:

  • Executable Binary: Original sample/00_video_decode/video_decode
  • My test stream data
  • Your test stream data
    Without:
  • jetson_clock.sh
  • change smp_affinity for interrupt
  • tmpfs for logging

Unfortunately, the issue was reproduced on R28.2 every trial (5-6 times).
On R28.1, the issue was not reproduced.

I have uploaded 2 videos to your ftp server, please see them.

  • With my test data: MOV_0424.mp4
  • With your test data: MOV_0425.mp4

Best Regards

Hi mynaemi,

We can’t reproduce corrupt image issue.
Decoding and playing “testStream.h264” result looks good without problem.

List our reproduced steps:

Install JetPack-3.2 (include all package)
scp your "testStream.h264" H264 file to tx2 device
Start decoding and playing
nvidia@tegra-ubuntu:~/tegra_multimedia_api/samples/00_video_decode$ ./video_decode H264 /home/nvidia/testStream-user.h264
---> App run was successful, check 4K monitor in resolution change, I don't see corrupt image issue.

Is your steps the same with us? Could you also share your steps (if different), I can try to repro issue.

Hi carolyuu,

Thanks for your reply and effort.
Have you already watched all the videos I uploaded ?

The difference is only between “scp” and “sftp”.

I could not decode and play without any corrupt image,
when I tried both:

% ./00_video_decode/video_decode H264 --input-nalu --copy-timestamp 12345 30 ./testStream.h264
% ./00_video_decode/video_decode H264 --input-nalu 12345 30 ./testStream.h264
% ./00_video_decode/video_decode H264 ./testStream.h264
% ./video_decode H264 ../testStream.h264

and after both

% sudo nvpmodel -m 0
% sudo nvpmodel -m 3

with and without

% sudo ./jetson_clocks.sh

File /usr/lib/aarch64-linux-gnu/tegra/libtegrav4l2.so

  • Size: 145,160 Bytes
  • Time: Tue, Mar. 20 2018 17:51:39

File /proc/version

Linux version 4.4.38-tegra (buildrain@mobile-u64-592) (gcc version 4.8.5 (GCC) ) #1 SMP PREEMPT Thu Mar 1 20:49:20 PST 2018

Using https://github.com/jetsonhacks/jetsonUtilities

NVIDIA Jetson TX2
 L4T 28.2.0 [ JetPack 3.2 ]
 Board: t186ref
 Ubuntu 16.04 LTS
 Kernel Version: 4.4.38-tegra
 CUDA 9.0.252

File /sys/module/tegra_fuse/parameters/tegra_chip_id

24

File /sys/module/tegra_fuse/parameters/tegra_chip_rev

4

File /sys/module/tegra_fuse/parameters/tegra_crod_mode

1

What else do I check ?

Best Regards

Hi mynaemi,

I can reproduced issue now.
We’ll investigation issue and update to you.

Hi carolyuu,

I appreciate your effort.
And I’m anxious for your report about the root cause and the permanent measure.

What is the difference between before the reproduction and now ?

Best Regards

The issue looks to be triggered by

% sudo nvpmodel -m 0
% sudo nvpmodel -m 3

Hi DaneLLL and carolyuu,

I appreciate your support so much.

I have confirmed:

Corrupted

  • Multimedia API R28.2
  • and
  • nvpmodel -m 0 or 3 or 4

Normaly

  • Multimedia API R28.1
  • or
  • nvpmodel -m 1 or 2

BTW,
“nvpmodel -m 4” seems wrong.
It does not make 2 Denver Cores enabled, but it makes 1 Denver and 1 A57 enabled.
I will not use this mode.

Best Regards

Hi carolyuu,

It has passed three weeks from there.
Has there been any progress?

Best Regards

Hi mynaemi,
We are still checking it.

It runs fine without configuring nvpmodel. Are you able to run your case without switching nvpmodel mode?

Hi DaneLLL and crolyuu,

I have been waiting for new information for the past 4 months,
but unfortunately no new (good or bad) information has been announced.

What is the state of your analysis ?

Now I’m using 4 Core mode for H.264 decoder with L4T R28.1,
because my system is one way communication.
But our next plan is using the bidirectional communication,
this problem will be fatal.
(For H.264 encoder, higher throughput is required,
then using 6 Core mode and L4T R28.2 or R28.2.1.)

Best Regards,

Hi,
Please contact NVIDIA salesperson so that we can understand your usecase/project and prioritize this issue.

when i disable dpb,I found that there were still several frames of delay.

I feel that the algorithm has set up an input buffer. When several frames are read, the decoding starts. My problem is how to make the algorithm read a frame of data and decode a frame of data.