r24.2.3 : double free and NvHost

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,

I have new issu, that’s record a MJPEG buffer from eCan camera.

I understand “double free or corruption” , an error releases resources prematurely (unusable) while the application releases later.

ioctl(fd, VIDIOC_DQBUF ... => parse jpeg is good => push on Conver RGB => nvosd_put_text => Conver I420 => Encode H264 => write MP4 (FFMPEG context)

i think the message “Bogus DQT index 5” in nvjpeg lib come from USB2 bandwidth problem wih multi camera, so i try to parse good and bad MJPEG buffer.

My real problems is where i can find “falcon vic03: nvhost_ioctl_channel_submit” in NV process. I search way to find double free zone.

unfortunately , i failed to activate core dump on TX1. I can’t used gdb , this issu come after 12 hours.

Sep 19 04:26:01 tegra-ubuntu launch_apps.sh[1034]: Bogus DQT index 5
Sep 19 04:26:01 tegra-ubuntu rsyslogd-2007: action 'action 9' suspended, next retry is Wed Sep 19 04:27:31 2018 [v8.16.0 try http://www.rsyslog.com/e/2007 ]
Sep 19 04:26:01 tegra-ubuntu kernel: [31227.135126] falcon vic03: nvhost_ioctl_channel_submit: failed with err -22
Sep 19 04:26:01 tegra-ubuntu kernel: [31227.135126]
Sep 19 04:26:01 tegra-ubuntu kernel: [31227.153210] falcon vic03: nvhost_ioctl_channel_submit: failed with err -22
Sep 19 04:26:01 tegra-ubuntu kernel: [31227.153210]
Sep 19 04:26:01 tegra-ubuntu launch_apps.sh[1034]: *** Error in `/home/ubuntu/myappsecord/apps': double free or corruption (out): 0x0000007f8048d360 ***
Sep 19 04:26:02 tegra-ubuntu launch_apps.sh[1034]: Aborted
2

Hi Syd,
It looks like HW video converter VIC fails in some reason. You probably need to break down your pipeline to see which step fails.

Also not pretty sure if we support nvosd_put_text() well on r24. You can try to remove it first.

3

Thx for information about nvosd_put_text , we testing that.

There are a JPEG parse : that’s check start 0xFFD8 and End 0xFFD9 in jpeg data.

Today, i succeed to have same issu before a crash : "V4L2 capture + WRITE MP4 on USBKEY 1.0 + WRITE other file in one USB2 controller "

Corrupt JPEG data: 12 extraneous bytes before marker 0xd3
Invalid JPEG file structure: two SOI markers

bandwidth problem for cpature a good jpeg data.

In fine , I assume that the index errors come from corrupted images in the controller or USB brivers. So, I can parse double 0xFFD8 for detect a 2nd jpeg data in USB pipe.

But that will not prevent another index error ( like Bogus DQT index 5 ) if that didn’t find another SOI marks before.

I only have one trace in the trace log without any feedback for exclude this wrong Jpeg data.
Can NvJpeg raise this kind of index error? how?

4

Please share a test sample so that we can reproduce the issue.

5

it’s very difficult for share sample test .

in few word , we works with see3cam CU130.
After talk about this with e-Con support. Curiously, when it’s in darkened environment and there are many plugged in USB2 port , like SSD, other camera in jobs , we see a bandwidth problems. in result, wrong jpeg data not a vic error.

i share FD buffers (VIDIOC_QUERYBUF) : length, offset with mmap for a buffer.

_ctx->buffers[i].cam_start = (unsigned char *)mmap(NULL, _ctx->cam_buf.length,
                                PROT_READ | PROT_WRITE, MAP_SHARED, fd,
                                _ctx->buffers[i].cam_offset);

From “cam_start” , i search 0xD8 and 0xD9 marks, if not the jpeg was excluded.

if(*(_ctx->buffers[v4l2_buf.index].cam_start) != 0xFF || *(_ctx->buffers[v4l2_buf.index].cam_start + 1) != 0xD8)
 ...

 if(*(_ctx->buffers[v4l2_buf.index].cam_start + length -1) != 0xFF || *(_ctx->buffers[v4l2_buf.index].cam_start + length) != 0xD9)

I add news tests :

  • between this marks, i search if there are twice 0xD8 for exclud the jpeg data.
  • research a “true end” 0xFFD9 since the end of data : length to cam_start offset.

I do not know if the explanation is clear enough.
The idea is to filter JPEG images before processing them via libraries such as MMAPI or Gstreamer.

6

Hi Syd,
On r24.2.3, NvJpegDecoder is not verfied with continuous jpeg decoding. We suggest you get YUV422(UYVY or YUYV) form USB camera or use SW jpeg decoder.

7

I confirm … I try to filter but after a moment , there are a Jpeg error. may be a suspicious JPEG data.

a first image is filtered. But corrupted data must pass the established filter.

p 24 21:09:44 tegra-ubuntu launch_apps.sh[1037]: WARN: Nv_MJPEG_Encoder_HEVC#011:02695:#011 parseJpegImage():#011CAMERA 0 : WRONG JPEG Image ( index = 1 ) with twice mark SOI
Sep 24 21:09:44 tegra-ubuntu launch_apps.sh[1037]: dlopen: Failed to load jTegra library for Tegra Acceleration
Sep 24 21:09:44 tegra-ubuntu launch_apps.sh[1037]: libv4l2_nvvidconv (2):(1564) (ERROR) : BLIT FAILED err=2
Sep 24 21:09:44 tegra-ubuntu launch_apps.sh[1037]: libv4l2_nvvidconv (2):(1582) (ERROR) : NVVIDCONV ERROR 2
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.379987] autorecord[3451]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x83000006
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.379996] pgd = ffffffc0e3763000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.383964] [00000000] *pgd=00000001705c6003, *pmd=0000000000000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390857]
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390873] CPU: 0 PID: 3451 Comm: autorecord Not tainted 3.10.96-tegra #1
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390879] task: ffffffc05616e140 ti: ffffffc01979c000 task.ti: ffffffc01979c000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390885] PC is at 0x0
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390888] LR is at 0x7f8ab85bd8
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390893] pc : [<0000000000000000>] lr : [<0000007f8ab85bd8>] pstate: 60000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390896] sp : 0000007ee16dfcb0
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390899] x29: 0000007ee16dfd90 x28: 0000007ee16e0e20
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390905] x27: 0000007ed4000a90 x26: 0000007ed4000c30
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390910] x25: 0000007ee36e1510 x24: 0000007f8d91e000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390915] x23: 0000000000000001 x22: 000000000051f3e8
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390920] x21: 0000000000000001 x20: 000000000051f3e8
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390925] x19: 0000007eac000e18 x18: 0000000000000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390929] x17: 0000007f8ab9e928 x16: 0000000000000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390934] x15: 0000007f8e83c000 x14: 0000000000000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390939] x13: 0000007eac000e18 x12: 0000000000000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390943] x11: 0000007ee16dfc40 x10: 0000007ee16dfc40
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390948] x9 : ffffff80ffffffd0 x8 : 0000000000000040
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390953] x7 : 0000000000000000 x6 : 0000007ee16dd32c
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390958] x5 : 0000000000000000 x4 : 0000000000000000
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390962] x3 : 0000007ee16e1510 x2 : 0000007ee16e0e20
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390967] x1 : 0000000000000000 x0 : 0000007eace70370
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390971]
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.390980] Library at 0x0: 0x400000 /home/ubuntu/myappsecord/apps
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.398288] Library at 0x7f8ab85bd8: 0x7f8ab49000 /usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so
Sep 24 21:09:44 tegra-ubuntu kernel: [  188.407627] vdso base = 0x7f8e83a000

next step : test SW jpeg decoder.

Question, That’s work on TX2/r28.2.1 ?

thx

8

Hi Syd,

We need to reproduce the issue first so that we can check further.

9

On TX1, the problem is related to USB2 bandwidth and to many connected devices. My solution is to limit connected device.

for the future project, it’s migrate to TX2.
I read that’s better to use “NvBufferTransform”.
so, for this Jpeg problems , i creat a new post on TX2 section. If not, i write news here.