R32.3.1.tx2-4g gstream app fails to run nvargus_daemon message

hello terrysu50z,

you may upgrade the JetPack release to include the fixes.

FYI,
we’re having some fixes for camera stack, there’re changes included in the latest JetPack release.
please refer to r32.5 Release Notes for further details.
thanks

My camera driver only works and is released for on r32.3.1, so how do I update r32.3.1

Also other hardware support is only tested with r32.3.1, it is not possible to change releases at this time.

Thanks,
Terry

hello terrysu50z,

please check with sensor vendor, we had deliver error resilience patches to several camera partners based-on r32.3.1.
thanks

hello terrysu50z,

you may upgrade your JetPack release version to JetPack-4.5 / l4t-r32.5 to include error handling,
please refer to L4T Driver Package (BSP) Sources,
there’s added handling for the EVENT_TYPE_ERROR events, thanks

I can’t upgrade to r32.5, my camera driver is only good for r32.3.1

can you tell me and the camera vendor more about enhancement from r32.5 that need to be back ported to r32.3.1

Do these changes go in the camera driver?

Do these changes go in the OS?

Please give more details about what to look at in the BSP sources,

Can you forward me the info that you sent to the camera partners? I might have to do the changes the vendor gave me the camera driver code, so I think I might need to make the changes.

I did not write the camera driver, I have no information that a camera partner would have, I am experienced in linux drivers and such, but I am a camera newbee.

Thanks,
Terry

hello terrysu50z,

please download this package, Argus_Error_Resiliency_phase2.7z (2.5 MB)

there’re kernel patches, you may integrate the changes and rebuilt/update the kernel;
please check the binaries under /usr/lib/, you should replace those binaries and perform a warm-reboot to make it works.
please note that, it’s only works for JetPack-4.3 / l4t-r32.3.1, please apply those for verification.
thanks

@JerryChang ,

I very carefully put in the kernel changes and backed up the lib files, and when I rebooted, my MIPI DSI display was black and I am really in a bind.

What in those libs could of killed my MIPI DSI display.

Terry

hello terrysu50z,

I don’t think those patches effect your display driver, as you can see, those were updates to the VI, and camera driver.
please share your kernel messages for more details, thanks

@JerryChang
added some debug to dsi side, rebuilt kernel, installed it, rebooted and Mipi DSI display came up, appears the first reboot after adding patches and libs, failed to read the device tree nvidia,dsi-reset element. It now does and more reboots are booting.

The patches might or might not have helped. I still am not 100%. I still see the messages from above when trying to start my app. It does seem to be less often, but it still does happen. Timing problem?

Next suggestion please.

Where these patches really available back in 5/2020, if so why am I just hearing about the patches now?

Thanks,
Terry

You better sharing some dmesg instead of only saying something not work. We don’t know what kind of error you have now. Share the log and save your own time.

hello terrysu50z,

as I mentioned before, it’s patches based-on r32.3.1; JetPack release is now available for JetPack-4.5.1 / l4t-r32.5.1
please share your kernel messages for more details, thanks

after putting in the patches the gstreamer video app still fails to display video on the display 100% of the time

Attached is the syslog that was generated during the app start/stop/start/stop

One of the starts did not display video.

Terry

file4Nvidia.5.7 (178.9 KB)

hello terrysu50z,

according to the logs, there are syncpt timed-out, which show low-level issue.
could you please confirm this sensor driver works without Argus Error Resiliency patches ?
thanks

Don’t the messages match the messages in the top of this topic, before you gave me the patch? So the messages as far as I can tell are the same before and after the patch.

What are we going to try next?

Is there debugging I can turn no?

Thanks,
Terry

hello terrysu50z,

the concept is…
VI engine expect sensor signaling coming continuously without failures.
the Argus Error Resiliency patches allow user-space crash, it could handles error events and shutdown the app gracefully.
you need to have implementation on application side to handle it.
for example,

...
            } else if (iEvent->getEventType() == EVENT_TYPE_ERROR) {
                const IEventError* iEventError =
                    interface_cast<const IEventError>(event);
               EXIT_IF_NOT_OK(iEventError->getStatus(), "ERROR event");

the error you seen is syncpt timed-out, the major error here’s low-level issue of sensor driver.
you need to check with sensor vendor to root cause the issue. please check with sensor vendor to ensure sensor signaling coming continuously without failures.

in addition,
please check this session, Applications Using V4L2 IOCTL Directly.
it uses V4L2 IOCTL to verify basic functionality during sensor bring-up.
please ensure the sensor stability by V4L2 IOCTL.
for example,
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

@JerryChang I will need alot more information about the application side of error handling, how does that fit into my gstreamer app?

Ran v4l2-ctl
Serial console filled up with endless messages

[327060.649168] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[327060.657119] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[327060.868832] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[327060.875292] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[327060.885187] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[327060.893341] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[327061.104828] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[327061.111329] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[327061.121265] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[327061.129212] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004

Waiting on more information about the error checking and waiting for information on the v4l2-ctl messages.

What can I do next?

Terry

hello terrysu50z

PLEASE CONTACT WITH YOUR SENSOR VENDOR
it appear an issue with camera streaming, even the basic camera access is malfunction.

please check Camera Architecture Stack. v4l2 standard control is very basic utility to access camera stream.
there’s nothing software can do if the camera stream is not working.

@JerryChang [quote=“terrysu50z, post:28, topic:170086”]
I will need alot more information about the application side of error handling, how does that fit into my gstreamer app?
[/quote]

STILL WAITING ON THIS. The vendor can’t give me the error handling information.

Terry

@JerryChang what information in the syslog does the vendor need to look at, if Nvidia is missing/timing out, do you do a retry and get the info you need?

Thanks,
Terry

I think what you need to let your vendor know is just in the comment you answered on 5/11.

If running simple v4l2-ctl command:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

can cause below error:

[327060.649168] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[327060.657119] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[327060.868832] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[327060.875292] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[327060.885187] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[327060.893341] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004
[327061.104828] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[327061.111329] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[327061.121265] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) INTR_STATUS 0x00000004
[327061.129212] nvcsi 150c0000.nvcsi: csi4_stream_check_status (0) ERR_INTR_STATUS 0x00000004

Then there is no need to check Argus Error Resiliency patches, which is for userspace error handling, at this moment.
The camera vendor should take care of such v4l2-ctl stability first.