iBufferOutputStream Segfault while reading capture metadata

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hallo,
When I use the libargus.so lib from https://forums.developer.nvidia.com/t/ibufferoutputstream-acquirebuffer-timeout-dosnt-fire/219059/16?u=rbayr, the main timeout problem was fixed (as discussed in the other thread). However, the iCaptureMetadata->getSensorTimestamp() function throws a segfault after the frame was acquired. I don’t get a segfault if the acquireBuffer dosn’t timeout.

The nvargus log:

=== NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...=== VCCamTest[15212]: Connection established (7FAC6071D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
(NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
(NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
(NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 107)
NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
NvPclStartPlatformDrivers: Failed to start module drivers
NvPclStateControllerOpen: Failed ImagerGUID 1. (error 0xA000E)
NvPclOpen: PCL Open Failed. Error: 0xf
SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 593)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 437)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 305)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 471)
Acquiring SCF Camera device source via index 0 has failed. ---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
=== VCCamTest[15212]: CameraProvider initialized (0x7fa4864040)LSC: LSC surface is not based on full res!
acquireBuffer++: timeout val= 1000000000
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 267)
acquireBuffer++: timeout val= 1000000000
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 267)
acquireBuffer++: timeout val= 1000000000
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 0, capture sequence ID = 2 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1018)
SCF: Error InvalidState: Something went wrong with waiting on frame start (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1077)
PowerServiceCore:handleRequests: timePassed = 2466
acquireBuffer: remainingTimeout= 264925
acquireBuffer--: bufferAcquired
(NvCapture) Error Timeout: capture waiting for status timeout (in /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_hw_linux.c, function NvCaptureLinuxGetCaptureStatus(), line 362)
(NvCapture) Error Timeout:  (propagating from /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture.c, function NvCaptureDequeue(), line 1055)
SCF: Error Timeout:  (propagating from src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 898)
SCF: Error InvalidState: Timeout waiting on frame end sensor guid 0, capture sequence ID = 2, channel = 1/1 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 905)
SCF: Error InvalidState: Something went wrong with waiting on frame end (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 991)
SCF: Error InvalidState: Something went wrong with waiting on frame end (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 991)
SCF: Error InvalidState: Something went wrong with waiting on frame end (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 991)
=== VCCamTest[15212]: Connection closed (7FAC6071D0)=== VCCamTest[15212]: WARNING: CameraProvider was not destroyed before client connection terminated.=== VCCamTest[15212]:          The client may have abnormally terminated. Destroying CameraProvider...=== VCCamTest[15212]: CameraProvider destroyed (0x7fa4864040)=== VCCamTest[15212]: WARNING: Cleaning up 1 outstanding requests...=== VCCamTest[15212]: WARNING: Cleaning up 1 outstanding stream settings...=== VCCamTest[15212]: WARNING: Cleaning up 1 outstanding sessions...SCF: Error InvalidState: Timeout waiting on frame start sensor guid 0, capture sequence ID = 4 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1018)
SCF: Error InvalidState: Something went wrong with waiting on frame start (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1077)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCompletion(), line 341)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function pause(), line 940)
SCF: Error Timeout: During capture abort, syncpoint wait timeout waiting for current frame to finish (in src/services/capture/CaptureServiceDevice.cpp, function handleCancelSourceRequests(), line 1034)
PowerServiceCore:handleRequests: timePassed = 1507
SCF: Error InvalidState: Sequence order error (6 received, 2 expected, channel 0) (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 937)
SCF: Error InvalidState: Sequence order error (7 received, 3 expected, channel 0) (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 937)
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 0, capture sequence ID = 6 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1018)
SCF: Error InvalidState: Something went wrong with waiting on frame start (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1077)
PowerServiceCore:handleRequests: timePassed = 1532
SCF: Error InvalidState: Sequence order error (8 received, 4 expected, channel 0) (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 937)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDeviceViCsi.cpp, function waitCompletion(), line 341)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function pause(), line 940)
SCF: Error Timeout: During capture abort, syncpoint wait timeout waiting for current frame to finish (in src/services/capture/CaptureServiceDevice.cpp, function handleCancelSourceRequests(), line 1034)
SCF: Error InvalidState: Timeout waiting on frame start sensor guid 0, capture sequence ID = 8 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1018)
SCF: Error InvalidState: Something went wrong with waiting on frame start (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1077)
(NvCapture) Error Timeout: capture waiting for status timeout (in /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_hw_linux.c, function NvCaptureLinuxGetCaptureStatus(), line 362)
(NvCapture) Error Timeout:  (propagating from /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture.c, function NvCaptureDequeue(), line 1055)
SCF: Error Timeout:  (propagating from src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 898)
SCF: Error InvalidState: Timeout waiting on frame end sensor guid 0, capture sequence ID = 9, channel = 1/1 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 905)
SCF: Error Timeout: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 997)
SCF: Error BadParameter: CC has already been disposed (in src/components/CaptureContainerManager.cpp, function dispose(), line 161)
PowerServiceCore:handleRequests: timePassed = 3014
=== VCCamTest[15212]: WARNING: Cleaning up 1 outstanding streams...=== VCCamTest[15212]: WARNING: Cleaning up 10 outstanding buffers...root@user-desktop:/usr/lib/aarch64-linux-gnu/tegra#
3

hello rbayr,

note, this library from Topic 219059, comment #13 is built from JetPack-4.6/ l4t-r32.6.1;
I did not include any fixes but only debug prints to Topic219059_Jul11_libnvargus.so. hence, there might be some dependency issue if you’re using JetPack-4.6.1/ l4t-r32.7.1 or later release version.

could you please confirm you’re able to repo the same with the latest JetPack release image?

4

I am using the one for l4t-r32.7.2 (Topic 219059, comment #16), with JetPack 4.6.2 . I have accidentally linked the wrong comment.

5

what I meant is there’s only debug print included. do you see the Segfault while reading capture metadata on native JetPack-4.6.2 ?

6

With the native version it dosen’t Segfault. However the acquireBuffer Timeout dosn’t work for some reason.

7

hello rbayr,

how about test with native JetPack-4.6.2,
you may have argus daemon running at foreground and checking the status.
for example,

$ sudo su
# pkill nvargus-daemon
# nvargus-daemon 2>&1 | tee argus-output.log
8

If I run nvargus I get the following messages before the segfault occurs.

root@user-desktop:/home/user# nvargus-daemon 2>&1 | tee argus-output.log
=== NVIDIA Libargus Camera Service (0.98.3)=== Listening for connections...=== VCCamTest[12837]: Connection established (7F9C2051D0)OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module0
OFParserListModules: module list: /proc/device-tree/tegra-camera-platform/modules/module1
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: WARNING: Could not map module to ISP config string
NvPclHwGetModuleList: No module data found
OFParserGetVirtualDevice: NVIDIA Camera virtual enumerator not found in proc device-tree
---- imager: No override file found. ----
(NvCamV4l2) Error ModuleNotPresent: V4L2Device not available (in /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function findDevice(), line 256)
(NvCamV4l2) Error ModuleNotPresent:  (propagating from /dvs/git/dirty/git-master_linux/camera/utils/nvcamv4l2/v4l2_device.cpp, function initialize(), line 60)
(NvOdmDevice) Error ModuleNotPresent:  (propagating from dvs/git/dirty/git-master_linux/camera-partner/imager/src/devices/V4L2SensorViCsi.cpp, function initialize(), line 107)
NvPclDriverInitializeData: Unable to initialize driver v4l2_sensor
NvPclInitializeDrivers: error: Failed to init camera sub module v4l2_sensor
NvPclStartPlatformDrivers: Failed to start module drivers
NvPclStateControllerOpen: Failed ImagerGUID 1. (error 0xA000E)
NvPclOpen: PCL Open Failed. Error: 0xf
SCF: Error BadParameter: Sensor could not be opened. (in src/services/capture/CaptureServiceDeviceSensor.cpp, function getSourceFromGuid(), line 593)
SCF: Error BadParameter:  (propagating from src/services/capture/CaptureService.cpp, function addSourceByGuid(), line 437)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function addSourceByIndex(), line 305)
SCF: Error BadParameter:  (propagating from src/api/CameraDriver.cpp, function getSource(), line 471)
Acquiring SCF Camera device source via index 0 has failed. ---- imager: No override file found. ----
LSC: LSC surface is not based on full res!
=== VCCamTest[12837]: CameraProvider initialized (0x7f94863d40)LSC: LSC surface is not based on full res!
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 265)
PowerServiceCore:handleRequests: timePassed = 1676
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 265)
PowerServiceCore:handleRequests: timePassed = 1009
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 265)
PowerServiceCore:handleRequests: timePassed = 962
PowerServiceCore:handleRequests: timePassed = 963
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 265)
PowerServiceCore:handleRequests: timePassed = 1420
(Argus) Error Timeout:  (propagating from src/api/BufferOutputStreamImpl.cpp, function acquireBuffer(), line 265)
PowerServiceCore:handleRequests: timePassed = 1468
(NvCapture) Error Timeout: capture waiting for status timeout (in /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_hw_linux.c, function NvCaptureLinuxGetCaptureStatus(), line 362)
(NvCapture) Error Timeout:  (propagating from /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture.c, function NvCaptureDequeue(), line 1055)
SCF: Error Timeout:  (propagating from src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 898)
SCF: Error InvalidState: Timeout waiting on frame end sensor guid 0, capture sequence ID = 7, channel = 1/1 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 905)
SCF: Error Timeout: Sending critical error event (in src/api/Session.cpp, function sendErrorEvent(), line 997)
SCF: Error InvalidState: Something went wrong with waiting on frame end (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 991)
SCF: Error InvalidState: Something went wrong with waiting on frame end (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameEnd(), line 991)
=== VCCamTest[12837]: Connection closed (7F9C2051D0)=== VCCamTest[12837]: WARNING: CameraProvider was not destroyed before client connection terminated.=== VCCamTest[12837]:          The client may have abnormally terminated. Destroying CameraProvider...=== VCCamTest[12837]: CameraProvider destroyed (0x7f94863d40)=== VCCamTest[12837]: WARNING: Cleaning up 1 outstanding requests...=== VCCamTest[12837]: WARNING: Cleaning up 1 outstanding stream settings...=== VCCamTest[12837]: WARNING: Cleaning up 1 outstanding sessions...SCF: Error InvalidState: Timeout waiting on frame start sensor guid 0, capture sequence ID = 8 (in src/services/capture/NvCaptureViCsiHw.cpp, function waitCsiFrameStart(), line 1018)
root@user-desktop:/home/user#
9

hello rbayr,

okay, so there’s actually acquireBuffer timeout, and it keeps waits till application side sending critical error event,
it’s the camera stack error handling feature, this is in order to keep system safe for sending Segfault to shutdown the camera app.

10

Yes, with the Topic219059_Jul11_libnvargus.so version of libnvargus I get the Timeouts. However, I also get a segfault directly afterwards (Which doesn’t appear in the native version).
Is there any possibility to prevent the Segfault?

11

hello rbayr,

we have normal camera use-case that did not report Segfault.
so, it depends-on your use-case, IIRC, you’ve configure very long timeout values to camera stack.

12

Yes I use long timeouts because the camera needs to wait for an external trigger signal.

I also have no problems if the (long) timeout dosn’t fire, than i get no Segfaults. However, if the timeout fires one of the next few frames will result in a Segfault when reading the capture mtadata.

13

hello rbayr,

since you’ve timeout failure occurred, it’s the internal error handling mechanism to shutdown the application.

let me try to explain this to you…

it’s camera hardware to output the signal, internal camera stack to allocate the buffers for receiving the data and fill the capture buffers, so, the consumer side, an user-space application for processing these buffers for the usage.

the internal camera stack expect all coming frames to fill capture buffers without failures. unfortunately, not every capture meets this expectation. so, please see-also Topic 170086, there’s Argus Error Resiliency to allow user-space crash, it could handles error events and shutdown the app gracefully. it keeps system in the steady state instead of kernel panic, or any unexpected failures.

14

so it is not possible to user the acquireBuffer Timeout to wait for new frames without blocking the program?

15

As far as I can tell all frames generated by the camera are also processes correctly by the camera stack, even if there is a long delay between them.
(with the exception of three frames directly after the application was started)

16

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

hello rbayr,

please have your use-case with long timeouts. thanks