Can't open CSI IMX219 camera after wake up from suspend

My NX board information is below:
Linux txkj-nx 4.9.253-tegra

  • NVIDIA Jetson Xavier NX (Developer Kit Version)
    • Jetpack 4.6 [L4T 32.6.1]
    • NV Power Mode: MODE_15W_6CORE - Type: 2
    • jetson_stats.service: active
  • Board info:
    • Type: Xavier NX (Developer Kit Version)
    • SOC Family: tegra194 - ID:25
    • Module: P3668 - Board: P3509-000
    • Code Name: jakku
    • CUDA GPU architecture (ARCH_BIN): 7.2
    • Serial Number: 1424920027702
  • Libraries:
    • CUDA: 10.2.300
    • cuDNN: 8.2.1.32
    • TensorRT: 8.0.1.6
    • Visionworks: 1.6.0.501
    • OpenCV: 4.5.1 compiled CUDA: YES
    • VPI: ii libnvvpi1 1.1.12 arm64 NVIDIA Vision Programming Interface library
    • Vulkan: 1.2.70

I have CSI camera IMX219 on my NX board. In my program, I use gstreamer pipeline in opencv to catprue camera. My issue is when I suspend the system, then wake up from suspend, the camera does not work.
My gstreamer pipeline string is "“nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1640, height=(int)1232, format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, width=(int)800, height=(int)600, format=(string)BGRx ! videoconvert ! appsink”.
Before suspend, I call cap.release() to release the opencv capture.
The error log is below:
Pipeline is live and does not need PREROLL …
Setting pipeline to PLAYING …
New clock: GstSystemClock
Error generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:725 No cameras available
Got EOS from element “pipeline0”.
Execution ended after 0:00:00.004495419
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
(Argus) Error EndOfFile: Unexpected error in reading socket (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 266)
(Argus) Error EndOfFile: Receive worker failure, notifying 1 waiting threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 340)
(Argus) Error InvalidState: Argus client is exiting with 1 outstanding client threads (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadCore(), line 357)
(Argus) Error EndOfFile: Receiving thread terminated with error (in src/rpc/socket/client/ClientSocketManager.cpp, function recvThreadWrapper(), line 368)
(Argus) Error EndOfFile: Client thread received an error from socket (in src/rpc/socket/client/ClientSocketManager.cpp, function send(), line 145)
(Argus) Error EndOfFile: (propagating from src/rpc/socket/client/SocketClientDispatch.cpp, function dispatch(), line 91)

BTW, can I open IMX219 camera with v4l2src? I failed to use v4l2src for IMX219, I also found some topics said jetson can’t open RG10 format with v4l2src.

hello harry_xiaye,

it looks camera is not available after you wake-up from suspend.
may I know how you setup environment for testing this? what’s your procedures.

BTW,
you cannot open IMX219 camera with v4l2src, it’s not supported.

@JerryChang, I have a simple program to reproduce this issue.
Below is the source code of the program. You can build it and run on your NX board.
This program is just creating VideoCapture and open with CAP_GSTREAMER string, then do nothing and release. 20 seconds count is to give you time to “sudo systemctl suspend” system.

The fist test step is:

  1. run nvgstcapture to check CSI camera is working, then quit nvgstcapture
  2. run the simple program, during 20 seconds count, you can “sudo systemctl suspend” in another terminal
  3. then wakeup system by moving and clicking mouse before the 20 seconds count
  4. wait for the program exit, then run nvgstcapture again, you should get error this time.

The 2nd test step:

  1. run nvgstcapture to check CSI camera is working, then quit nvgstcapture
  2. run the simple program, DONT suspend the system, just wait for the program exit after 20 seconds.
  3. run nvgstcapture again, you should see it working.

test.cpp
#include
#include “unistd.h”
#include “opencv2/opencv.hpp”
#include “opencv2/videoio.hpp”

using namespace cv;

int main()
{
std::string capStr = “nvarguscamerasrc
! video/x-raw(memory:NVMM), width=(int)1640, height=(int)1232, format=(string)NV12
! nvvidconv
! video/x-raw, width=(int)800, height=(int)600, format=(string)BGRx
! videoconvert ! appsink”;

VideoCapture * cap = new VideoCapture();
if (cap == nullptr) {
std::cout << “fail to new VideoCapture” << std::endl;
return 0;
}

cap->open(capStr, CAP_GSTREAMER);
if (cap->isOpened()) {
cap->release();
}

delete cap;

unsigned int count = 0;
while(true) {
count++;
std::cout << "count is " << count << std::endl;
if (count > 20) {
break;
}
usleep(1000 * 1000);
}

return 0;
}

hello harry_xiaye,

could you plese try to restart Argus daemon service when you wake-up from suspend.
for example,
$ sudo pkill nvargus-daemon
$ sudo systemctl start nvargus-daemon

@JerryChang , I just tried, but it can’t work.

hello harry_xiaye,

I don’t think this use-case is tested before. let me arrange resources to check this locally.
BTW, may I know what’s the actual use-case to enter suspends when camera is on preview streaming?

Our device has a camera and display camera video on a screen. When the device is powered on, it will always display camera video on screen.
The user can press the power key quickly to make the device go into sleep mode to save power like mobile phone. So normally the device screen always display camera video, and the user can press the power key anytime to put the device into sleep mode.

I ran the test program and got nvargus-daemon log. See the attached file.

below log is after wake up from suspend. If dont do suspend, we have no below logs.

11月 02 17:00:01 txkj-nx nvargus-daemon[6818]: === test_open[9920]: CameraProvider destroyed (0x7fa4c4a5c0)(NvCapture) Error IoctlFailed: isp capture reset ioctl failed: 5 (in /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_isp_hw_linux.c, function NvCaptureIspLinuxChannelReset(), line 239)
11月 02 17:00:01 txkj-nx nvargus-daemon[6818]: (NvCapture) Error IoctlFailed: (propagating from /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_isp.c, function NvCaptureIspChannelReset(), line 1236)
11月 02 17:00:01 txkj-nx nvargus-daemon[6818]: SCF: Error IoctlFailed: (propagating from src/services/capture/NvCaptureIspHw.cpp, function closeIspHw(), line 408)
11月 02 17:00:01 txkj-nx nvargus-daemon[6818]: (NvCapture) Error IoctlFailed: isp capture release ioctl failed: 5 (in /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_isp_hw_linux.c, function NvCaptureIspLinuxChannelClose(), line 217)
11月 02 17:00:01 txkj-nx nvargus-daemon[6818]: (NvCapture) Error IoctlFailed: (propagating from /dvs/git/dirty/git-master_linux/camera/capture/nvcapture/capture_isp.c, function NvCaptureIspChannelClose(), line 1148)
11月 02 17:00:01 txkj-nx nvargus-daemon[6818]: SCF: Error IoctlFailed: (propagating from src/services/capture/NvCaptureIspHw.cpp, function closeIspHw(), line 409)

The log.
nvargus_suspend.log (5.7 KB)

hello harry_xiaye,

to clarify, would you like to restore the camera preview frames and render it to display monitor as soon as the target resume from deep sleep? this is not supported.

FYI,
lots of services has terminated, since it need as less as possible power consumption for entering deep sleep. those logs look expected, it’s user-space to shutdown the camera service, without notify to low-level driver side to close the stream to trigger IOCTL failed.

note.
you may moving to the latest releases, there’re changes check-in which add error handling for the camera stream,
however, you still need handle camera stream to shutdown before entering deep sleep, and start camera stream manually after it wake-up.

@JerryChang , I added my comments and questions after yours as below.

to clarify, would you like to restore the camera preview frames and render it to display monitor as soon as the target resume from deep sleep? this is not supported.
Harry: You are asking me to try to restore the camera and render it to display as soon as resuming, or you are saying this is not supported?

however, you still need handle camera stream to shutdown before entering deep sleep, and start camera stream manually after it wake-up.
Harry: We did try to release or close camera stream to shutdown before entering deep sleep. You can check the source code of our simple program. We release the VideoCapture and delete the object. We did suspend after that, but we still have the issue.

Hi harry_xiaye,

Put device into suspend when camera is on preview streaming, it’s not real use-case.
Please try below steps:

  1. Run gstreamer pipeline to preview streaming
  2. Terminate gst pipeline
  3. Put device into suspend
  4. Resume from suspend
  5. Run gst pipeline.

@carolyuu ,

We did terminate gst pipeline, then put the device into suspend, still see the issue.

Hi harry_xiaye,

Please reboot your device and try again.
Stop gst pipeline before suspend device, after resume, run gst pipeline again, we confirmed it’s working.

@carolyuu ,

We rebooted the device and tested again (stop gst pipeline before suspend, after resume, run gst pipeline again), it did not work.

hello harry_xiaye,

did you test with your customize program?
please try with simple gst pipeline for confirmation.

  1. Enable gst pipeline.
    $ gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvoverlaysink -ev
  2. ctrl^c to shutdown the preview stream.
  3. Put device into suspend
  4. Resume from suspend.
  5. Go to step-1 to check preview stream.

@JerryChang ,

With your above test steps, we have no issue.

Please note, if test my simple program, let my simple program exit without suspend, we have no this issue too.

The issue happens if we suspend before the simple program exits( but after release gst pipeline ).

I think the point is: if the program does not exit, even it release gst pipeline, if suspend at this time, the camera won’t work after wakeup anymore.

hello harry_xiaye,

why don’t you shutdown the app completely before entering suspend.

Our program starts when power up and display our UI to user always. The user won’t see ubuntu desk UI, and wo dont want the user to see any other UI than ours.
So we can’t exit our program.