Sekonix GMSL camera poll at lower frequency

Autonomous Vehicles DRIVE AGX Xavier DRIVE AGX Xavier General
1

Please provide the following info (check/uncheck the boxes after clicking “+ Create Topic”):
Software Version
DRIVE OS Linux 5.2.0
DRIVE OS Linux 5.2.0 and DriveWorks 3.5
NVIDIA DRIVE™ Software 10.0 (Linux)
NVIDIA DRIVE™ Software 9.0 (Linux)
other DRIVE OS version
other

Target Operating System
Linux
QNX
other

Hardware Platform
NVIDIA DRIVE™ AGX Xavier DevKit (E3550)
NVIDIA DRIVE™ AGX Pegasus DevKit (E3550)
other

SDK Manager Version
1.5.0.7774
other

Host Machine Version
native Ubuntu 18.04
other

Hello, I would like to get images from my Sekonix 3324 camera at rates lower than 30Hz. (E.g. 10 or 20 Hz). I get my images like so :

  status_ = dwSensorCamera_readFrameNew(&cameraFrameHandle_, 0, sensorHandle_);
  if (status_ == DW_NOT_READY)
  {
    std::this_thread::sleep_for(std::chrono::milliseconds(5));
    return false;
  }

  if (status_ == DW_TIME_OUT)
  {
    ROS_WARN_STREAM("DROPPED FRAME FOR : " << frame_.str());
    return false;
  }

  if (status_ != DW_SUCCESS)
  {
    ROS_FATAL_STREAM("FAILED TO READ FRAME FOR : " << frame_.str());
    CHECK_DW_ERROR_ROS(status_);
    return false;
  }

  // Get image from cameraFrameHandle
  status_ = dwSensorCamera_getImage(&imageHandleOriginal_, DW_CAMERA_OUTPUT_NATIVE_PROCESSED, cameraFrameHandle_);
  if (status_ != DW_SUCCESS)
  {
    ROS_FATAL_STREAM("dwSensorCamera_getImage() Failed");
    return false;
  }

But I get those errors when I run the above code at 20 Hz :

NOTIF_WARN_ICP_FRAME_DROP

Which doesn’t seem normal. I shouldn’t get those errors.

Another thing. The function dwSensorCamera_readFrameNew will get a frame from the camera frame buffer. (Please correct me if I’m wrong.)
When polling at less than 30Hz this buffer will contain a list of images and dwSensorCamera_readFrameNew will get the oldest from this buffer. So dwSensorCamera_readFrameNew will always read old frame when the polling rate is below 30 hz. (Please correct me if I’m wrong.)

How do you solve this problem ? Right now I call dwSensorCamera_readFrameNew until I get DW_TIME_OUT so I know I got the last frame.

What’s the good practice ?

Thanks !

3

Dear @maxandre.ogeret ,
May I know how you are accessing 20HZ rate in sample snippet? Also, do you see any issue if you increase timeout value in dwSensorCamera_readFrameNew?

4

@SivaRamaKrishnaNV Thanks for your answer.

I use ROS rate functionnality which loops at a given frequency relatively accurately :

ros::Rate rate(20.0);

while (ros::ok())
{
  driver->poll_and_process()
  ros::spinOnce();
  rate.sleep();
}

Adding a timeout

  • Polling at 20Hz, timeout = 0us. Average delay : 0.204s
  • Polling at 20Hz, timeout = 100us. Average delay : 0.200s
  • Polling at 20Hz, timeout = 100000us. Average delay : 0.193s
  • Polling at 20Hz, timeout = 100000000us. Average delay : 0.193s
  • Polling at 20Hz, timeout = 100000000000us. Average delay : 0.193s

The effect of adding a timeout is negligible.

Get last frame from buffer

Here I poll the camera until the buffer is empty :

bool Camera::get_last_frame() {
  dwStatus status_old = DW_NOT_READY;

  while (true) {
    status_ = dwSensorCamera_readFrameNew(&cameraFrameHandle_, 0, sensorHandle_);

    if (status_ == DW_NOT_READY) {
      std::this_thread::sleep_for(std::chrono::milliseconds(5));
      return false;
    }

    if (status_ == DW_TIME_OUT && status_old == DW_SUCCESS) {
      return true;
    }

    status_old = status_;
  }
}
  • Polling frequency : 20hz, timeout 0us. Average delay : 0.046s
  • Polling frequency : 20hz, timeout 100000us. Average delay : 0.114s

Using this snippet makes a huge difference but it feels pretty hacky. What would be the best way to make sure we get the most recent frame from the buffer ?

5

@SivaRamaKrishnaNV Any more information on this?

6

Dear @maxandre.ogeret,
I am checking on your queries internally. Do you see any issue if you use camera with 30hz?

7

@SivaRamaKrishnaNV When running at 30hz everything is fine.

8

Dear @maxandre.ogeret,
The captured frames will be stored in a FIFO queue. So accessing the most recent frame(end of FIFO queue) is not possible with out reading earlier frames. As you said you are reading all frames until you see DW_TIME_OUT. That is the right way to read frames.
You may change the FIFO size with fifo-size parameter. NOTIF_WARN_ICP_FRAME_DROP is warning and not error. If you ignore if it is not effecting your application use case.