Control camera, capture frame, process frame pipeline - approach ?

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hello,

I have been working with the Jetson TX1 platform for several months now for a large project. We opted to use Nvidia Jetson technology as our product needs to capture and process frames at high speed using GPU.

At first glance Jetson seemed ideal but after working with it I am officially confused.

Let me elaborate.

Our main process will comprise of the following steps :

  1. Connect to 3 CSI cameras at 4K resolution
  2. Capture frames
  3. Run auto-exposure and color correction algorithm
  4. Update the camera controls accordingly
  5. Capture frames
  6. Run analysis algorithms
  7. Store the results
  8. Return to step 5 until shutdown. Return to step 2 every 1 minute.

Nvidia propses the Argus library to alter camera controls and such an option is not available in gstreamer or VisionWorks. Our processing is implemented in OpenCV and is currrently being implemented in VisionWorks. However there is no interface between Argus and OpenCV/VisionWorks. Currently we store and read the image locally…

How do you propose I proceed ? Is eventually the Jetson platform the right choice ? Is there a plan to support Argus + VisionWorks/OpenCV or camera control in VisionWorks/Gstreamer and if so when is it expected ?

Thank you in advance for your reply.

2

Hi pavlos.stavrou,

Would you please elaborate more about your requirement for what FPS you want to run at?
That could let us to get more understanding and have suggestions.

Thanks

3

Hello,

At the moment we are targeting for the entire process to be run at 1 FPS and it is to be increased to 25-30 FPS for the final stage of the product.

My main issue is that I cannot find a way to set the camera cotrnols, capture frames and process them with VisionWorks using a coherent pipeline. Ideally I would also like to get raw frame data but going through the forums I understand that this is only possible using V4L.

4

Hi,

We didn’t enable Arugs + VisionWorks and no future plan currently.

What kind of algorithm do you use?
Could you check if our NPP library cover your use case?
https://developer.nvidia.com/npp

If yes, you can find some sample to illustrate Argus → CUDA:
[EGLStream]
tegra_multimedia_api\argus\samples\histogram
tegra_multimedia_api\argus\samples\openglBox
[CUeglStream]
tegra_multimedia_api\argus\samples\cudaHistogram

Thanks.