Stats from Camera

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi Folks,

I am sure there are different image-statistics parameter generated by Tx1 hardware (ISP). Is there any example which shows how to ‘obtain’ these parameters (e.g. histogram) from ISP , and not from running code on GPU and/or CPU ?

Thanks,

2

hello dumbogeorge,

may i have more details about your use-case?
you could also refer to Argus camera for histogram example.
thanks

3

Hi Jerry,

For public description - our use case involves - reading videos from two cameras (concurrently) and doing computer vision processing on every frame of video, from each channel. Our vision processing is divided between CPU and GPU. First up we process frames on GPU and resulting images/frame are then further processed on CPU. While CV processing in going on we also encode input video from each camera as two individual .mp4 bitstreams.

Our CV algorithms needs -

  1. histogram, (and we would not like to spend CPU/GPU cycles , and extra DDR SW to find this histogram
  2. knowledge of whether successive video frames are “indentical” (if ISP give any parameters about ‘activity’ of video frames - we would like to use it to conclude - whether current frame is ‘somewhat similar’ to its preceding frame).

Hopefully, this gives an idea of where my question is coming from.

I looked at histogram example (i.e. cudaHistogram / histogram.cu) and that does not quite fit the purpose. We are looking to make use of values which pre-calculated by ISP, such that we do not have to pay for any -

a) GPU/CPU cycles (they are already loaded with other workload)
b) do not want to pull frame from memory (DDR) for purpose of generating some of these parameters - as that loads up memory controller, and increases latency of memory transactions for CPU/GPU/Encoder.

My understanding is that ISP is already calculating few image statistics while is it doing its “regular” job. I would like to see if we can make use of those parameters.

Thanks,

4

hello dumbogeorge,

please refer to this argus example,

argus/public/samples/histogram

there’s reading the ISP static result to the ‘histogram’ structure already.

// Read the Bayer histogram.
std::vector< BayerTuple<uint32_t> > histogram;
const IBayerHistogram* bayerHistogram =
    interface_cast<const IBayerHistogram>(iMetadata->getBayerHistogram());
if (!bayerHistogram || bayerHistogram->getHistogram(&histogram) != STATUS_OK)
5

Hi JerryChang,

Thanks for your pointer. I guess I missed spotting it.

It seems this example is using histogram generated by ISP. Are there other parameters that ISP can generate ? I am looking to find some parameters and/or stats that can help in identifying whether two successive images are identical. Additionally, I would like to see if we can find some parameter from ISP which can help in image segmentation.

Thanks

6

hello dumbogeorge,

there are several algorithms to perform image segmentation.
i would suggest you looking for proper algorithm instead of starting from ISP statistics.
may i have more details about what’s the scenario/ use-case.
is this an approach for single image or video streaming?
thanks

7

Hi Jerry - #3 gives details of use case. This will be for video streaming. Would it be possible to just list out what is given out by ISP (other than histogram ?) May be it can help me identify identical frames .

Thanks

8

hello dumbogeorge,

please check [url]https://developer.nvidia.com/embedded/twodaystoademo[/url]
there’s image segmentation example for your reference.
thanks