# ISP Function evaluate

**URL:** <https://forums.developer.nvidia.com/t/isp-function-evaluate/49711>\
**Category:** Jetson TX1\
**Created:** [May 4, 2017, 2:11am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711 "2017-05-04T02:11:26Z")\
**Posts on this page:** 19\
**Page:** 1

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [May 4, 2017, 2:11am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/1 "2017-05-04T02:11:26Z")

</div>

## I would like to use TX1’s ISP funtion, to make a camera product . And I did not use the 3A(AF/AE/AWB) fuction inside, but use my own 3A algorithm. So, I need to evaluate 2 points: First, if the ISP fuction meet the my requirements; Second,if the 3A statistics match my own 3A Alg. I would like to find the detailed datasheet about ISP , But only found very simple like these: -Bayer domain hardware noise reduction -Per-channel black-level compensation -High-order lens-shading compensation -3x3 color transform -Bad pixel correction -Programmable coefficients for de-mosaic with color artifact reduction Color Artifact Reduction: a two-level (horizontal and vertical) low-pass filtering scheme that is used to reduce/remove any color artifacts that may result from Bayer signal processing and the effects of sampling an image. -Enhanced down scaling quality -Edge Enhancement Attachments

It is too simple , And My question is:  
1)if “noise reduction” including temporal niose reduction(TNR) and spatial noise reduction?  
2)if "Bad pixel corrction " including static bad pixel correction and dynamic correction?(and can correct how many pixels most with static bad pixel correction?)  
3)if the TX1 support D-zoom fution ,and max d-zoom rate is ?  
4)about 3A statistics :  
A) AE ---- if the histograms is the all picture’s or only some select zones?  
B) AWE---- if the region averages exclude any pixel ,such as some big red pixel which not gray pixel?  
C) AF----- what kind of filter ? FIR or IIR ? and how many filter? how many regions provide (12\*12,or more?)?

1. how many frames delay in the TX1 ISP process? such as “mipi sensor—\>VI(1frame)—\>ISP(1frame)—\>VPSS(1frame)—\>VO(1frame), 4frame delay for local HDMI Display”.
2. can you provide more datasheet about the isp fuction detaild introduce, and the softwarwe control interface?

thank you.

---

<div class="post-metadata">

**Author:** ![ShaneCCC](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/shaneccc/32/14043_2.png) [@ShaneCCC](https://forums.developer.nvidia.com/u/ShaneCCC)\
**Post date:** [May 4, 2017, 3:08am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/2 "2017-05-04T03:08:08Z")

</div>

Hi lihaimin  
We are collecting the information for you.

---

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [May 4, 2017, 3:15am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/3 "2017-05-04T03:15:57Z")

</div>

Hi, ShaneCCC  
Thanks. Waiting for your replay.

---

<div class="post-metadata">

**Author:** ![mingyigoodluck](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mingyigoodluck](https://forums.developer.nvidia.com/u/mingyigoodluck)\
**Post date:** [May 9, 2017, 3:20am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/4 "2017-05-09T03:20:40Z")

</div>

I have the same enquiries too.

RASAP

Thanks.

---

<div class="post-metadata">

**Author:** ![Yugui](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@Yugui](https://forums.developer.nvidia.com/u/Yugui)\
**Post date:** [May 9, 2017, 3:43am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/5 "2017-05-09T03:43:51Z")

</div>

Finally there is a same question, also waiting …

---

<div class="post-metadata">

**Author:** ![ShaneCCC](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/shaneccc/32/14043_2.png) [@ShaneCCC](https://forums.developer.nvidia.com/u/ShaneCCC)\
**Post date:** [May 10, 2017, 3:34am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/6 "2017-05-10T03:34:03Z")

</div>

Hi All  
There’s some data that public below.  
And the there’s a doc located the tegra\_multimedia\_api/argus/docs/Argus.0.96.pdf, and some useful sample code for tegra\_multimedia\_api/argus/samples/bayerAverageMap cudaHistogram …

```auto
Sliders:
http://on-demand.gputechconf.com/gtc/2016/webinar/getting-started-jetpack-camera-api.pdf
Video:
https://www.youtube.com/watch?v=m4ZpB_Ip2qg

```

---

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [May 12, 2017, 2:30am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/7 "2017-05-12T02:30:36Z")

</div>

Hi, ShaneCCC

```
Thanks for your reply.
I had read the document and video you provided. And I found that, this is all most about the app level, which describing how to open a camera device, and get video stream. But what i need is in the driver level(frameworks level, kernal level), which i can config the isp parameter such as RGB2RGB Matrix,Gamma table,demosaicing, noise reduce .... these fuction is provided by TX1's hardware isp pipeline, and not provided by GPU. 
the page 11 of "getting-started-jetpack-camera-api.pdf" document showing the ISP subsysterm function,this is help me to understand the isp pipeline. but i need more information about the isp pipeline, such as the question above:
 1)if "noise reduction" including temporal niose reduction(TNR) and spatial noise reduction?
 2)if "Bad pixel corrction " including static bad pixel correction and dynamic correction?(and can 
    correct how many pixels most with static bad pixel correction?)

 3)if the TX1 support D-zoom fution ,and max d-zoom rate is ?
 4)about 3A statistics : 
     A) AE ---- if the histograms is the all picture's or only some select zones?
     B) AWE---- if the region averages exclude any pixel ,such as some big red pixel which not 
                gray pixel? 
     C) AF----- what kind of filter ? FIR or IIR ? and how many filter? how many regions provide 
                (12*12,or more?)? 
 5) how many frames delay in the TX1 ISP process? such as "mipi sensor--->VI(1frame)--->ISP
    (1frame)--->VPSS(1frame)--->VO(1frame), 4frame delay for local HDMI Display". 

```

can you show me more details about the isp pipeline & isp turning tool, and the source interfaces in frameworks/kernal to config the isp paremeter.

thank you.

---

<div class="post-metadata">

**Author:** ![nklc2009](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@nklc2009](https://forums.developer.nvidia.com/u/nklc2009)\
**Post date:** [May 16, 2017, 6:55am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/8 "2017-05-16T06:55:12Z")

</div>

Hi, ShaneCCC  
I have the same quetions as above,also need yout help,thank you!

---

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [May 18, 2017, 7:21am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/9 "2017-05-18T07:21:05Z")

</div>

any nvidia’s engineer else ? who knowns ISP function…

---

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [May 18, 2017, 10:47am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/10 "2017-05-18T10:47:31Z")

</div>

I try to find the isp parameter setting(such as gamma,rgb2rgb) in the kernal driver, but found nothing. who can show me where is the code ?

---

<div class="post-metadata">

**Author:** ![watsoncao](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@watsoncao](https://forums.developer.nvidia.com/u/watsoncao)\
**Post date:** [May 20, 2017, 2:15am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/11 "2017-05-20T02:15:19Z")

</div>

Hi, ShaneCCC  
I have the same quetions as above,also need yout help,thank you!

---

<div class="post-metadata">

**Author:** ![LiYuan](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@LiYuan](https://forums.developer.nvidia.com/u/LiYuan)\
**Post date:** [May 20, 2017, 2:22am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/12 "2017-05-20T02:22:50Z")

</div>

These questions are very important, I want to use TX1/TX2 too. Tks.

---

<div class="post-metadata">

**Author:** ![ShaneCCC](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/shaneccc/32/14043_2.png) [@ShaneCCC](https://forums.developer.nvidia.com/u/ShaneCCC)\
**Post date:** [May 24, 2017, 9:36am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/13 "2017-05-24T09:36:56Z")

</div>

Hi All

Update some information here

No static bad pixel correction, only dynamic bad pixel correction.  
digital zoom is supported. It’s through VIC, so we’d need to look up those limits. Think it can be pretty large.  
Histogram is full image  
Region averages exclude clipped pixels. The API docu have more about Bayer Average Map.  
We expose a 64x64 sharpness map using an FIR filter.

---

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [May 26, 2017, 2:03am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/14 "2017-05-26T02:03:08Z")

</div>

Hi ShaneCCC

Thanks for your kindly reply . it really helps me . could i ask for some more detail?

1)Histogram is full image ----- in the MM API , user can get x-axis counter by “histogram.size()” interface. but in my opinion, the x-axis counter shuld is a const, such as 256(0~255) for 8bit or 1024(0~1023)for 10bit. why to provide this “histogram.size()” interface? cause it need to return 256 for 8bit mipi input,and need to return 1024 for 10bit mipi input?

2)We expose a 64x64 sharpness map using an FIR filter-----Gerenally, it should provide two FIR filter,one is high-pass filter, the other is low-pass filter. cause when using the optical zoom lens ,the focus range may be very long, it can focus at the 0.3M ~ 20M. when the image is very fuzzy, the high-pass filter’s value is mostly equal to zero, at this time we need to use the low-pass filter to decide the lens motor moving deriction. So, I worried about that the TX-1 Only provide a sharpess map using only a FIR filter, how to adapt the optical zoom lens’ AF?

---

<div class="post-metadata">

**Author:** ![lihaimin](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@lihaimin](https://forums.developer.nvidia.com/u/lihaimin)\
**Post date:** [June 1, 2017, 6:19am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/15 "2017-06-01T06:19:47Z")

</div>

Hi ShaneCCC

Thanks for your kindly reply . it really helps me . could i ask for some more detail?

1)Histogram is full image ----- in the MM API , user can get x-axis counter by “histogram.size()” interface. but in my opinion, the x-axis counter shuld is a const, such as 256(0~255) for 8bit or 1024(0~1023)for 10bit. why to provide this “histogram.size()” interface? cause it need to return 256 for 8bit mipi input,and need to return 1024 for 10bit mipi input?

2)We expose a 64x64 sharpness map using an FIR filter-----Gerenally, it should provide two FIR filter,one is high-pass filter, the other is low-pass filter. cause when using the optical zoom lens ,the focus range may be very long, it can focus at the 0.3M ~ 20M. when the image is very fuzzy, the high-pass filter’s value is mostly equal to zero, at this time we need to use the low-pass filter to decide the lens motor moving deriction. So, I worried about that the TX-1 Only provide a sharpess map using only a FIR filter, how to adapt the optical zoom lens’ AF?  
Attachments

---

<div class="post-metadata">

**Author:** ![ShaneCCC](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/shaneccc/32/14043_2.png) [@ShaneCCC](https://forums.developer.nvidia.com/u/ShaneCCC)\
**Post date:** [June 13, 2017, 3:35am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/16 "2017-06-13T03:35:34Z")

</div>

Hi lihaimin

1. The current version of libArgus has not been validated for autofocus systems. There are some preliminary API controls that we expect will support user-layer autofocus logic, but this has not been a lower priority area for our development and validation efforts. We are familiar with the limitations of our sharpness map when the lens is significantly out of ofocus. One way that a lower frequency sharpness score can be obtained for use along with the high frequency sharpness map is to run a small convolution over the region of interest on the bayer sharpness map. The amount of data is relatively small and so this can be done with CPU. Another possibility is to request a downscaled output from the ISP and use CUDA to compute a custom sharpness metric.

2. libArgus is designed so that it could be used on a variety of hardware. Different ISPs might choose to use a different number of bins for their histograms to trade off hardware cost with precision. Allowing the application to query the size means that we won’t force unnecessary remapping operations that could consume CPU cycles and would prevent the application from seeing the raw data generated by the ISP.

And for the image quality tuning we suggest to have the scaling partner to help.

---

<div class="post-metadata">

**Author:** ![vikasbijalwan4](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@vikasbijalwan4](https://forums.developer.nvidia.com/u/vikasbijalwan4)\
**Post date:** [March 18, 2020, 9:20am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/17 "2020-03-18T09:20:09Z")

</div>

@ShaneCCC In the ISP pipeline can we pass static images rather than camera input. If yes is there a LibArgus api for that? Referred to :

> **[✅ Jetson Image Processing: ISP Libargus and Fastvideo SDK](https://www.fastcompression.com/blog/jetson-image-processing.htm)**
>
> ✅ Jetson Image Processing: ISP libargus and Fastvideo SDK to create camera applications on mobile GPU. ✅ Jetson ISP vs CUDA SDK from Fastvideo.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![TomNVIDIA](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/tomnvidia/32/14181_2.png) [@TomNVIDIA](https://forums.developer.nvidia.com/u/TomNVIDIA)\
**Post date:** [October 18, 2021, 6:57pm UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/18 "2021-10-18T18:57:03Z")

</div>

Closed on October 18, 2021, 6:57pm UTC



---

<div class="post-metadata">

**Author:** ![ShaneCCC](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/shaneccc/32/14043_2.png) [@ShaneCCC](https://forums.developer.nvidia.com/u/ShaneCCC)\
**Post date:** [August 5, 2022, 2:48am UTC](https://forums.developer.nvidia.com/t/isp-function-evaluate/49711/19 "2022-08-05T02:48:46Z")

</div>

Sorry, to tell current don’t support this kind of use case.

Thanks
