Electronically Assisted Astronomy with a Jetson Nano

Hello,

i try to improve multiple frames mean denoise filter to get better result on dynamic scene.

As i said, common filter is the mean of 5 frames using a rolling buffer.

This time, i keep the general idea but i add a map to the frame i want to denoise.

This map is the size of the frame and each pixel value is the number of frames i will mean. If there is a transition in the pixel value, then the map will get the value 1 and only 1 frame will be use for the final result. if there is no new transition in the pixel value, then the map value is increased until it reach the maximum value (5 frames to mean).

That is to say each pixel gets its own number of values to be meant depending of the changes in the state of the pixel.

With that kind of treatment, the “ghost effect” is lower.

A small video to show the different kind of denoise results.

UP video from left to right : Raw video with heavy noise, KNN, NLM2
DOWN video from left to right : 5 frames mean, 25 frames mean, independent pixels mean with number of mean map

https://youtu.be/JJTWukOiiyk

For sure, my last method is not perfect but it’s interesting result.

Alain

1 Like

Hello,

still working on denoise filter for very noisy video and THIS TIME, i get something interesting.

I don’t need several frames any more.

The filter works like a car shock absorber. Noise generate very rough changes in the real pixel value (from the camera). The idea is to slow down the changes from the noise but preserve real changes.

Here is an example of what to expect :

You can see :

  • the real signal (pure RAW signal without noise) we expect to get
  • the RAW signal (with noise)
  • the result with 5 frames mean
  • the result with my adaptative absorber denoise filter

I say “adaptative” because it depends of the signal change (small or big) and is controlled with the function you can see on upper right part of the picture.

So, does this filter works ? Yes it works quite well and quite fast.

Here is an example of the different denoise filters i get :

With my new filter (Adaptative Absorber Denoise Filter Paillou) that is to say AADFP, each chanel gets it’s own correction, the denoise is very good and details are preserved.

How fast is it ? On my laptop :

6K video (3000x2000) :
KNN : 38ms
Fast NLM2 : 53ms
5 images Mean : 160ms
AADFP : 99ms

1600x1200 video :
KNN : 15ms
Fast NLM2 : 21ms
5 images Mean : 37ms
AADFP : 31ms

I think it is quite fast.

I think the result is really good and it is really interesting to be able to manage filter response with the transfer function. The reactivity of the filter can be controlled with this function and you can change the dynamic of the filter response as you want.

And now, an example with a video :

I am really happy with this result. Champagne !

Alain

2 Likes

Hello,

as you can this in the previous video, the main problem with AADFP filter is the “ghost” effect when a pixel change his state (high value to low value).

As i said, this filter is manage with a transfer function which can be modified to get a different result.

I think i have solved the “ghost effect” issue by changing the transfer function.

Now, i have a high dynamic AADFP in opposition with my previous “low dynamic” filter.

Here is a video to illustrate the changes :

Finally, i get a very high efficiency denoise routine to clean very noisy video !!!

I have just tested my denoise routine on the Jetson Nano and it works fine. The benchmarks with the Nano :

3000x2000 px video :
AADFP : 150ms
KNN : 220ms
NLM2 : 627ms

1600x1200 px video :
AADFP : 48ms
KNN : 67ms
NLM2 : 190ms

For memory, on my laptop (i7 8750H + GTX1060) :
6K video (3000x2000) :
KNN : 38ms
Fast NLM2 : 53ms
AADFP : 99ms (Nano : 150ms)

1600x1200 video :
KNN : 15ms
Fast NLM2 : 21ms
AADFP : 31ms (Nano : 48ms)

This is really impressive and Nano can manage my denoise filter without problem. If can denoise a 1600x1200 video at 20 fps (and my software uses Python and PyCuda, far from being the fastest tools) ! Better than KNN which is really fast (but lowest results for my use).

I am really surprised with the Nano results. They are near my laptop results. Quite strange.

Alain

An example of denoising and light pollution removing on the Perseus double cluster with Jetson Nano :

Video treatment brings real improvements.

Alain

1 Like

Hello,

i made new benchmarks with my AADFP filter and my laptop because my previous results were a bit strange.

So, after debug session, here are the results :

my laptop (i7 8750H + GTX1060) :
6K video (3000x2000) :
KNN : 20ms
Fast NLM2 : 35ms
AADFP : 48ms (Nano : 150ms)

1600x1200 video :
KNN : 8ms
Fast NLM2 : 16ms
AADFP : 17ms (Nano : 48ms)

I think this time, benchmarks are correct.

Alain

A small video to illustrate the real time filtering of a RAW colour capture of the Moon with SkyNano.

First, you will see the RAW capture, then a simple colour saturation enhancement and finally, a much better way to reveal colours.

The meaning of this story : you can get much more results with the good method.

Alain

1 Like

Hello,

the Jetson Xavier NX has arrived ! Many thanks Nvidia and many thanks Dustin !

A picture of the Xavier NX just in front the Nano :

And the first light of the Xavier NX :

Jetson Xavier NX is a very new and young SBC but software seems to be very good. I have installed all the libraries i need for my software and everything seems to be ok.

So, SkyNano is now JetsonSky and the software works perfectly with the Xavier NX. No issue, no problem.

I made a quick benchmark with the Xavier NX and my denoise filters.

3000x2000 pixels video acquisition :

KNN : Nano 220ms VS Xavier NX : 75ms
NLM2 : Nano 627ms VS Xavier NX : 200ms
AADFP : Nano 150ms VS Xavier NX : 90ms

1600x1200 pixels video acquisition :

KNN : Nano 67ms VS Xavier NX : 30ms
NLM2 : Nano 190ms VS Xavier NX : 62ms
AADFP : Nano 48ms VS Xavier NX : 25ms

For memory, on my laptop (i7 8750H + GTX1060) :
6K video (3000x2000) :
KNN : 38ms (Xavier NX 75ms)
Fast NLM2 : 53ms (Xavier NX 200ms)
AADFP : 99ms (Xavier NX 90ms)

1600x1200 video :
KNN : 15ms (Xavier NX 30ms)
Fast NLM2 : 21ms (Xavier NX 62ms)
AADFP : 31ms (Xavier NX 25ms)

I must say Jetson Xavier NX outperform Jetson Nano (not a big surprise but it’s really cool).

My first contact with Jetson Xavier NX is really great.

Everything is perfect. Champagne.

Alain

3 Likes

Hi Alain, glad to see that you got the NX up and running already!