Electronically Assisted Astronomy with a Jetson Nano

Hello,

i have uploaded in GithHub a new version of JetsonSky (V15_03) :

Some bugs have been fixed and V15_02 has been removed.

I made some tests with Cupy which is a GPU version of Numpy. It looks promising but unfortunately, there is a problem with Pycuda when using Cupy (Cuda context for Pycuda gets issue when using Cupy).

Those libraries are a nightmare. Pycuda is really fast so i must keep it. For some treatment routines, the only solution i get is to write them using Pycuda to avoid numpy and OpenCV.

For Jetson cards, i hope i will be able to use VPI but for now, now way to get VPI result in numpy array. I asked for a solution in NVidia dev forum but no response for now.

Alain

2 Likes

Hello,

i have uploaded in GithHub a new version of JetsonSky (V15_04) :

Some bugs have been fixed and bring small optimizations. V15_03 has been removed.

Clear sky

Alain

Great as ever!
I have a little question. The dark-subtraction only belongs to images not video. Is this correct ?

Hello Thom,

Dark subtraction is supposed to work with both images and videos.

This afternoon, i have successfully migrated Xavier NX to JetPack 5.0.2.

I have installed all libraries for JetsonSky and it works fine.

I guess new Jetson Orin Nano will be able to support JetsonSky.

Alain

1 Like

Wow, then this must be a (huge) improvement for the video-quality. I didn’t have tried this out so far, but I will do next

1 Like

Hello,

i made some tests those days and with the help of Honey_Patouceul, it is now possible to get Pycuda working with cupy and VPI libraries.

cupy is a kind of numpy for GPU and VPI is a NVidia library for image treatment which uses GPU capabilities.

Those libraries could be useful to replace some numpy / opencv functions and get faster treatments.

cupy can be used with both Linux and Windows systems.

VPI can be used with Jetson computers only.

To install cupy on Jetsons, you can do this (many thanks to Dusty_NV) :

From your work directory :

mkdir cupy
cd cupy

CUPY_VERSION=v10.2.0
CUPY_NVCC_GENERATE_CODE="arch=compute_53,code=sm_53;arch=compute_62,code=sm_62;arch=compute_72,code=sm_72;arch=compute_87,code=sm_87"

git clone -b ${CUPY_VERSION} --recursive https://github.com/cupy/cupy

cd cupy
pip3 install --no-cache-dir fastrlock
python3 setup.py install --user
cd ../
rm -rf cupy

I need to make some tests with those libraries. Later, i will release a JetsonSky version which uses both cupy and VPI libraries.

Alain

Hi Alain and all,

FYI, CuPy is offering binary packages for JetPack (aarch64) as well, so you can also use it instead of compiling it by your own:

https://docs.cupy.dev/en/latest/install.html#installing-cupy

Thanks,
Kenichi

Hello Kenichi,

many thanks for this information. In fact, i used this link to install cupy on my windows laptop but i did not noticed there was an aarch64 precompiled libreary !!!

You make great job with cupy. Really great. Many thanks.

Concerning Pycuda, i have forgotten to give here the solution Honey_Patouceul has found :

Change :

import pycuda.autoinit

into

import pycuda.autoprimaryctx

Quite magical.

Alain

Dear Alain,

I have thought again about the timelapse-probblem we discussed earlier. I won’t really bother you. I have bought the better 585MC. With it I can really go down with the exposure and I only use the amplification. So the post processing time is really short about 15 ms. Nonetheless I have these extreme timelapse problem, which seem even worse with the new cam. It is quite undefended what parameter I put in (or none), there is no difference in the recording speed. Therefor I do really think that it can not be the problem of exposuvretime or preprocessing time. It must have something to do with the recording itself. It would be really really nice if you could look into that. Well I have to do more investigation. I recorded the videos not as raw. I have never tried this, maybe there is an improvement. And I have used full res, maybe there is an improvement to go down with the res. Or I’m doing something stupid with the parameter, but I don’t think so. Could it be something with the videowriter-function?

Found this on StackOverflow:

“I have no idea why this fixed it… but changing the forcc to fourcc = cv2.VideoWriter_fourcc(*‘MP4V’) and then changing the output file from .avi to .mp4 fixed the problem.”

And this one:
I encountered the same issues. I think it was caused by the write action. You tried to save the video at 30fps, but the write action can’t handle that. For example, you want to save 180frame at 30fps, that video should be 6 seconds. But the write action can only save 10 frames per second, so 20 frames will be dropped per second. But the video still played at 30fps, so 6 seconds video becomes 2 seconds. Seems faster.

Best
Thomas

Hello Thomas,

JetsonSky saves the frames when they are ready. So, if JetsonSky can provide 10 fps, it will save only 10 frames per seconds. As the video setting is 25 fps in JetsonSky, you will have at the end a 25 fps video. This means the read speed of the video will be 2.5 times more fast than the real capture (which was only 10 fps).

In JetsonSky, i show the treatment time but there more things than treatment (acquisition, text in picture, resize, display of the frame in the window, etc.). If treatment time is only 15ms, you will get extra ms for other things to do the it will take more than 15ms to do the all job (acquisition, treatment, extra stuff).

I think you use ASI585MC full res, which is 4K resolution. It can be a bit hard for the Jetson to save 4K video very fast. Do you have the same problems with fullHD video (BIN2) ?

You can try to change the CODEC in JetsonSky.

Find those lines :

def video_capture() :
    global image_traitee,start_video,nb_cap_video,nb_acq_video,labelInfo1,flag_cap_video,video_path,val_nb_capt_video,video,echelle11,flag_pause_video,timer1,val_deltat,FDIF,flag_FDIF
    if nb_cap_video == 1 :
        if flag_HQ == 0:
            fourcc = cv2.VideoWriter_fourcc(*'XVID') # video compressée
        else :
            fourcc = 0 # video RAW
            #fourcc = cv2.VideoWriter_fourcc(*'DIB ') # video non compressee
        if flag_filter_wheel == True:
            nom_video = start_video.strftime('VID%Y%m%d_%H%M%S') + '_F' + "%01d" % fw_position + '.avi'
        else :
            nom_video = start_video.strftime('VID%Y%m%d_%H%M%S') + '.avi'
        if image_traitee.ndim == 3 :
            height,width,layers = image_traitee.shape

And change them into this :

def video_capture() :
    global image_traitee,start_video,nb_cap_video,nb_acq_video,labelInfo1,flag_cap_video,video_path,val_nb_capt_video,video,echelle11,flag_pause_video,timer1,val_deltat,FDIF,flag_FDIF
    if nb_cap_video == 1 :
        if flag_HQ == 0:
            fourcc = cv2.VideoWriter_fourcc(*'MP4V') # video compressée
            nom_video = start_video.strftime('VID%Y%m%d_%H%M%S') + '.mp4'
        else :
            fourcc = 0 # video RAW
            nom_video = start_video.strftime('VID%Y%m%d_%H%M%S') + '.avi'
        if image_traitee.ndim == 3 :
            height,width,layers = image_traitee.shape

I will make some tests asap. As i try to optimized JetsonSky, i will also see how i can improve video writing.

Just tell me the results of your tests.

Alain

Thanks for looking into that. I will try the codec change and then reply asap.
Best

First weird thing: i get avi at the end not mp4, don’t know why, everything seems to be ok
I took 1 minute videos:

Full res: with Filter off (Amplify) recorded 150 Frames Videolength is 6 Sek
on 100 Frames 5 Sek.

Bin2: off 400 Frames 16 Sek.
on 210 Frames 8 Sek.

I have recorded directly on USB3-Stick, maybe that is not so good also, it seems slightly better
if I record directly on the flash drive. I will check this later.

But now I understand better, what you have explained. There are too less frames recorded for a
25fps video and the result is a timelapse video because 3 frames dropped and only one is recorded in
best case.

If you choose HQ recording, you will get Avi uncompressed video.
If you don’t choose HQ recording, you will get MP4 compressed video.

USB3 stick may be a weak point. Data writing on that kind of device is not good. If you want high speed data writing, you will need USB3 SSD.

There is no frame dropped. Each time a frame is ready, this frame is recorded. Some frame can be dropped if camera read time is faster than treatment and recording. This is something I need to work on.

Alain

1 Like

This is just curiosity, but what devices would you normally save to? Is eMMC normally fast enough? Would an average old style platter-based hard drive be fast enough? I am assuming HQ.

Hello linuxdev,

well, if you get 4K video with RAW format and quite high speed framerate (i should say 20fps and above) you will need high bandwidth and i guess 100MB write speed or higher will be required.

eMMC could do the job but you also need plenty of room. When i was making Moon acquisition, with 4K video and no debayer format saving, my videos were quite heavy (more than 4GB for quite small videos; about 1500 frames). So, most of time, i used external USB3 drive (1TB, 300MB write speed) for my capture cessions).

Alain

Hello,

I have released a new version of JetsonSky on Github. It’s V17_01d Beta version.

I added support for CUPY and VPI libraries. VPI library only works with Linux systems and mainly with Jetson SBC.

Alain

1 Like

Back to astronomy.

I used frames from a 4K capture with JetsonSky, targetting Lagoon Nebula and Triphid Nebula.

I extracted 56 frames ans stacked them with DeepSkyStacker. The result is not really good but it is not bad.

Alain

1 Like

Hello,

i have released V17_02 for JetsonSky and JetsonTreatment on Github. The previous versions have been removed.

Just bug correction.

Alain

I have find some bugs in JetsonSky V17_02. I get some issues when changing resolution of the camera. I already have found the solution.
I also get lag when using small resolutions. Need to understand what’s going on.
I will release a new version asap.

Alain

2 Likes