Electronically Assisted Astronomy with a Jetson Nano

Ok, thx for clearing this out.

Maybe I’'m too stupid to understand that.
In theory I can follow your explanations. But as I mentioned earlier, if
I do absolutely NO processing I still get the same (timelapse) recording results.
And the time for processing is around 20 ms ( do not know it exactly
for the moment, have to figure it out again). It is nearly equal if I turn
on some processing or not. I will get this timelapse x6 every time. As I understand
you correct, there must be a difference in recording time, which is not.
I have the feeling that there must be a problem under the hood, as I
tried to mention earlier.
That’s why I asked again with the new gestreamer. I hoped that it could
have influence on this problem, which I guess now, it has not.
Best
Thommy

If you record 4K compressed file without hardware acceleration, it may take some time to compress the image and save it in the video file.

Maybe the problem comes from encoding time, i don’t know.

You should compile opencv with Gstreamer enabled to see if things are better or not.

It is not a question “stupid or not” but i can’t always imagine what could be the problem if i don’t have this problem. Hard for me to know what is going on with an issue if i don’t understand the issue.

Alain

I made tests with Jetson Xavier NX and JetsonSky (V30_01RC and V40_03RC).

Both work fine. The frame rate is a bit low but it is quite good (from 4 to 10 fps, depending of the settings).

Cupy, Pytorch and opencv with Cuda are ok.

If someone could test JetsonSky with a Orin Nano or a Orin NX, i would be interested with the results.

Alain

hm, I’m totally afraid of getting into compiling opencv with gestreamer. That is why I asked.

I’m hardly trying to install cupy. That is the first challenge for me (ohh Linux and me we will not getting the best
friends for ever, grrrrr).
Do not get it running.
I tried “pip install cupy” which installed me V12. I believe I have cuda 11.4 (maybe 11.7) installed, so that should be wrong. But I’m not able to install the correct cupy version. What do I have to do? setting path required ?
Installation went through without error. But if I import cupy I get tons of errors.
Thx for help.

Installing Cupy is not really complicated but it is better to try to find some informations before trying anything.

https://docs.cupy.dev/en/stable/install.html

If you have Jetpack 4 installed on your Jetson :

pip install cupy-cuda102 -f https://pip.cupy.dev/aarch64

If you have Jetpack 5 and CUDA 11.2 to 11.8 :

pip install cupy-cuda11x -f https://pip.cupy.dev/aarch64

If you have Jetpack 5 and CUDA 12 :

pip install cupy-cuda12x -f https://pip.cupy.dev/aarch64

You should uninstall cupy V12 and reinstall the good version of Cupy.

To install PyTorch :

pip3 install torch torchvision torchaudio

This afternoon, i have compiled opencv 4.6.0 with Gstreamer & Cuda for the Xavier NX. I have also installed Cupy and Pytorch and everything was ok.

Alain

There were too many bugs in V30_01RC and V40_03RC.

New versions have been uploaded (V30_02RC and V40_04RC) and V30_01RC and V40_03RC have been removed.

Alain

Considering my tests with Xavier NX and quite low FPS, i think AGX Xavier will give similar results.

4K video is rather big and need high end CPU and GPU. Memory transfers between CPU and GPU costs time and encoding also costs time.

JetsonSky was not created to get x1 speed videos so i guess it will never bring this.

You would need more powerful computer to get x1 video speed. For sure, if exposure time exceed 30ms or something like that, it will be impossible to get 1x video speed, whatever will be the computer. And JetsonSky is interesting if you use heavy treatments, which means time cost.

You should use Firecapture or Sharpcap instead of JetsonSky. Those software would probably match your needs.

Alain

I made some tests with V40 and V30 versions and Jetson AGX Orin.

Now, with better frame rate management, AGX Orin is about 30% faster than my laptop (i7-8750H & GTX1060 6G).

V30_02RC and V40_04RC seem to be more stable.

V30_02RC is the last version of JetsonSky with PyCuda. I will remove bugs but this version won’t have further evolution.

Cupy version (V40 and more) will be my new development platform.

Alain

Thanks Alain for help.
That information I have allready found. I have Jetpack 5 with my AGX.
I have found three folders cuda 11.4 and 11.7, the other one I do not remember.
I had deinstalled cupy 12 and tried to install 114 and 117. I get an error message,
that this is not the right version.
Maybe I try to upgrade cuda first to 12, do not know if it is already installed.

Question: do I have to write cupy-cuda11x??? I thought I have to put in the correct version
so I used 114 and 117. Maybe that is my fault.

Best Thommy

user@ubuntu:~$ pip3 install cupy-cuda11x -f CuPy: Arm Wheels
Defaulting to user installation because normal site-packages is not writeable
Looking in links: CuPy: Arm Wheels
Collecting cupy-cuda11x
Using cached cupy_cuda11x-12.0.0-cp38-cp38-manylinux2014_aarch64.whl (105.9 MB)
Requirement already satisfied: numpy<1.27,>=1.20 in ./.local/lib/python3.8/site-packages (from cupy-cuda11x) (1.24.3)
Requirement already satisfied: fastrlock>=0.5 in ./.local/lib/python3.8/site-packages (from cupy-cuda11x) (0.8.1)
Installing collected packages: cupy-cuda11x
Successfully installed cupy-cuda11x-12.0.0
user@ubuntu:~$ python3
Python 3.8.10 (default, Mar 13 2023, 10:26:41)
[GCC 9.4.0] on linux
Type “help”, “copyright”, “credits” or “license” for more information.

import cupy
Traceback (most recent call last):
File “”, line 1, in
File “/home/user/.local/lib/python3.8/site-packages/cupy/init.py”, line 31, in
import cupyx as _cupyx # NOQA
File “/home/user/.local/lib/python3.8/site-packages/cupyx/init.py”, line 8, in
from cupyx import linalg # NOQA
File “/home/user/.local/lib/python3.8/site-packages/cupyx/linalg/init.py”, line 2, in
from cupyx.linalg import sparse # NOQA
File “/home/user/.local/lib/python3.8/site-packages/cupyx/linalg/sparse/init.py”, line 3, in
from cupyx.linalg.sparse._solve import lschol # NOQA
File “/home/user/.local/lib/python3.8/site-packages/cupyx/linalg/sparse/_solve.py”, line 7, in
from cupyx.scipy import sparse
File “/home/user/.local/lib/python3.8/site-packages/cupyx/scipy/init.py”, line 4, in
from cupyx.scipy.sparse._base import spmatrix as _spmatrix
File “/home/user/.local/lib/python3.8/site-packages/cupyx/scipy/sparse/init.py”, line 1, in
from cupyx.scipy.sparse._base import issparse # NOQA
File “/home/user/.local/lib/python3.8/site-packages/cupyx/scipy/sparse/_base.py”, line 10, in
import scipy.sparse as _sparse
File “/usr/lib/python3/dist-packages/scipy/sparse/init.py”, line 229, in
from .base import *
File “/usr/lib/python3/dist-packages/scipy/sparse/base.py”, line 8, in
from .sputils import (isdense, isscalarlike, isintlike,
File “/usr/lib/python3/dist-packages/scipy/sparse/sputils.py”, line 16, in
supported_dtypes = [np.typeDict for x in supported_dtypes]
File “/usr/lib/python3/dist-packages/scipy/sparse/sputils.py”, line 16, in
supported_dtypes = [np.typeDict for x in supported_dtypes]
File “/home/user/.local/lib/python3.8/site-packages/numpy/init.py”, line 320, in getattr
raise AttributeError("module {!r} has no attribute "
AttributeError: module ‘numpy’ has no attribute ‘typeDict’

It doesen’t matter if I use pip or pip3 or sudo. I do get this errors every time.

I think you should not install Cuda 12. It will be useless and from what i know, Jetson do not support Cuda 12.

Try to update scipy :

pip install -U scipy

1 Like

You’re my hero. Thanks. That went through AND cupy AND
Jeston V40 running too (without camera for now, have to check
it later, but look very good)

Love you Mr. Magic!!!

Best Thommy

1 Like

Is it possible to explain a little more how you get these colors of the moon images?
Or is it your personal secret (I would understand and respect this).
I took a video of the moon and then tried to make some post processings with Denoise,
clahre (whatever) and saturation. But I only got rubbish.

I will explain how to over saturate Moon a bit later.

Just 1 thing : the video must be uncompressed format and white balance must be adjust to get equal response for R G & B (the Moon must be as grey as possible) before applying saturation enhancement.

Alain

1 Like

Hello,

If you are interested in :

When you get colour image (RGB format), if you want to make some calculus with this image, you may need to get separate channels arrays (2D) instead of the 3D array of the original image.

If you want 3 2D arrays (R, G and B), you will have to split your RGB image.

OpenCV gives you a useful function : cv2.split

ImgR, ImgG, ImgB = cv2.split(ImgRGB)

ImgR, ImgG, ImgB are 2D numpy arrays

This will give you real 2D arrays but cv2.split() cost a lot of time.

If you try this :

ImgR, ImgG, ImgB = ImgRGB[:, :, 0], ImgRGB[:, :, 1], ImgRGB[:, :, 2]

it will be fast but you will be disappointed because you won’t get real 2D arrays and you will get problems when you will use those arrays in a function which needs 2D arrays.

To avoid cv2.split() use with numpy or cupy, you can use numpy.ascontiguousarray() or cupy.ascontiguousarray().

As i want to use cupy instead of numpy, here are some useful functions that will split 1 3D array into 3 2D arrays :

import cupy as cp
import numpy as np

def cupy_RGBImage_2_cupy_separateRGB(cupyImageRGB):

    cupy_B = cp.ascontiguousarray(cupyImageRGB[:,:,0], dtype=cp.uint8)
    cupy_G = cp.ascontiguousarray(cupyImageRGB[:,:,1], dtype=cp.uint8)
    cupy_R = cp.ascontiguousarray(cupyImageRGB[:,:,2], dtype=cp.uint8)
    
    return cupy_B,cupy_G,cupy_R

def numpy_RGBImage_2_numpy_separateRGB(numpyImageRGB):

    numpy_B = np.ascontiguousarray(numpyImageRGB[:,:,0], dtype=np.uint8)
    numpy_G = np.ascontiguousarray(numpyImageRGB[:,:,1], dtype=np.uint8)
    numpy_R = np.ascontiguousarray(numpyImageRGB[:,:,2], dtype=np.uint8)
    
    return numpy_B,numpy_G,numpy_R

def numpy_RGBImage_2_cupy_separateRGB(numpyImageRGB):

    cupyImageRGB = cp.asarray(numpyImageRGB)
    cupy_B = cp.ascontiguousarray(cupyImageRGB[:,:,0], dtype=cp.uint8)
    cupy_G = cp.ascontiguousarray(cupyImageRGB[:,:,1], dtype=cp.uint8)
    cupy_R = cp.ascontiguousarray(cupyImageRGB[:,:,2], dtype=cp.uint8)
    
    return cupy_B,cupy_G,cupy_R

def cupy_RGBImage_2_numpy_separateRGB(cupyImageRGB):

    cupy_B = cp.ascontiguousarray(cupyImageRGB[:,:,0], dtype=cp.uint8)
    cupy_G = cp.ascontiguousarray(cupyImageRGB[:,:,1], dtype=cp.uint8)
    cupy_R = cp.ascontiguousarray(cupyImageRGB[:,:,2], dtype=cp.uint8)
    
    numpy_B = cupy_B.get()
    numpy_G = cupy_G.get()
    numpy_R = cupy_R.get()
   
    return numpy_B,numpy_G,numpy_R

Using cupy to split a 3D array is at least as fast as cv2.split() (in fact, cv2.cuda.split()).

With JetsonSky V40_XX, as i mainly use cupy, my GPU load (GTX1060 or Jetson AGX Orin GPU) has dramatically increase (about 40% with heavy filtering instead of about 10 to 20% with pycuda JetsonSky old versions).

That’s all.

Alain

1 Like

Hello,

i have uploaded 2 new versions of JetsonSky V30_03RC (Pycuda version) and V40_05RC (Cupy version).

Some bugs are solved.
FPS information is more stable (many thanks to @Honey_Patouceul).
Speed optimization.
Python priority is now set to HIGHEST. Need to start Python in super user mode with Windows. Increase FPS and decrease treatments time.

Alain

1 Like

Many work done with JetsonSky V40_05RC to remove bugs.

New version V40_06RC released on github.

I made a test on Jupiter (post processing). The result :

Alain

For linuxdev :

i already tried to get the Blue lake on the Moon. A bit hard with only a Celestron C9.25 because the lake is really small and needs very big magnification (9.25" mirror is a bit small for that).

Here is a very bad result i get some years ago. The pictures are not beautiful because treatment is pushed too far.


I do have to use my C9.25 again. I will next winter (in summer, Moon is too low above horizon here).

Alain

The images are spectacular, even when you don’t think of them as “the best”. Regarding Blue lake, I am curious, could you choose to only enhance the colors related to Blue lake? For example, use a color “eyedropper” to pick colors of the lake, and then choose only to enhance color for that spectrum? You’d still enhance clarity of the rest of the image, but it’d be interesting to see something which essentially highlights Blue lake.

Hi linuxdev,

it would be impossible to select only the Blue lake part to enhance colours only for it. Really to small and the RAW informations won’t allow me to choose the spectrum i want to enhance.

With real spectrometry, maybe we could choose what we want to enhance but spectrometry can’t really be done on an image.