Hello AI World - now supports Python and onboard training with PyTorch!

dusty_nv · July 10, 2019, 8:04pm

Hi all, just merged a large set of updates and new features into jetson-inference master:

Python API support for imageNet, detectNet, and camera/display utilities
Python examples for processing static images and live camera streaming
Support for interacting with numpy ndarrays from CUDA
Onboard re-training of ResNet-18 models with PyTorch
Example datasets: 800MB Cat/Dog and 1.5GB PlantCLEF
Camera-based tool for collecting and labeling custom datasets
Text UI tool for selecting/downloading pre-trained models
New pre-trained image classification models (on 1000-class ImageNet ILSVRC)
- ResNet-18, ResNet-50, ResNet-101, ResNet-152
- VGG-16, VGG-19
- Inception-v4
New pre-trained object detection models (on 90-class MS-COCO)
- SSD-Mobilenet-v1
- SSD-Mobilenet-v2
- SSD-Inception-v2
API Reference documentation for C++ and Python
Command line usage info for all examples, run with --help
Output of network profiler times, including pre/post-processing
Improved font rasterization using system TTF fonts

Screencast video - Realtime Object Detection in 10 Lines of Python Code on Jetson Nano

Here’s an object detection example in 10 lines of Python code using SSD-Mobilenet-v2 (90-class MS-COCO) with TensorRT, which runs at 25FPS on Jetson Nano on a live camera stream with OpenGL visualization:

import jetson.inference
import jetson.utils

net = jetson.inference.detectNet("ssd-mobilenet-v2")
camera = jetson.utils.gstCamera()
display = jetson.utils.glDisplay()

while display.IsOpen():
	img, width, height = camera.CaptureRGBA()
	detections = net.Detect(img, width, height)
	display.RenderOnce(img, width, height)
	display.SetTitle("Object Detection | Network {:.0f} FPS".format(1000.0 / net.GetNetworkTime()))

Thanks to all the beta testers of the new features from here on the forums!

Project Link…https://github.com/dusty-nv/jetson-inference/
Model Mirror…https://github.com/dusty-nv/jetson-inference/releases

dusty_nv · January 31, 2020, 7:00pm

Hi all, we’ve just posted a screencast tutorial for Hello AI World - check it out!

Realtime Object Detection in 10 Lines of Python Code on Jetson Nano

camposromeromiguel · February 10, 2020, 11:11am

Hello,

I am trying to use a Basler acA5472-17uc with the code of your repository but I don’t know how do it to work. The objetive is do object detection with this device, I have already installed pypylon and the needed drivers.

This is how my code looks

import jetson.inference
import jetson.utils

camera = jetson.utils.gstCamera(5468, 3648, "/dev/video0")
display = jetson.utils.glDisplay()

while display.IsOpen():
	img = camera.CaptureRGBA()
	print("+++++", type(img))
	print("-----", type(in_arr))
	display.RenderOnce(img, width, height)

but it doesn’t work.

Also i have another code with pypylon:

#!/usr/bin/python

from pypylon import pylon
import cv2
import jetson.inference
import jetson.utils

camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
camera.Open()
display = jetson.utils.glDisplay()

# load network
net = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)

# demonstrate some feature access
new_width = camera.Width.GetValue() - camera.Width.GetInc()
if new_width >= camera.Width.GetMin():
	camera.Width.SetValue(new_width)

#camera.CaptureRGBA(zeroCopy=1)
camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)

while camera.IsGrabbing():
	grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)

	if grabResult.GrabSucceeded():
		# Access the image data.
		width, height, img =  grabResult.Width, grabResult.Height, grabResult.Array
                display.RenderOnce(img, width, height)

	grabResult.Release()
camera.Close()

My focus actually is in pre visualize what the basler can grab.
I will appreciate any help

dusty_nv · February 10, 2020, 3:53pm

Hi camposromeromiguel, what kind of data does pylon return from camera capture? Is it a numpy ndarray? If so, you could use the jetson.utils.cudaFromNumpy() function to convert it into GPU memory. There is an example of cudaFromNumpy() in /utils/python/examples

camposromeromiguel · February 11, 2020, 8:31am

Hello dusty_nv,

With respect the second block of code:

the camera get a numpy.array and a pycapsule after the jetson.utils.cudaFromNumpy().
Then I am working on try convert the image to RGBA from RGB because jetson expect get a 4 channel image.

With respect the first block of code:

Can I use the basler with

camera = jetson.utils.gstCamera()

At the moment I can’t

dusty_nv · February 12, 2020, 3:31pm

Hi camprosromeromiguel, see my reply here: https://devtalk.nvidia.com/default/topic/1071288/jetson-tx2/how-make-inference-with-basler-camera/post/5427936/#5427936

30216 · March 11, 2020, 7:12am

hello

I am trying to install jetson inference in my tx2 nvidia developer kit but this message display

[ 1%] Linking CXX shared library aarch64/lib/libjetson-inference.so
/usr/bin/ld: cannot find -lopencv_core
/usr/bin/ld: cannot find -lopencv_calib3d
collect2: error: ld returned 1 exit status
CMakeFiles/jetson-inference.dir/build.make:1242: recipe for target ‘aarch64/lib/libjetson-inference.so’ failed
make[2]: *** [aarch64/lib/libjetson-inference.so] Error 1
CMakeFiles/Makefile2:67: recipe for target ‘CMakeFiles/jetson-inference.dir/all’ failed
make[1]: *** [CMakeFiles/jetson-inference.dir/all] Error 2
Makefile:129: recipe for target ‘all’ failed
make: *** [all] Error 2

How can I solve his problem
best regards

dusty_nv · March 11, 2020, 3:59pm

Hi 30216, when you run cmake, do you see a message like “didn’t find OpenCV on system, disabling OpenCV” or “OpenCV version less than 3.0, disabling OpenCV”?

Try removing ‘opencv_core opencv_calib3d’ from this line of CMakeLists.txt: https://github.com/dusty-nv/jetson-inference/blob/fd65f317ec81606b419c7f1f285f5348ef206d9a/CMakeLists.txt#L154

#target_link_libraries(jetson-inference nvonnxparser opencv_core opencv_calib3d)
target_link_libraries(jetson-inference nvonnxparser)

30216 · March 26, 2020, 11:13am

Thank you, the second problem appears:
nujet@nujet-desktop:~$ cd jetson-inference
nujet@nujet-desktop:~/jetson-inference$ cd build
nujet@nujet-desktop:~/jetson-inference/build$ cmake …/

Note: this project uses git submodules in the source tree.
if you haven’t already, run the following command from
the project’s root directory:

       git submodule update --init

– using patched FindCUDA.cmake
– CUDA version: 10.0
– CUDA 10 detected, enabling SM_72
– OpenCV version: 4.1.1
– OpenCV version >= 3.0.0, enabling OpenCV
– system arch: aarch64
– output path: /home/nujet/jetson-inference/build/aarch64
– Copying /home/nujet/jetson-inference/c/detectNet.h
– Copying /home/nujet/jetson-inference/c/homographyNet.h
– Copying /home/nujet/jetson-inference/c/imageNet.h
– Copying /home/nujet/jetson-inference/c/segNet.h
– Copying /home/nujet/jetson-inference/c/superResNet.h
– Copying /home/nujet/jetson-inference/c/tensorNet.h
– Copying /home/nujet/jetson-inference/c/imageNet.cuh
– Copying /home/nujet/jetson-inference/calibration/randInt8Calibrator.h
– Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
– found Qt5Widgets version: 5.9.5
– found Qt5Widgets defines: -DQT_WIDGETS_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB
– found Qt5Widgets library: Qt5::Widgets
– found Qt5Widgets include: /usr/include/aarch64-linux-gnu/qt5/;/usr/include/aarch64-linux-gnu/qt5/QtWidgets;/usr/include/aarch64-linux-gnu/qt5/QtGui;/usr/include/aarch64-linux-gnu/qt5/QtCore;/usr/lib/aarch64-linux-gnu/qt5//mkspecs/linux-g++
– camera-capture: building as submodule, /home/nujet/jetson-inference/tools
– jetson-utils: building as submodule, /home/nujet/jetson-inference
– Copying /home/nujet/jetson-inference/utils/XML.h
– Copying /home/nujet/jetson-inference/utils/commandLine.h
– Copying /home/nujet/jetson-inference/utils/filesystem.h
– Copying /home/nujet/jetson-inference/utils/mat33.h
– Copying /home/nujet/jetson-inference/utils/pi.h
– Copying /home/nujet/jetson-inference/utils/rand.h
– Copying /home/nujet/jetson-inference/utils/timespec.h
– Copying /home/nujet/jetson-inference/utils/camera/gstCamera.h
– Copying /home/nujet/jetson-inference/utils/camera/v4l2Camera.h
– Copying /home/nujet/jetson-inference/utils/codec/gstDecoder.h
– Copying /home/nujet/jetson-inference/utils/codec/gstEncoder.h
– Copying /home/nujet/jetson-inference/utils/codec/gstUtility.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaFont.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaMappedMemory.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaNormalize.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaOverlay.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaRGB.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaResize.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaUtility.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaWarp.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaYUV.h
– Copying /home/nujet/jetson-inference/utils/display/glDisplay.h
– Copying /home/nujet/jetson-inference/utils/display/glTexture.h
– Copying /home/nujet/jetson-inference/utils/display/glUtility.h
– Copying /home/nujet/jetson-inference/utils/image/imageIO.h
– Copying /home/nujet/jetson-inference/utils/image/loadImage.h
– Copying /home/nujet/jetson-inference/utils/input/devInput.h
– Copying /home/nujet/jetson-inference/utils/input/devJoystick.h
– Copying /home/nujet/jetson-inference/utils/input/devKeyboard.h
– Copying /home/nujet/jetson-inference/utils/network/Endian.h
– Copying /home/nujet/jetson-inference/utils/network/IPv4.h
– Copying /home/nujet/jetson-inference/utils/network/NetworkAdapter.h
– Copying /home/nujet/jetson-inference/utils/network/Socket.h
– Copying /home/nujet/jetson-inference/utils/threads/Event.h
– Copying /home/nujet/jetson-inference/utils/threads/Mutex.h
– Copying /home/nujet/jetson-inference/utils/threads/Process.h
– Copying /home/nujet/jetson-inference/utils/threads/Thread.h
– trying to build Python bindings for Python versions: 2.7;3.6;3.7
– detecting Python 2.7…
– found Python version: 2.7 (2.7.17)
– found Python include: /usr/include/python2.7
– found Python library: /usr/lib/aarch64-linux-gnu/libpython2.7.so
– CMake module path: /home/nujet/jetson-inference/utils/cuda;/home/nujet/jetson-inference/utils/python/bindings
– NumPy ver. 1.13.3 found (include: /usr/lib/python2.7/dist-packages/numpy/core/include)
– found NumPy version: 1.13.3
– found NumPy include: /usr/lib/python2.7/dist-packages/numpy/core/include
– detecting Python 3.6…
– found Python version: 3.6 (3.6.9)
– found Python include: /usr/include/python3.6m
– found Python library: /usr/lib/aarch64-linux-gnu/libpython3.6m.so
– CMake module path: /home/nujet/jetson-inference/utils/cuda;/home/nujet/jetson-inference/utils/python/bindings
– NumPy ver. 1.16.1 found (include: /usr/local/lib/python3.6/dist-packages/numpy/core/include)
– found NumPy version: 1.16.1
– found NumPy include: /usr/local/lib/python3.6/dist-packages/numpy/core/include
– detecting Python 3.7…
– Python 3.7 wasn’t found
– Copying /home/nujet/jetson-inference/utils/python/examples/camera-viewer.py
– Copying /home/nujet/jetson-inference/utils/python/examples/cuda-from-numpy.py
– Copying /home/nujet/jetson-inference/utils/python/examples/cuda-to-numpy.py
– Copying /home/nujet/jetson-inference/utils/python/examples/gl-display-test.py
– trying to build Python bindings for Python versions: 2.7;3.6;3.7
– detecting Python 2.7…
– found Python version: 2.7 (2.7.17)
– found Python include: /usr/include/python2.7
– found Python library: /usr/lib/aarch64-linux-gnu/libpython2.7.so
– detecting Python 3.6…
– found Python version: 3.6 (3.6.9)
– found Python include: /usr/include/python3.6m
– found Python library: /usr/lib/aarch64-linux-gnu/libpython3.6m.so
– detecting Python 3.7…
– Python 3.7 wasn’t found
– Copying /home/nujet/jetson-inference/python/examples/detectnet-camera.py
– Copying /home/nujet/jetson-inference/python/examples/detectnet-console.py
– Copying /home/nujet/jetson-inference/python/examples/imagenet-camera.py
– Copying /home/nujet/jetson-inference/python/examples/imagenet-console.py
– Copying /home/nujet/jetson-inference/python/examples/my-detection.py
– Copying /home/nujet/jetson-inference/python/examples/my-recognition.py
– Copying /home/nujet/jetson-inference/python/examples/segnet-batch.py
– Copying /home/nujet/jetson-inference/python/examples/segnet-camera.py
– Copying /home/nujet/jetson-inference/python/examples/segnet-console.py
– Linking jetson-inference with OpenCV 4.1.1
– Configuring done
– Generating done
– Build files have been written to: /home/nujet/jetson-inference/build
nujet@nujet-desktop:~/jetson-inference/build$ make -j$(nproc)
[ 1%] Linking CXX shared library …/aarch64/lib/libjetson-utils.so
/usr/bin/ld: cannot find -lGL
collect2: error: ld returned 1 exit status
utils/CMakeFiles/jetson-utils.dir/build.make:2627: recipe for target ‘aarch64/lib/libjetson-utils.so’ failed
make[2]: *** [aarch64/lib/libjetson-utils.so] Error 1
CMakeFiles/Makefile2:927: recipe for target ‘utils/CMakeFiles/jetson-utils.dir/all’ failed
make[1]: *** [utils/CMakeFiles/jetson-utils.dir/all] Error 2
Makefile:129: recipe for target ‘all’ failed
make: *** [all] Error 2
nujet@nujet-desktop:~/jetson-inference/build$

dusty_nv · March 26, 2020, 3:47pm

Hi @30216, what does your output of ls -ll /usr/lib/aarch64-linux-gnu/libGL.so* look like? Perhaps you have a symbolic link broken - sometimes other 3rd-party graphical packages can overwrite libGL:

nvidia@jetson:~$ ls -ll /usr/lib/aarch64-linux-gnu/libGL.so*
lrwxrwxrwx 1 root root     14 May 10  2019 /usr/lib/aarch64-linux-gnu/libGL.so -> libGL.so.1.0.0
lrwxrwxrwx 1 root root     14 May 10  2019 /usr/lib/aarch64-linux-gnu/libGL.so.1 -> libGL.so.1.0.0
-rw-r--r-- 1 root root 972968 May 10  2019 /usr/lib/aarch64-linux-gnu/libGL.so.1.0.0

You may want to try restoring the symbolic links, or reinstalling the libglvnd-dev package.

30216 · March 26, 2020, 7:30pm

Hi, Thank you for your response and helping me, but the problem not solved as shown :
nujet@nujet-desktop:~$ sudo apt install libglvnd-dev

**[sudo] password for nujet:**

**Reading package lists... Done**

**Building dependency tree**

**Reading state information... Done**

**libglvnd-dev is already the newest version (1.0.0-2ubuntu2.3).**

**0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.**

**nujet@nujet-desktop:~$ ls -ll /usr/lib/aarch64-linux-gnu/libGL.so***

**lrwxrwxrwx 1 root root 14** **آذار** **25 21:53 /usr/lib/aarch64-linux-gnu/libGL.so -> tegra/libGL.so**

**lrwxrwxrwx 1 root root 14** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1 -> libGL.so.1.0.0**

**-rw-r--r-- 1 root root 972968** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1.0.0**

**nujet@nujet-desktop:~$ cd jetson-inference**

**nujet@nujet-desktop:~/jetson-inference$ cd build**

**nujet@nujet-desktop:~/jetson-inference/build$ make**

**[ 1%] Linking CXX shared library ../aarch64/lib/libjetson-utils.so**

**/usr/bin/ld: cannot find -lGL**

**collect2: error: ld returned 1 exit status**

**utils/CMakeFiles/jetson-utils.dir/build.make:2627: recipe for target 'aarch64/lib/libjetson-utils.so' failed**

**make[2]: *** [aarch64/lib/libjetson-utils.so] Error 1**

**CMakeFiles/Makefile2:927: recipe for target 'utils/CMakeFiles/jetson-utils.dir/all' failed**

**make[1]: *** [utils/CMakeFiles/jetson-utils.dir/all] Error 2**

**Makefile:129: recipe for target 'all' failed**

**make: *** [all] Error 2**

**nujet@nujet-desktop:~/jetson-inference/build$** **nujet@nujet-desktop:~$ sudo apt install libglvnd-dev**

**[sudo] password for nujet:**

**Reading package lists... Done**

**Building dependency tree**

**Reading state information... Done**

**libglvnd-dev is already the newest version (1.0.0-2ubuntu2.3).**

**0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.**

**nujet@nujet-desktop:~$ ls -ll /usr/lib/aarch64-linux-gnu/libGL.so***

**lrwxrwxrwx 1 root root 14** **آذار** **25 21:53 /usr/lib/aarch64-linux-gnu/libGL.so -> tegra/libGL.so**

**lrwxrwxrwx 1 root root 14** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1 -> libGL.so.1.0.0**

**-rw-r--r-- 1 root root 972968** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1.0.0**

**nujet@nujet-desktop:~$ cd jetson-inference**

**nujet@nujet-desktop:~/jetson-inference$ cd build**

**nujet@nujet-desktop:~/jetson-inference/build$ make**

**[ 1%] Linking CXX shared library ../aarch64/lib/libjetson-utils.so**

**/usr/bin/ld: cannot find -lGL**

**collect2: error: ld returned 1 exit status**

**utils/CMakeFiles/jetson-utils.dir/build.make:2627: recipe for target 'aarch64/lib/libjetson-utils.so' failed**

**make[2]: *** [aarch64/lib/libjetson-utils.so] Error 1**

**CMakeFiles/Makefile2:927: recipe for target 'utils/CMakeFiles/jetson-utils.dir/all' failed**

**make[1]: *** [utils/CMakeFiles/jetson-utils.dir/all] Error 2**

**Makefile:129: recipe for target 'all' failed**

**make: *** [all] Error 2**

best regards,

dusty_nv · March 26, 2020, 7:34pm

Try running this:

sudo rm /usr/lib/aarch64-linux-gnu/libGL.so
sudo ln -s /usr/lib/aarch64-linux-gnu/libGL.so.1 /usr/lib/aarch64-linux-gnu/libGL.so

30216 · March 27, 2020, 1:56pm

Hi, Thank you.

The problem was solved.

best regards,

Topic		Replies	Views
Is there any demos available for python jetson inference Jetson Nano	82	11649	September 30, 2019
Jetson-inference cmake fail with xvaier nx and jetpack5.0.1 Jetson Xavier NX jetson-inference	4	861	July 21, 2022
"Hello AI World" issue with building repository from source Deep Learning (Training & Inference)	0	332	November 6, 2020
How to solve the error when I test the "Hello AI World" example of Nvidia Jetson TX2	2	680	October 18, 2021
Error in Setting Up Hello AI World - Jetson Inference Jetson TX1 jetson-inference	4	573	October 18, 2021
Jetpack 4.4 Broke one of my programs Jetson Nano cudnn	24	3106	October 18, 2021
Create Object Detection Model without DIGITS? Jetson TX2	25	3281	October 18, 2021
Build OpenCV 3.4 with CUDA on NVIDIA Jetson TX2 Jetson TX2 opencv	14	9463	October 18, 2021
Jetpack 6.0 Pytorch & Torchvision Compatiblity Error Jetson Orin Nano pytorch	2	959	January 29, 2024
Issues with installing torchvision 0.13.0 Jetson AGX Orin pytorch	13	68	October 30, 2024

Hello AI World - now supports Python and onboard training with PyTorch!

Related topics