Hello AI World - now supports Python and onboard training with PyTorch!

dusty_nv · July 10, 2019, 8:04pm

Hi all, just merged a large set of updates and new features into jetson-inference master:

Python API support for imageNet, detectNet, and camera/display utilities
Python examples for processing static images and live camera streaming
Support for interacting with numpy ndarrays from CUDA
Onboard re-training of ResNet-18 models with PyTorch
Example datasets: 800MB Cat/Dog and 1.5GB PlantCLEF
Camera-based tool for collecting and labeling custom datasets
Text UI tool for selecting/downloading pre-trained models
New pre-trained image classification models (on 1000-class ImageNet ILSVRC)
- ResNet-18, ResNet-50, ResNet-101, ResNet-152
- VGG-16, VGG-19
- Inception-v4
New pre-trained object detection models (on 90-class MS-COCO)
- SSD-Mobilenet-v1
- SSD-Mobilenet-v2
- SSD-Inception-v2
API Reference documentation for C++ and Python
Command line usage info for all examples, run with --help
Output of network profiler times, including pre/post-processing
Improved font rasterization using system TTF fonts

Screencast video - Realtime Object Detection in 10 Lines of Python Code on Jetson Nano

Here’s an object detection example in 10 lines of Python code using SSD-Mobilenet-v2 (90-class MS-COCO) with TensorRT, which runs at 25FPS on Jetson Nano on a live camera stream with OpenGL visualization:

import jetson.inference
import jetson.utils

net = jetson.inference.detectNet("ssd-mobilenet-v2")
camera = jetson.utils.gstCamera()
display = jetson.utils.glDisplay()

while display.IsOpen():
	img, width, height = camera.CaptureRGBA()
	detections = net.Detect(img, width, height)
	display.RenderOnce(img, width, height)
	display.SetTitle("Object Detection | Network {:.0f} FPS".format(1000.0 / net.GetNetworkTime()))

Thanks to all the beta testers of the new features from here on the forums!

Project Link…https://github.com/dusty-nv/jetson-inference/
Model Mirror…https://github.com/dusty-nv/jetson-inference/releases

dusty_nv · January 31, 2020, 7:00pm

Hi all, we’ve just posted a screencast tutorial for Hello AI World - check it out!

Realtime Object Detection in 10 Lines of Python Code on Jetson Nano

camposromeromiguel · February 10, 2020, 11:11am

Hello,

I am trying to use a Basler acA5472-17uc with the code of your repository but I don’t know how do it to work. The objetive is do object detection with this device, I have already installed pypylon and the needed drivers.

This is how my code looks

import jetson.inference
import jetson.utils

camera = jetson.utils.gstCamera(5468, 3648, "/dev/video0")
display = jetson.utils.glDisplay()

while display.IsOpen():
	img = camera.CaptureRGBA()
	print("+++++", type(img))
	print("-----", type(in_arr))
	display.RenderOnce(img, width, height)

but it doesn’t work.

Also i have another code with pypylon:

#!/usr/bin/python

from pypylon import pylon
import cv2
import jetson.inference
import jetson.utils

camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateFirstDevice())
camera.Open()
display = jetson.utils.glDisplay()

# load network
net = jetson.inference.detectNet("ssd-mobilenet-v2", threshold=0.5)

# demonstrate some feature access
new_width = camera.Width.GetValue() - camera.Width.GetInc()
if new_width >= camera.Width.GetMin():
	camera.Width.SetValue(new_width)

#camera.CaptureRGBA(zeroCopy=1)
camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)

while camera.IsGrabbing():
	grabResult = camera.RetrieveResult(5000, pylon.TimeoutHandling_ThrowException)

	if grabResult.GrabSucceeded():
		# Access the image data.
		width, height, img =  grabResult.Width, grabResult.Height, grabResult.Array
                display.RenderOnce(img, width, height)

	grabResult.Release()
camera.Close()

My focus actually is in pre visualize what the basler can grab.
I will appreciate any help

dusty_nv · February 10, 2020, 3:53pm

Hi camposromeromiguel, what kind of data does pylon return from camera capture? Is it a numpy ndarray? If so, you could use the jetson.utils.cudaFromNumpy() function to convert it into GPU memory. There is an example of cudaFromNumpy() in /utils/python/examples

camposromeromiguel · February 11, 2020, 8:31am

Hello dusty_nv,

With respect the second block of code:

the camera get a numpy.array and a pycapsule after the jetson.utils.cudaFromNumpy().
Then I am working on try convert the image to RGBA from RGB because jetson expect get a 4 channel image.

With respect the first block of code:

Can I use the basler with

camera = jetson.utils.gstCamera()

At the moment I can’t

dusty_nv · February 12, 2020, 3:31pm

Hi camprosromeromiguel, see my reply here: https://devtalk.nvidia.com/default/topic/1071288/jetson-tx2/how-make-inference-with-basler-camera/post/5427936/#5427936

30216 · March 11, 2020, 7:12am

hello

I am trying to install jetson inference in my tx2 nvidia developer kit but this message display

[ 1%] Linking CXX shared library aarch64/lib/libjetson-inference.so
/usr/bin/ld: cannot find -lopencv_core
/usr/bin/ld: cannot find -lopencv_calib3d
collect2: error: ld returned 1 exit status
CMakeFiles/jetson-inference.dir/build.make:1242: recipe for target ‘aarch64/lib/libjetson-inference.so’ failed
make[2]: *** [aarch64/lib/libjetson-inference.so] Error 1
CMakeFiles/Makefile2:67: recipe for target ‘CMakeFiles/jetson-inference.dir/all’ failed
make[1]: *** [CMakeFiles/jetson-inference.dir/all] Error 2
Makefile:129: recipe for target ‘all’ failed
make: *** [all] Error 2

How can I solve his problem
best regards

dusty_nv · March 11, 2020, 3:59pm

Hi 30216, when you run cmake, do you see a message like “didn’t find OpenCV on system, disabling OpenCV” or “OpenCV version less than 3.0, disabling OpenCV”?

Try removing ‘opencv_core opencv_calib3d’ from this line of CMakeLists.txt: https://github.com/dusty-nv/jetson-inference/blob/fd65f317ec81606b419c7f1f285f5348ef206d9a/CMakeLists.txt#L154

#target_link_libraries(jetson-inference nvonnxparser opencv_core opencv_calib3d)
target_link_libraries(jetson-inference nvonnxparser)

30216 · March 26, 2020, 11:13am

Thank you, the second problem appears:
nujet@nujet-desktop:~$ cd jetson-inference
nujet@nujet-desktop:~/jetson-inference$ cd build
nujet@nujet-desktop:~/jetson-inference/build$ cmake …/

Note: this project uses git submodules in the source tree.
if you haven’t already, run the following command from
the project’s root directory:

       git submodule update --init

– using patched FindCUDA.cmake
– CUDA version: 10.0
– CUDA 10 detected, enabling SM_72
– OpenCV version: 4.1.1
– OpenCV version >= 3.0.0, enabling OpenCV
– system arch: aarch64
– output path: /home/nujet/jetson-inference/build/aarch64
– Copying /home/nujet/jetson-inference/c/detectNet.h
– Copying /home/nujet/jetson-inference/c/homographyNet.h
– Copying /home/nujet/jetson-inference/c/imageNet.h
– Copying /home/nujet/jetson-inference/c/segNet.h
– Copying /home/nujet/jetson-inference/c/superResNet.h
– Copying /home/nujet/jetson-inference/c/tensorNet.h
– Copying /home/nujet/jetson-inference/c/imageNet.cuh
– Copying /home/nujet/jetson-inference/calibration/randInt8Calibrator.h
– Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
– found Qt5Widgets version: 5.9.5
– found Qt5Widgets defines: -DQT_WIDGETS_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB
– found Qt5Widgets library: Qt5::Widgets
– found Qt5Widgets include: /usr/include/aarch64-linux-gnu/qt5/;/usr/include/aarch64-linux-gnu/qt5/QtWidgets;/usr/include/aarch64-linux-gnu/qt5/QtGui;/usr/include/aarch64-linux-gnu/qt5/QtCore;/usr/lib/aarch64-linux-gnu/qt5//mkspecs/linux-g++
– camera-capture: building as submodule, /home/nujet/jetson-inference/tools
– jetson-utils: building as submodule, /home/nujet/jetson-inference
– Copying /home/nujet/jetson-inference/utils/XML.h
– Copying /home/nujet/jetson-inference/utils/commandLine.h
– Copying /home/nujet/jetson-inference/utils/filesystem.h
– Copying /home/nujet/jetson-inference/utils/mat33.h
– Copying /home/nujet/jetson-inference/utils/pi.h
– Copying /home/nujet/jetson-inference/utils/rand.h
– Copying /home/nujet/jetson-inference/utils/timespec.h
– Copying /home/nujet/jetson-inference/utils/camera/gstCamera.h
– Copying /home/nujet/jetson-inference/utils/camera/v4l2Camera.h
– Copying /home/nujet/jetson-inference/utils/codec/gstDecoder.h
– Copying /home/nujet/jetson-inference/utils/codec/gstEncoder.h
– Copying /home/nujet/jetson-inference/utils/codec/gstUtility.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaFont.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaMappedMemory.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaNormalize.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaOverlay.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaRGB.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaResize.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaUtility.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaWarp.h
– Copying /home/nujet/jetson-inference/utils/cuda/cudaYUV.h
– Copying /home/nujet/jetson-inference/utils/display/glDisplay.h
– Copying /home/nujet/jetson-inference/utils/display/glTexture.h
– Copying /home/nujet/jetson-inference/utils/display/glUtility.h
– Copying /home/nujet/jetson-inference/utils/image/imageIO.h
– Copying /home/nujet/jetson-inference/utils/image/loadImage.h
– Copying /home/nujet/jetson-inference/utils/input/devInput.h
– Copying /home/nujet/jetson-inference/utils/input/devJoystick.h
– Copying /home/nujet/jetson-inference/utils/input/devKeyboard.h
– Copying /home/nujet/jetson-inference/utils/network/Endian.h
– Copying /home/nujet/jetson-inference/utils/network/IPv4.h
– Copying /home/nujet/jetson-inference/utils/network/NetworkAdapter.h
– Copying /home/nujet/jetson-inference/utils/network/Socket.h
– Copying /home/nujet/jetson-inference/utils/threads/Event.h
– Copying /home/nujet/jetson-inference/utils/threads/Mutex.h
– Copying /home/nujet/jetson-inference/utils/threads/Process.h
– Copying /home/nujet/jetson-inference/utils/threads/Thread.h
– trying to build Python bindings for Python versions: 2.7;3.6;3.7
– detecting Python 2.7…
– found Python version: 2.7 (2.7.17)
– found Python include: /usr/include/python2.7
– found Python library: /usr/lib/aarch64-linux-gnu/libpython2.7.so
– CMake module path: /home/nujet/jetson-inference/utils/cuda;/home/nujet/jetson-inference/utils/python/bindings
– NumPy ver. 1.13.3 found (include: /usr/lib/python2.7/dist-packages/numpy/core/include)
– found NumPy version: 1.13.3
– found NumPy include: /usr/lib/python2.7/dist-packages/numpy/core/include
– detecting Python 3.6…
– found Python version: 3.6 (3.6.9)
– found Python include: /usr/include/python3.6m
– found Python library: /usr/lib/aarch64-linux-gnu/libpython3.6m.so
– CMake module path: /home/nujet/jetson-inference/utils/cuda;/home/nujet/jetson-inference/utils/python/bindings
– NumPy ver. 1.16.1 found (include: /usr/local/lib/python3.6/dist-packages/numpy/core/include)
– found NumPy version: 1.16.1
– found NumPy include: /usr/local/lib/python3.6/dist-packages/numpy/core/include
– detecting Python 3.7…
– Python 3.7 wasn’t found
– Copying /home/nujet/jetson-inference/utils/python/examples/camera-viewer.py
– Copying /home/nujet/jetson-inference/utils/python/examples/cuda-from-numpy.py
– Copying /home/nujet/jetson-inference/utils/python/examples/cuda-to-numpy.py
– Copying /home/nujet/jetson-inference/utils/python/examples/gl-display-test.py
– trying to build Python bindings for Python versions: 2.7;3.6;3.7
– detecting Python 2.7…
– found Python version: 2.7 (2.7.17)
– found Python include: /usr/include/python2.7
– found Python library: /usr/lib/aarch64-linux-gnu/libpython2.7.so
– detecting Python 3.6…
– found Python version: 3.6 (3.6.9)
– found Python include: /usr/include/python3.6m
– found Python library: /usr/lib/aarch64-linux-gnu/libpython3.6m.so
– detecting Python 3.7…
– Python 3.7 wasn’t found
– Copying /home/nujet/jetson-inference/python/examples/detectnet-camera.py
– Copying /home/nujet/jetson-inference/python/examples/detectnet-console.py
– Copying /home/nujet/jetson-inference/python/examples/imagenet-camera.py
– Copying /home/nujet/jetson-inference/python/examples/imagenet-console.py
– Copying /home/nujet/jetson-inference/python/examples/my-detection.py
– Copying /home/nujet/jetson-inference/python/examples/my-recognition.py
– Copying /home/nujet/jetson-inference/python/examples/segnet-batch.py
– Copying /home/nujet/jetson-inference/python/examples/segnet-camera.py
– Copying /home/nujet/jetson-inference/python/examples/segnet-console.py
– Linking jetson-inference with OpenCV 4.1.1
– Configuring done
– Generating done
– Build files have been written to: /home/nujet/jetson-inference/build
nujet@nujet-desktop:~/jetson-inference/build$ make -j$(nproc)
[ 1%] Linking CXX shared library …/aarch64/lib/libjetson-utils.so
/usr/bin/ld: cannot find -lGL
collect2: error: ld returned 1 exit status
utils/CMakeFiles/jetson-utils.dir/build.make:2627: recipe for target ‘aarch64/lib/libjetson-utils.so’ failed
make[2]: *** [aarch64/lib/libjetson-utils.so] Error 1
CMakeFiles/Makefile2:927: recipe for target ‘utils/CMakeFiles/jetson-utils.dir/all’ failed
make[1]: *** [utils/CMakeFiles/jetson-utils.dir/all] Error 2
Makefile:129: recipe for target ‘all’ failed
make: *** [all] Error 2
nujet@nujet-desktop:~/jetson-inference/build$

dusty_nv · March 26, 2020, 3:47pm

Hi @30216, what does your output of ls -ll /usr/lib/aarch64-linux-gnu/libGL.so* look like? Perhaps you have a symbolic link broken - sometimes other 3rd-party graphical packages can overwrite libGL:

nvidia@jetson:~$ ls -ll /usr/lib/aarch64-linux-gnu/libGL.so*
lrwxrwxrwx 1 root root     14 May 10  2019 /usr/lib/aarch64-linux-gnu/libGL.so -> libGL.so.1.0.0
lrwxrwxrwx 1 root root     14 May 10  2019 /usr/lib/aarch64-linux-gnu/libGL.so.1 -> libGL.so.1.0.0
-rw-r--r-- 1 root root 972968 May 10  2019 /usr/lib/aarch64-linux-gnu/libGL.so.1.0.0

You may want to try restoring the symbolic links, or reinstalling the libglvnd-dev package.

30216 · March 26, 2020, 7:30pm

Hi, Thank you for your response and helping me, but the problem not solved as shown :
nujet@nujet-desktop:~$ sudo apt install libglvnd-dev

**[sudo] password for nujet:**

**Reading package lists... Done**

**Building dependency tree**

**Reading state information... Done**

**libglvnd-dev is already the newest version (1.0.0-2ubuntu2.3).**

**0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.**

**nujet@nujet-desktop:~$ ls -ll /usr/lib/aarch64-linux-gnu/libGL.so***

**lrwxrwxrwx 1 root root 14** **آذار** **25 21:53 /usr/lib/aarch64-linux-gnu/libGL.so -> tegra/libGL.so**

**lrwxrwxrwx 1 root root 14** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1 -> libGL.so.1.0.0**

**-rw-r--r-- 1 root root 972968** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1.0.0**

**nujet@nujet-desktop:~$ cd jetson-inference**

**nujet@nujet-desktop:~/jetson-inference$ cd build**

**nujet@nujet-desktop:~/jetson-inference/build$ make**

**[ 1%] Linking CXX shared library ../aarch64/lib/libjetson-utils.so**

**/usr/bin/ld: cannot find -lGL**

**collect2: error: ld returned 1 exit status**

**utils/CMakeFiles/jetson-utils.dir/build.make:2627: recipe for target 'aarch64/lib/libjetson-utils.so' failed**

**make[2]: *** [aarch64/lib/libjetson-utils.so] Error 1**

**CMakeFiles/Makefile2:927: recipe for target 'utils/CMakeFiles/jetson-utils.dir/all' failed**

**make[1]: *** [utils/CMakeFiles/jetson-utils.dir/all] Error 2**

**Makefile:129: recipe for target 'all' failed**

**make: *** [all] Error 2**

**nujet@nujet-desktop:~/jetson-inference/build$** **nujet@nujet-desktop:~$ sudo apt install libglvnd-dev**

**[sudo] password for nujet:**

**Reading package lists... Done**

**Building dependency tree**

**Reading state information... Done**

**libglvnd-dev is already the newest version (1.0.0-2ubuntu2.3).**

**0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.**

**nujet@nujet-desktop:~$ ls -ll /usr/lib/aarch64-linux-gnu/libGL.so***

**lrwxrwxrwx 1 root root 14** **آذار** **25 21:53 /usr/lib/aarch64-linux-gnu/libGL.so -> tegra/libGL.so**

**lrwxrwxrwx 1 root root 14** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1 -> libGL.so.1.0.0**

**-rw-r--r-- 1 root root 972968** **أيار** **10 2019 /usr/lib/aarch64-linux-gnu/libGL.so.1.0.0**

**nujet@nujet-desktop:~$ cd jetson-inference**

**nujet@nujet-desktop:~/jetson-inference$ cd build**

**nujet@nujet-desktop:~/jetson-inference/build$ make**

**[ 1%] Linking CXX shared library ../aarch64/lib/libjetson-utils.so**

**/usr/bin/ld: cannot find -lGL**

**collect2: error: ld returned 1 exit status**

**utils/CMakeFiles/jetson-utils.dir/build.make:2627: recipe for target 'aarch64/lib/libjetson-utils.so' failed**

**make[2]: *** [aarch64/lib/libjetson-utils.so] Error 1**

**CMakeFiles/Makefile2:927: recipe for target 'utils/CMakeFiles/jetson-utils.dir/all' failed**

**make[1]: *** [utils/CMakeFiles/jetson-utils.dir/all] Error 2**

**Makefile:129: recipe for target 'all' failed**

**make: *** [all] Error 2**

best regards,

dusty_nv · March 26, 2020, 7:34pm

Try running this:

sudo rm /usr/lib/aarch64-linux-gnu/libGL.so
sudo ln -s /usr/lib/aarch64-linux-gnu/libGL.so.1 /usr/lib/aarch64-linux-gnu/libGL.so

30216 · March 27, 2020, 1:56pm

Hi, Thank you.

The problem was solved.

best regards,