Hi
I have successfully run the demos available in jetson inference. As I have been devloping all my code in python so I am bit comfortable in python instead of C++. But all the demo in jetson inference are with C++ only.
Just wanted to know if there is any python demo available. ?
Any link/tutorials/documentation.
Thanks
The C++ is very well commented. My preferred language is not C++ but I was able to follow. You might give it a shot anyway and post any problems you run into here.
Hi guys, thanks for the feedback - I do plan to add Python support and samples to jetson-inference.
For the time being, TensorRT also has a Python API that you could use directly.
Any link/tutorial for tensorRT python for jetson nano. Thanks
You should see them on your Jetson under /usr/src/tensorrt
Here’s also a link to the TensorRT Python API documentation:
[url]https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#python_topics[/url]
are you looking for object detection or classification? I have some tfrt object detection code that works ok, although I found the pytorch is faster.
Do you have any examples on github or anywhere else. I need some examples for reference so that I can later work it on my own.
Thanks
Not yet, i plan on putting on a zoo but currently still too busy exploring. If you let me know what are you looking for and on what framework i can send you something to play with
I am looking for tensorflow with basic object detection in python. If you can send any examples for this, that would be really helpful. Thanks
this should start you up - it requires no special setup IIRC. Remember that the one off stages are notoriously slow so don’t cntrl-c… If you improve on that, please post here:
from PIL import Image
from tf_trt_models.detection import download_detection_model
import numpy as np
import tensorflow as tf
import time
from tf_trt_models.detection import download_detection_model, build_detection_graph
import tensorflow.contrib.tensorrt as trt
import graphsurgeon as gs
import sys
from tf_trt_models.classification import download_classification_checkpoint
from tf_trt_models.classification import build_classification_graph
if len(sys.argv) > 1 and sys.argv[1] == "download":
config_path, checkpoint_path = download_detection_model('ssd_mobilenet_v1_coco')
print(config_path, checkpoint_path)
print("building frozen graph....")
frozen_graph, input_names, output_names = build_detection_graph(
config=config_path,
checkpoint=checkpoint_path
)
print(input_names,output_names)
print("building inference graph....")
trt_graph = trt.create_inference_graph(
input_graph_def=frozen_graph,
outputs=output_names,
max_batch_size=1,
max_workspace_size_bytes=1 << 25,
precision_mode='FP16',
minimum_segment_size=50
)
print("saving...")
with open('./ssd_mobilenet_v1_coco_trt.pb', 'wb') as f:
f.write(trt_graph.SerializeToString())
load_file = False
else:
load_file = True
trt_graph= tf.GraphDef()
with tf.gfile.GFile('./ssd_mobilenet_v1_coco_trt.pb', 'rb') as pf:
trt_graph.ParseFromString(pf.read())
input_names = ['image_tensor']
output_names = ['detection_boxes', 'detection_classes', 'detection_scores', 'num_detections']
print("creating session...")
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
tf_sess = tf.Session(config=tf_config)
tf.import_graph_def(trt_graph, name='')
tf_input = tf_sess.graph.get_tensor_by_name(input_names[0] + ':0')
tf_scores = tf_sess.graph.get_tensor_by_name('detection_scores:0')
tf_boxes = tf_sess.graph.get_tensor_by_name('detection_boxes:0')
tf_classes = tf_sess.graph.get_tensor_by_name('detection_classes:0')
tf_num_detections = tf_sess.graph.get_tensor_by_name('num_detections:0')
image = Image.open("t.jpg")
image_resized = np.array(image.resize((300, 300)))
image = np.array(image)
print("inference...")
for _ in range(10):
s = time.time()
scores, boxes, classes, num_detections = tf_sess.run([tf_scores, tf_boxes, tf_classes, tf_num_detections], feed_dict={
tf_input: image_resized[None, ...]
})
print(time.time()-s)
boxes = boxes[0] # index by 0 to remove batch dimension
scores = scores[0]
classes = classes[0]
num_detections = num_detections[0]
for i in range(int(num_detections)):
print('%d (%0.2f) ' % (classes[i], scores[i]))
oh, sorry, it does require the tf_trt_models from GitHub - NVIDIA-AI-IOT/tf_trt_models: TensorFlow models accelerated with NVIDIA TensorRT
did it work for you?
UPDATE (7/9): Python support has been merged into the master branch. No longer use the Python branch. See here for more info: https://devtalk.nvidia.com/default/topic/1057006/jetson-nano/hello-ai-world-now-supports-python-and-onboard-training-with-pytorch-/
OK, I’ve checked-in initial support for Python in Hello AI World here:
https://github.com/dusty-nv/jetson-inference/
If you could test it and let me know if it works for you, that would be helpful - thanks.
Here is the test to run:
# build the python branch
$ git clone -b python https://github.com/dusty-nv/jetson-inference jetson-inference-python
$ cd jetson-inference-python
$ git submodule update --init
$ mkdir build
$ cd build
$ cmake ../
$ sudo make install
# run the python recognition sample
$ cd aarch64/bin
$ python my-recognition.py --network=googlenet black_bear.jpg
And here is what the Python code looks like:
import jetson.inference
import jetson.utils
import argparse
# parse the command line
parser = argparse.ArgumentParser()
parser.add_argument("filename", type=str, help="filename of the image to process")
parser.add_argument("--network", type=str, default="googlenet", help="model to use, can be: alexnet, googlenet, googlenet-12")
opt = parser.parse_args()
print(opt)
# load an image (into shared CPU/GPU memory)
img, width, height = jetson.utils.loadImageRGBA(opt.filename)
# load the recognition network
net = jetson.inference.imageNet(opt.network)
# classify the image
class_idx, confidence = net.Classify(img, width, height)
# find the object description
class_desc = net.GetClassDesc(class_idx)
# print out the result
print("image is recognized as '{:s}' (class #{:d}) with {:f}% confidence".format(class_desc, class_idx, confidence * 100))
Underneath it is using the Python C API to call the C++ objects from jetson-inference using TensorRT, so the performance should still be good.
Currently I have the jetson.inference.imageNet ready, next I will work on the bindings for detectNet. I also plan to add support for numpy arrays.
Thank you dusty,
some comments:
it carries with it the overwhelming weight of qt4. If it can be make into something cleaner that does not require graphics it would be better as a library.
model downloads: It would be nice to be selective about what you want to download - it takes a while and take a lot of disk space. Also, the progress creates so many lines that you can’t go back and see what actually happened before.
while trying to run your example i got:
m@m-jetson-tests:/mnt/usbdrive/jetson-inference-python/build/aarch64/bin$ python3 my-recognition.py --network=googlenet black_bear.jpg
jetson.inference.__init__.py
Traceback (most recent call last):
File "my-recognition.py", line 23, in <module>
import jetson.inference
File "/usr/lib/python3.6/dist-packages/jetson/inference/__init__.py", line 4, in <module>
from jetson_inference_python import *
ImportError: libjetson-utils.so: cannot open shared object file: No such file or directory
Thanks, forgot that! Added that step above.
I will have to look into this, in the future perhaps it makes more sense to just use X11/XGL directly. The GUI is used for the camera applications.
That’s another good idea, perhaps I could make a curses-like interface to select the models. And maybe I can change wget to be less noisy. Thanks.
Can you try running “sudo ldconfig” and see if that helps?
Also are you able to run the existing C++ applications from jetson-inference-python/build/aarch64/bin (like imagenet-console)?
Thanks for beta testing!
sudo ldconfig did the trick!
it works well, around 20ms/image on my 10w nano (I know, I know, the power supply is just around the corner)
now for object detection…
OK thanks for letting me know, I did previously add this as an install step that runs during “sudo make install”, but I will take another look at why it didn’t seem to work initially for you (presumably related to a system running this branch for the first time).
I’m working for the bindings for detectNet now, and also camera. Will let you know when that branch of the repo is ready to try again.
Hi guys, ok the Python bindings for detectNet are ready to try, along with camera streaming and OpenGL display.
Re-clone the python branch of jetson-inference repo, and after building/installing it, try running these:
$ cd jetson-inference-python/build/aarch64/bin
$ ./camera-viewer.py # test camera streaming
$ ./detectnet-console.py --network=coco-dog dog_1.jpg test_dog_1.jpg
$ ./detectnet-console.py --network=pednet peds-004.jpg test_peds_4.jpg
$ ./detectnet-camera.py --network=facenet
And here is the Python code for realtime object detection with camera streaming, pretty simple!
import jetson.inference
import jetson.utils
import argparse
# parse the command line
parser = argparse.ArgumentParser()
parser.add_argument("--network", type=str, default="pednet", help="model to use, can be: pednet, multiped, facenet, coco-dog, coco-bottle, coco-chair, coco-airplane")
parser.add_argument("--threshold", type=float, default=0.5, help="minimum detection threshold to use")
parser.add_argument("--width", type=int, default=1280, help="desired width of camera stream (default is 1280 pixels)")
parser.add_argument("--height", type=int, default=720, help="desired height of camera stream (default is 720 pixels)")
parser.add_argument("--v4l2_device", type=int, default=-1, help="if using VL42 camera, index of the desired /dev/video node")
opt, argv = parser.parse_known_args()
# load the object detection network
net = jetson.inference.detectNet(opt.network, argv, opt.threshold)
# create the camera and display
camera = jetson.utils.gstCamera(opt.width, opt.height, opt.v4l2_device)
display = jetson.utils.glDisplay()
# process frames until user exits
while display.IsOpen():
# capture the image
img, width, height = camera.CaptureRGBA()
# detect objects in the image (with overlay)
detections = net.Detect(img, width, height)
# print the detections
print("detected {:d} objects in image".format(len(detections)))
for detection in detections:
print(detection)
# render the image
display.RenderOnce(img, width, height)
# update the title bar
display.SetTitle("{:s} | {:.0f} FPS".format(opt.network, display.GetFPS()))
Next I am going to add support for importing arrays into memory from numpy.
Hi dusty,
Thank you again for all the amazing work you both are doing. Couple of questions:
Thank again,
Moshe
Hi Moshe, thanks for your interest, here are my responses to your questions above:
The example camera programs (imagenet-camera.py and detectnet-camera.py) are setup to render the video with OpenGL, but if you remove the few lines of code in them for OpenGL display they will work headless. The imagenet-console.py and detectnet-console.py programs already work headless.
Not by default, but you can add RTSP to the GStreamer pipeline here:
https://github.com/dusty-nv/jetson-utils/blob/00fd5284b7a10dd9c9e6dbe12dc18b5736c61fb4/camera/gstCamera.cpp#L427
Right now it passes around CUDA memory using opaque Python capsule containers, so I am adding numpy as a way of getting data into and out of those containers. The capsules are used to achieve the best performance so that Python isn’t needlessly touching and copying around the GPU memory, which brings me to your question #4 below…
In comparing the C++ versions of the jetson-inference programs to their new Python counterparts, I see the same performance from both, which was my goal. To do this I wrote the Python bindings so they called into the C++ classes for image recognition and object detection that do the heavy lifting, and using the memory capsule containers mentioned above to route the buffers around without any extra memory copies.