Hello AI World - new object detection training and video interfaces

[dusty_nv]

Hi everyone,

Happy to have just merged a significant set of updates and new features to jetson-inference master:

• Re-training SSD-Mobilenet Object Detection tutorial with PyTorch
• Support for collection of object detection datasets and bounding-box labeling in camera-capture tool
• videoSource and videoOutput APIs for C++/Python that supports multiple types of video streams:
  • MIPI CSI cameras
  • V4L2 cameras
  • RTP / RTSP
  • Videos & Images
  • Image sequences
  • OpenGL windows
• Unified the -console and -camera samples to process both images and video streams
  • imagenet.cpp / imagenet.py
  • detectnet.cpp / detectnet.py
  • segnet.cpp / segnet.py
• Support for uchar3/uchar4/float3/float4 images (default is now uchar3 as opposed to float4)
• Replaced opaque Python memory capsule with jetson.utils.cudaImage object
  • See Image Capsules in Python for more info
  • Images are now subscriptable/indexable from Python to directly access the pixel dataset
  • Numpy ndarray conversion now supports uchar3/uchar4/float3/float4 formats
• cudaConvertColor() automated colorspace conversion function (RGB, BGR, YUV, Bayer, grayscale, ect)
• Python CUDA bindings for cudaResize(), cudaCrop(), cudaNormalize(), cudaOverlay()
  • See Image Manipulation with CUDA and cuda-examples.py for examples of using these
• Transitioned to using Python3 by default since Python 2.7 is now past EOL
• DIGITS tutorial is now marked as deprecated (replaced by PyTorch transfer learning tutorial)
• Logging can now be controlled/disabled from the command line (e.g. --log-level=verbose)

note: API changes from this update are intended to be backwards-compatible, so previous code should still run.

The same code can now run from images/video/cameras and encoded network streams (RTP/RTSP):

import jetson.inference
import jetson.utils

import argparse
import sys

# parse the command line
parser = argparse.ArgumentParser(description="Locate objects in a live camera stream using an object detection DNN.")

parser.add_argument("input_URI", type=str, default="", nargs='?', help="URI of the input stream")
parser.add_argument("output_URI", type=str, default="", nargs='?', help="URI of the output stream")
parser.add_argument("--network", type=str, default="ssd-mobilenet-v2", help="pre-trained model to load (see below for options)")

try:
	opt = parser.parse_known_args()[0]
except:
	print("")
	parser.print_help()
	sys.exit(0)

# load the object detection network
net = jetson.inference.detectNet(opt.network, sys.argv)

# create video sources & outputs
input = jetson.utils.videoSource(opt.input_URI, argv=sys.argv)
output = jetson.utils.videoOutput(opt.output_URI, argv=sys.argv)

# process frames until EOS or the user exits
while True:
	img = input.Capture()
	detections = net.Detect(img)
	output.Render(img)
	output.SetStatus("{:s} | Network {:.0f} FPS".format(opt.network, net.GetNetworkFPS()))
	net.PrintProfilerTimes()

	if not input.IsStreaming() or not output.IsStreaming():
		break

For more info, see these new pages from the repo:

• Camera Streaming and Multimedia
• Image Manipulation with CUDA
• Re-training SSD-Mobilenet
• Collecting your own Detection Datasets

Thanks to everyone from the forums and GitHub who helped to test these updates in advance!

[ToTheFuture]

Than you very much, <<dusty_nv>>, an thanks to the whole nvidia-Jetson staff for maintaining up-to-date the Jetson-inference Tutorial.
Hope the same attention and approach will be dedicated to the whole < Two days to a demo>, Reinforcement and Tensorflow-RT (Cafee is deprecated?) : I think for all of us, even if we could have different interests, they are like Milestones and fundamental tutorials from which we could start.
Please keep them up-to-date with the evolutions of components and technologies that are packed within the evolving JetPacks.
My best to all !