Hi
I am developing an Image Classification application with customized architecture of deep neural nets.
The output layer is a dense layer with 8 output neurons, each predicting the probability of one class.
On a laptop with 1050Ti, I am able to run the inference just fine. I use the predict() method on the TF.Keras.Model (https://www.tensorflow.org/api_docs/python/tf/keras/Model), and obtain a vector with 8 elements each storing the probability for one class. How do I obtain the same output vector on Jetson Nano?
Using Jeston Nano flashed by SDK Manager (Jetpack 4.4), DeepStream 5.0.1, Triton IS (TF.Keras 2.2) and Python App. Able to run the examples given in the Deepstream SDK (including the deepstream-ssd-parser example given in Python-Apps; with CPU).
For my custom code, I have gotten to a point where I am able to create the DS pipeline, load my model without any error and also run the inference with Triton IS. However its unclear to me how to parse the inference output such that I obtain the 8-element output vector storing the class probabilities, just like how I get on laptop.
Following the SSD example provided for the PythonApp, here are some code snippets that I am using.
===== From the main() module that creates and runs the Gst pipeline. The snippet adds a probe to the pgie source pad ======
pgie = make_elm_or_print_err("nvinferserver", "primary-inference", "Nvinferserver")
# Add a probe on the primary-infer source pad to get inference output tensors
pgiesrcpad = pgie.get_static_pad("src")
pgiesrcpad.add_probe(Gst.PadProbeType.BUFFER, pgie_src_pad_buffer_probe_custom, 0)
=================
The complete code for pgie_src_pad_buffer_probe_custom() is attached. pgie_src_pad_buffer_probe_custom.py.txt (1.6 KB)
Now I try to write the parser function for the output - to extract the 8-element vector storing the class probabilities.
===============
def nvds_infer_parse_custom_tf_customparser(output_layer_info):
class_layer = layer_finder(output_layer_info, "dense_1")
print(class_layer.dims.d)
classId = pyds.get_detections(class_layer.buffer, 0)
print(classId)
print(type(classId))
return(classId) #placeholder, to update once we figure out how to parse the 8 probabilities
====================
With this, when I run the main() module, I see the following on the console:
====================================
Creating Pipeline
Creating Source
Creating H264Parser
Creating Decoder
Creating NvStreamMux
Creating Nvinferserver
Creating Nvvidconv
Creating OSD (nvosd)
Creating Queue
Creating Converter 2 (nvvidconv2)
Creating capsfilter
Creating Encoder
Creating Code Parser
Creating Container
Creating Sink
Playing file ./sample_720p.h264
Adding elements to Pipeline
Linking elements in the Pipeline
Starting pipeline
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:123: PyGIDeprecationWarning: GObject.MainLoop is deprecated; use GLib.MainLoop instead
[8 0 0 0 0 0 0 0]
0.0
<class 'float'>
========================================
As you can see, I am able to get the handle on the output_layer. But no matter what I do, the classID detected is always a scalar and zero. What I expect is a vector of 8 elements storing float values between 0 and 1 (class probs). Is pyds.get_detections() the right method for my purpose? Or is there some other method of pyds that I should use?
Is there an example of Triton IS + Python App on Jetson platform for multiclass Image classification? I would love to see the code used to parse the PGIE source pad.
Thanks in advance!