TensorRT Plugin layers

Polyhedrus · May 17, 2018, 8:33am

Hi,

Difficult to know where on the forums to post this…

I have tried the face-recognition sample, on git, and it all works fine!

I have used DIGITS, created a huge trainset (from VGG_Face) and tested it using DIGITS, it works as expected - I then deployed it, ran the code against the above sample and it failed in tensorNet.cpp

for (auto& s : outputs) network->markOutput(*blobNameToTensor->find(s.c_str()));
The blobname was bboxes_fd - I presume this fail for all of the blobs?

How did the recognition model get trained? How do you add the IPlugin layer and blob fields into the network - when I try it complains about unknown layer type IPlugin

I am trying write a plug in layer, as per the above, So thats why I ask, how did you train it so I can get my set working with the sample code?

I used nvCaffe with GoogleNet.

Thanks

AastaLLL · May 17, 2018, 9:06am

Hi,

Here are some related topics for your reference:
https://devtalk.nvidia.com/default/topic/1023699/jetson-tx2/questions-about-face-recongnition/
https://devtalk.nvidia.com/default/topic/1007313/how-to-build-the-objection-detection-framework-ssd-with-tensorrt-on-tx2-/

Thanks.

Polyhedrus · May 17, 2018, 12:33pm

Hi,

Thanks for the reference links - very helpful. I did think it maybe a merged net but wasnt sure that it was possible!!

Did you facenet that comes with the jetson-inference sample of did you retrain with the FDDB? I have created my classification network and was hoping to just merge with the facenet detection network?

Thanks

AastaLLL · May 18, 2018, 7:02am

Hi,

YES. facenet is trained with FDDB.
Try to follow the steps shared in comment #2 and you should be able to get a merged net.

Thanks.

Polyhedrus · May 18, 2018, 7:25am

Yes working through this now. Creating a detection network is new to us, so picking our way throught it.

I presume the training was done in DIGITS? So, again, I presume the FDDB structure has to be converted to KITTI and then start the training?

Thanks

AastaLLL · May 21, 2018, 8:15am

Hi,

We have documented our detectNet training process here:
[url]https://github.com/dusty-nv/jetson-inference#locating-object-coordinates-using-detectnet[/url]

Thanks.

ningjiel · January 9, 2019, 7:00am

Hi,

I have tried the face-recognition sample, on git. However, I met some problem like ‘vector’ is not a member of ‘std’ with jetpack3.3 and Jestson TX2. Is that caused by different jetpack version or other reasons? Must I use the jetpack3.1?

Thanks.

AastaLLL · January 9, 2019, 7:53am

Hi,

Have you include the vector header in your source code?

#include <vector>

Thanks.

ningjiel · January 9, 2019, 8:11am

Hi,

Which file do you mean? I just try the sample as the github said.
$ sudo apt-get install git cmake
$ git clone GitHub - AastaNV/Face-Recognition: Demonstrate Plugin API for TensorRT2.1
$ cd Face-Recognition
$ mkdir build
$ cd build
$ cmake …
$ make

And the error is as bellows:

/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:5:45: error: ‘std::vector<bboxProfile*> RecognitionLayer::bboxTable’ is not a static data member of ‘class RecognitionLayer’
std::vector<bboxProfile*> RecognitionLayer::bboxTable;
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:6:44: error: ‘std::vector<tagProfile*> RecognitionLayer::tagTable’ is not a static data member of ‘class RecognitionLayer’
std::vector<tagProfile*> RecognitionLayer::tagTable;
^
In file included from /usr/include/c++/5/cassert:43:0,
from /home/nvidia/liuningjie/Face-Recognition/build/aarch64/include/pluginImplement.h:4,
from /home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:1:
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp: In member function ‘virtual nvinfer1::IPlugin* PluginFactory::createPlugin(const char*, const nvinfer1::Weights*, int)’:
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:24:16: error: ‘mBboxMergeLayer’ was not declared in this scope
assert(mBboxMergeLayer.get() == nullptr);
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:25:27: error: ‘unique_ptr’ is not a member of ‘std’
mBboxMergeLayer = std::unique_ptr(new BboxMergeLayer());
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:25:57: error: expected primary-expression before ‘>’ token
mBboxMergeLayer = std::unique_ptr(new BboxMergeLayer());
^
In file included from /usr/include/c++/5/cassert:43:0,
from /home/nvidia/liuningjie/Face-Recognition/build/aarch64/include/pluginImplement.h:4,
from /home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:1:
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:30:16: error: ‘mDataRoiLayer’ was not declared in this scope
assert(mDataRoiLayer.get() == nullptr);
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:31:25: error: ‘unique_ptr’ is not a member of ‘std’
mDataRoiLayer = std::unique_ptr(new DataRoiLayer());
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:31:53: error: expected primary-expression before ‘>’ token
mDataRoiLayer = std::unique_ptr(new DataRoiLayer());
^
In file included from /usr/include/c++/5/cassert:43:0,
from /home/nvidia/liuningjie/Face-Recognition/build/aarch64/include/pluginImplement.h:4,
from /home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:1:
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:36:16: error: ‘mSelectLayer’ was not declared in this scope
assert(mSelectLayer.get() == nullptr);
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:37:24: error: ‘unique_ptr’ is not a member of ‘std’
mSelectLayer = std::unique_ptr(new RecognitionLayer(FunctionType::SELECT));
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:37:56: error: expected primary-expression before ‘>’ token
mSelectLayer = std::unique_ptr(new RecognitionLayer(FunctionType::SELECT));
^
In file included from /usr/include/c++/5/cassert:43:0,
from /home/nvidia/liuningjie/Face-Recognition/build/aarch64/include/pluginImplement.h:4,
from /home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:1:
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:42:16: error: ‘mSummaryLayer’ was not declared in this scope
assert(mSummaryLayer.get() == nullptr);
^
/home/nvidia/liuningjie/Face-Recognition/pluginImplement.cpp:43:25: error: ‘unique_ptr’ is not a member of ‘std’
mSummaryLayer = std::unique_ptr(new RecognitionLayer(FunctionType::SUMMARY));

AastaLLL · January 10, 2019, 8:32am

Hi,

May I know your environment.
Have you updated any C++ library or compiler?

Thanks.

ningjiel · January 10, 2019, 9:01am

Hi,

My environment is jetson TX2 with jetpack3.3. Gcc and g++ version is gcc(Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609.

Thanks.

AastaLLL · January 11, 2019, 2:58am

Hi,

Okay. We will try to reproduce this issue internally first.
Thanks.

AastaLLL · January 11, 2019, 5:56am

Hi,

Here is the change for JetPack3.3:

diff --git a/face-recognition/face-recognition.cpp b/face-recognition/face-recognition.cpp
index 33b5dab..da6502a 100644
--- a/face-recognition/face-recognition.cpp
+++ b/face-recognition/face-recognition.cpp
@@ -240,7 +240,7 @@ int main(int argc, char** argv)
         if( display != NULL )
         {
             char str[256];
-            sprintf(str, "TensorRT build %x | %4.1f FPS", NV_GIE_VERSION, display->GetFPS());
+            sprintf(str, "TensorRT build %x | %4.1f FPS", NV_TENSORRT_MAJOR, display->GetFPS());
             display->SetTitle(str);
         }

diff --git a/pluginImplement.h b/pluginImplement.h
index 8b78a0a..14a39a2 100644
--- a/pluginImplement.h
+++ b/pluginImplement.h
@@ -5,6 +5,8 @@
 #include <iostream>
 #include <cudnn.h>
 #include <cstring>
+#include <vector>
+#include <memory>

 #include "NvCaffeParser.h"
 #include "NvInferPlugin.h"

Thanks.

ningjiel · January 22, 2019, 7:25am

Hi,

I have modified the face-recognition sample as above, and it compiles well. But, why the image is inverted or upside down from the camera of Jetson TX2? And what if I want to recognize one unknown person using this sample?

Thanks.

AastaLLL · January 25, 2019, 6:18am

Hi,

You will need to re-train the detection model.
This tutorial can give you some information:
[url]https://github.com/dusty-nv/jetson-inference#locating-object-coordinates-using-detectnet[/url]

Thanks.

Topic		Replies	Views
can't run the face-recognition sample Jetson TX2	4	481	October 18, 2021
How to build the objection detection framework SSD with tensorRT on tx2? Jetson TX2	96	21866	February 21, 2018
Converting Caffe model to TensorRT Jetson TX2	33	11476	October 18, 2021
Face detection using jetson inference and custom model Jetson Nano tensorrt , jetson-inference	6	2217	March 9, 2022
Create Object Detection Model without DIGITS? Jetson TX2	25	3281	October 18, 2021
TensorRT 10.3 does not support legacy caffe models for Jetpack6.2 Jetson Orin Nano cudnn , jetson , deepstream	6	190	February 3, 2025
Merging object detection and object classification Jetson TX2	10	1431	October 18, 2021
camera demos seem to fail due to not finding imports? Jetson Nano	14	1130	September 23, 2019
Face Recognition on TX2 or jetson nano using TensorRT Jetson TX2	12	4620	October 18, 2021
about deploying the caffe model inside Jeston TX2 using TensorRT TensorRT	6	1085	November 19, 2019

TensorRT Plugin layers

Related topics