Re-trained Pytorch Mask-RCNN inferencing in Jetson Nano

CodeLogist · September 15, 2020, 8:24pm

Hi Community,

I have trained a Custom Trained Pytorch Mask-RCNN network which takes image as an input and gives outputs the bounding box, masks with class and class labels. I have used Mask-RCNN model directly for the torchvision v0.4.0. The training and data preprocessing code is similar to vision/references/segmentation at main · pytorch/vision · GitHub and to get mask-rcnn model I have just used from torchvision.models.detection import MaskRCNN with no changes and trained it for 2 classes.

I tried to test this trained model on Jetson Nano without any use of ONNX/ DeepStream/TensorRT conversion and the swap memory(4GB) and the main memory(4GB) got filled up just while loading the model. The model weights .pth file is of size 241MB(just for ref).
I just installed the pytorch 1.2.0 and torchvision 0.4.0 as per the ref: PyTorch for Jetson

Can someone tell me what I should do or how I should optimize this model to run on Jetson Nano. Is it possible to run this model in DeepStream or TensorRT? How can I convert the model to run in such system?

I’m new on this hardware, so in need of some guidance.

Thanks in advance.
Regards,

AastaLLL · September 16, 2020, 5:52am

Hi,

Do you want to re-train the model or inference the model?

1
For re-training, you can find some information and sample in this GitHub:

github.com

dusty-nv/jetson-inference/blob/master/docs/pytorch-transfer-learning.md

<img src="https://github.com/dusty-nv/jetson-inference/raw/master/docs/images/deep-vision-header.jpg" width="100%">
<p align="right"><sup><a href="depthnet.md">Back</a> | <a href="pytorch-cat-dog.md">Next</a> | </sup><a href="../README.md#hello-ai-world"><sup>Contents</sup></a>
<br/>
<sup>Transfer Learning</sup></s></p>

# Transfer Learning with PyTorch

Transfer learning is a technique for re-training a DNN model on a new dataset, which takes less time than training a network from scratch.  With transfer learning, the weights of a pre-trained model are fine-tuned to classify a customized dataset.  In these examples, we'll be using the <a href="https://arxiv.org/abs/1512.03385">ResNet-18</a> and [SSD-Mobilenet](pytorch-ssd.md) networks, although you can experiment with other networks too.

<p align="center"><a href="https://arxiv.org/abs/1512.03385"><img src="https://github.com/dusty-nv/jetson-inference/raw/python/docs/images/pytorch-resnet-18.png" width="600"></a></p>

Although training is typically performed on a PC, server, or cloud instance with discrete GPU(s) due to the often large datasets used and the associated computational demands, by using transfer learning we're able to re-train various networks onboard Jetson to get started with training and deploying our own DNN models.  

<a href=https://pytorch.org/>PyTorch</a> is the machine learning framework that we'll be using, and example datasets along with training scripts are provided to use below, in addition to a camera-based tool for collecting and labeling your own training datasets.  

## Installing PyTorch

If you are [Running the Docker Container](aux-docker.md) or optionally chose to install PyTorch back when you [Built the Project](building-repo-2.md#installing-pytorch), it should already be installed on your Jetson to use.  Otherwise, if you aren't using the container and want to proceed with transfer learning, you can install it now:

``` bash

This file has been truncated. show original

2
For inferencing, it’s recommended to convert your model into TensorRT to save memory first:
Here is a sample for Mask-RCNN TensorFlow-based model:
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleUffMaskRCNN
Here is another sample for an ONNX-based (pytorch) model:
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMNIST

Thanks.

Topic		Replies	Views
Re-Trained Pytorch Mask-RCNN inferencing on Jetson Nano Deep Learning (Training & Inference) tensorrt , cuda , gstreamer , pytorch , onnx	0	1074	September 15, 2020
Not able to deploy Mask RCNN on Jetson Nano Jetson Nano	5	3245	October 6, 2021
How to deploy custom trained Mask RCNN on Jetson Nano? Jetson Nano camera , cuda , jetson-inference , python , computer-vision-cv	6	1818	April 18, 2022
Mask-rcnn Jetson B01 Jetson Nano neural-network-framework	4	2078	October 18, 2021
Own trained nn with weights Jetson Nano pytorch	6	1039	October 18, 2021
How to run my custom MaskRCNN model on ROS in nano jetson Triton Inference Server (archived) ros	0	704	March 23, 2021
Running custom trained models on Jetson Nano Deep Learning (Training & Inference) tensorrt , tensorflow , jetson-inference , pytorch	0	582	July 2, 2020
How to inference AI model that are designed and trained outside onto the jetson nano baord? Jetson Nano tensorrt , jetson-inference	2	301	April 19, 2024
MaskRcnn on Jetson nano Jetson Nano tensorrt	4	1727	October 18, 2021
Course project using GPU acceleration Jetson Nano jetson-inference	3	516	October 18, 2021

Re-trained Pytorch Mask-RCNN inferencing in Jetson Nano

Related topics