Performance impact with jit coverted model using by libtorch on Jetson Xavier

We have a pre-trained segementation model with pytorch, to make it work also with libtorch for deployment on product, we used jit to convert the model.

After converting, we compared the performance, and found that the transfer learning speed is decreased(nearly half) with the convered model using libtorch on Jetson Xavier.
So the question is, is this as expected on Jetson Xavier? Any recommanded pratice for converting the model for usage with libtorch from pytorch on Jetson Xavier?

Environment

TensorRT Version : 7.1.3.0
GPU Type : Xavier NX
Nvidia Driver Version : Jetpack 4.4.1
CUDA Version : 10.2.89
CUDNN Version : 8.0.0.180
Operating System + Version : Ubuntu 18.04
PyTorch Version (if applicable) : 1.9
Baremetal or Container (if container which image + tag) :

Hi,

It’s recommended to upgrade your environment to our latest software version first.
Since libtorch is a third-party library, you can get more information from the library owner.

In general, we recommend users convert the model into a TensorRT engine.
It has optimized for the Jetson platform and can give you better performance and more friendly memory usage.

Thanks.

Thanks for the suggestion.

Any recommanded practice for coverting model from pytorch to TensorRT engine?

Hi,

You can check below for an example:

Thanks.

Thanks for the suggestion,

In our application, we’d like to enable the user with ability to re-train the model by annoting some positive/negative area.
However this is not supported by TensorRT?

Hi,

TensorRT is an inference engine so it doesn’t have a training algorithm. (ex. backpropagation).
But you can retrain the model with other frameworks (ex. PyTorch) and convert it into TensorRT after that.

Thanks.

Thanks for reply.

Actually, in our application, we used PyTorch, and when we wanted to integtration it to the product, we used jit tool to convert to libtorch.
And we found that, on Jetson Xavier, the converted model with libtorch is even slower in training when comparing to the original with PyTorch.
And this is very strange, we thought it might be we are not take a good opmization using the cuda resources.

Hi,

Just want to confirm first.

Do you see the slowness on Xaver compared to the desktop?
If yes, do you have a GPU on the desktop as well?

Thanks.

Hi,

Yes, we also see a sloness comparing performance on Xavier and windows desktop, but this is as expected, since we also have a GPU on the workstation desktop environment.

The problem is both on Xavier, we see a reduce of training performance comparing libtorch and pytorch.

BTW: we are considering could you kindly suggestted is any tutorial supporting building libtorch on Jetson Xavier? Thanks!

Hi,

Sorry that we don’t notice that libtorch runs slow on Xavier.

Do you also find the same behavior on inferencing time?
If yes, would you mind sharing a source and model for reproducing?
We want to give it a closer check.

Thanks.

Thanks.

We have loaded testing sample project in github, which also has this issue.
GitHub - lucify123/libtorch_train_slow_in_xavier
please kindly help to check if you can reproduce, thanks!

Hi,

Thanks for sharing the test sample.
We are trying this internally. Will share more information with you later.

Hi,

We try to reproduce this issue in our environment but meet some dependencies issues.

$ python3 pytorch_origin.py 
Traceback (most recent call last):
...
AttributeError: Can't get attribute 'Identity' on <module 'efficientnet_pytorch.utils' from '/home/nvidia/Pytorch/EfficientNet-PyTorch/efficientnet_pytorch/utils.py'>

Since some libraries are required for the sample.
Would you mind sharing the detailed installation steps and corresponding lib version with us as well?

Thanks.

Hi AastaLLL:
Sorry for your inconvenience.I work with yawei.yang.
The reason of this issue is for lack of model sturcture information.
You can just skip the" python3 pytorch_origin.py" steps to avoid this issue.because when use torch script,the structure of model is not needed.
Compared with " python3 pytorch_jit.py " and " * ./main", we can still get to the results to prove the issue.
We are using pytorch 1.8.0 from " PyTorch for Jetson - version 1.10 now available"

Thank you for your patient again~

Hi,

Thanks for the information.
Confirmed that we can reproduce the performance difference in our environment.
We are checking this with our internal team. Will share more information with you later.

1 Like

Hi,

Since there is a new JetPack and PyTorch package release recently.
Would you mind upgrading your environment to JetPack 4.6.1 + PyTorch v1.11 to see if the same issue occurs?

https://developer.download.nvidia.com/compute/redist/jp/v461/pytorch/

Thanks.

Hi,

As you suggest,I try JetPack 4.6.1 + PyTorch v1.11.But there’s compiled issue.
cmake … -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
– Found PythonInterp: /usr/bin/python (found version β€œ2.7.17”)
– The C compiler identification is GNU 7.5.0
– The CXX compiler identification is GNU 7.5.0
– Check for working C compiler: /usr/bin/cc
– Check for working C compiler: /usr/bin/cc – works
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Detecting C compile features
– Detecting C compile features - done
– Check for working CXX compiler: /usr/bin/c++
– Check for working CXX compiler: /usr/bin/c++ – works
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Detecting CXX compile features
– Detecting CXX compile features - done
– Looking for pthread.h
– Looking for pthread.h - found
– Looking for pthread_create
– Looking for pthread_create - not found
– Looking for pthread_create in pthreads
– Looking for pthread_create in pthreads - not found
– Looking for pthread_create in pthread
– Looking for pthread_create in pthread - found
– Found Threads: TRUE
– Found CUDA: /usr/local/cuda (found version β€œ10.2”)
– The CUDA compiler identification is NVIDIA 10.2.300
– Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
– Check for working CUDA compiler: /usr/local/cuda/bin/nvcc – works
– Detecting CUDA compiler ABI info
– Detecting CUDA compiler ABI info - done
– Caffe2: CUDA detected: 10.2
– Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
– Caffe2: CUDA toolkit directory: /usr/local/cuda
– Caffe2: Header version is: 10.2
– Found CUDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– /usr/local/cuda/lib64/libnvrtc.so shorthash is 7d272a04
– Autodetected CUDA architecture(s): 7.2
– Added CUDA NVCC flags for: -gencode;arch=compute_72,code=sm_72
– Found Torch: /home/lc/.local/lib/python3.6/site-packages/torch/lib/libtorch.so
– Found OpenCV: /usr (found version β€œ4.1.1”)
– Configuring done
– Generating done
– Build files have been written to: /home/lc/cc/libtorch_train_slow_in_xavier/build
lc@lc-desktop:~/cc/libtorch_train_slow_in_xavier/build$ make
Scanning dependencies of target main
[ 50%] Building CXX object CMakeFiles/main.dir/main.cpp.o
In file included from /usr/include/c++/7/bits/stl_algobase.h:59:0,
from /usr/include/c++/7/memory:62,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/aarch64-linux-gnu/c++/7/bits/c++config.h:250:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/aarch64-linux-gnu/c++/7/bits/c++config.h:438:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/bits/stl_algobase.h:59:0,
from /usr/include/c++/7/memory:62,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/aarch64-linux-gnu/c++/7/bits/c++config.h:563:3: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_DUAL_ABI && __cpp_transactional_memory >= 201505L
^~
In file included from /usr/include/c++/7/bits/stl_algobase.h:66:0,
from /usr/include/c++/7/memory:62,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/stl_iterator_base_funcs.h:107:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/string:52:0,
from /usr/include/c++/7/stdexcept:39,
from /usr/include/c++/7/array:39,
from /usr/include/c++/7/tuple:39,
from /usr/include/c++/7/bits/unique_ptr.h:37,
from /usr/include/c++/7/memory:80,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/basic_string.h:56:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/string:53:0,
from /usr/include/c++/7/stdexcept:39,
from /usr/include/c++/7/array:39,
from /usr/include/c++/7/tuple:39,
from /usr/include/c++/7/bits/unique_ptr.h:37,
from /usr/include/c++/7/memory:80,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/basic_string.tcc:50:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/array:39:0,
from /usr/include/c++/7/tuple:39,
from /usr/include/c++/7/bits/unique_ptr.h:37,
from /usr/include/c++/7/memory:80,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/stdexcept:46:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/stdexcept:127:28: error: operator β€˜||’ has no left operand
#if _GLIBCXX_USE_CXX11_ABI || _GLIBCXX_DEFINE_STDEXCEPT_COPY_OPS
^~
/usr/include/c++/7/stdexcept:211:28: error: operator β€˜||’ has no left operand
#if _GLIBCXX_USE_CXX11_ABI || _GLIBCXX_DEFINE_STDEXCEPT_COPY_OPS
^~
In file included from /usr/include/c++/7/bits/ios_base.h:41:0,
from /usr/include/c++/7/ios:42,
from /usr/include/c++/7/istream:38,
from /usr/include/c++/7/sstream:38,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/locale_classes.h:354:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/bits/ios_base.h:46:0,
from /usr/include/c++/7/ios:42,
from /usr/include/c++/7/istream:38,
from /usr/include/c++/7/sstream:38,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/system_error:91:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/ios:42:0,
from /usr/include/c++/7/istream:38,
from /usr/include/c++/7/sstream:38,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/ios_base.h:230:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/ios_base.h:254:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/sstream:297:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/sstream:826:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/sstream.tcc:92:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/locale:41:0,
from /usr/include/c++/7/iomanip:43,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/logging_is_not_google_glog.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/Logging.h:2,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorImpl.h:19,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/GeneratorImpl.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/Generator.h:22,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/Context.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/locale_facets_nonio.h:719:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/locale_facets_nonio.h:1567:33: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.h:1590:33: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.h:1712:33: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.h:1747:33: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
In file included from /usr/include/c++/7/bits/locale_facets_nonio.h:2013:0,
from /usr/include/c++/7/locale:41,
from /usr/include/c++/7/iomanip:43,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/logging_is_not_google_glog.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/Logging.h:2,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorImpl.h:19,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/GeneratorImpl.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/Generator.h:22,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/Context.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/locale_facets_nonio.tcc:352:33: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.tcc:564:33: error: operator β€˜&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
In file included from /usr/include/c++/7/list:63:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/OperatorEntry.h:17,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/Dispatcher.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/runtime/operator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/ir/ir.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/function_impl.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/method.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/object.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/frontend/tracer.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/autograd/generated/variable_factories.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/stl_list.h:326:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/stl_list.h:349:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/stl_list.h:1886:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/stl_list.h:1955:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/list:64:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/OperatorEntry.h:17,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/Dispatcher.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/runtime/operator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/ir/ir.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/function_impl.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/method.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/object.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/frontend/tracer.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/autograd/generated/variable_factories.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/list.tcc:178:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
CMakeFiles/main.dir/build.make:62: recipe for target β€˜CMakeFiles/main.dir/main.cpp.o’ failed
make[2]: *** [CMakeFiles/main.dir/main.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target β€˜CMakeFiles/main.dir/all’ failed
make[1]: *** [CMakeFiles/main.dir/all] Error 2
Makefile:83: recipe for target β€˜all’ failed
make: *** [all] Error 2

Hi,

Based on the suggestion online, it seems there are some issues with the g++.
Do you think it is possible to compile your app with clang?

Thanks.

Hi,

As you suggest,I use clang to avoid compiling issue: adding two lines in CMakeLists.txt before β€œproject”.
SET(CMAKE_C_COMPILER /usr/bin/clang)
SET(CMAKE_CXX_COMPILER /usr/bin/clang++)
#project(cpp_server)

But there’s another issue: OOM.
If we reduce input resolution,the training speed is still slower than pytorch’s jit version.
The source code can reproduce OOM issue.
Using jetpack 4.5,cudnn8.0,pytoch 1.8,there is no OOM issue.
Using jetpack 4.6,cudnn8.2,pytoch 1.8,There is OOM issue too.

Thanks

Thanks for the testing.

Not sure why the memory usage increased in JetPack 4.6.
We are still checking this internally.
Will share more information with you later.