Performance impact with jit coverted model using by libtorch on Jetson Xavier

Thanks for the suggestion,

In our application, we’d like to enable the user with ability to re-train the model by annoting some positive/negative area.
However this is not supported by TensorRT?

Hi,

TensorRT is an inference engine so it doesn’t have a training algorithm. (ex. backpropagation).
But you can retrain the model with other frameworks (ex. PyTorch) and convert it into TensorRT after that.

Thanks.

Thanks for reply.

Actually, in our application, we used PyTorch, and when we wanted to integtration it to the product, we used jit tool to convert to libtorch.
And we found that, on Jetson Xavier, the converted model with libtorch is even slower in training when comparing to the original with PyTorch.
And this is very strange, we thought it might be we are not take a good opmization using the cuda resources.

Hi,

Just want to confirm first.

Do you see the slowness on Xaver compared to the desktop?
If yes, do you have a GPU on the desktop as well?

Thanks.

Hi,

Yes, we also see a sloness comparing performance on Xavier and windows desktop, but this is as expected, since we also have a GPU on the workstation desktop environment.

The problem is both on Xavier, we see a reduce of training performance comparing libtorch and pytorch.

BTW: we are considering could you kindly suggestted is any tutorial supporting building libtorch on Jetson Xavier? Thanks!

Hi,

Sorry that we don’t notice that libtorch runs slow on Xavier.

Do you also find the same behavior on inferencing time?
If yes, would you mind sharing a source and model for reproducing?
We want to give it a closer check.

Thanks.

Thanks.

We have loaded testing sample project in github, which also has this issue.
GitHub - lucify123/libtorch_train_slow_in_xavier
please kindly help to check if you can reproduce, thanks!

Hi,

Thanks for sharing the test sample.
We are trying this internally. Will share more information with you later.

Hi,

We try to reproduce this issue in our environment but meet some dependencies issues.

$ python3 pytorch_origin.py 
Traceback (most recent call last):
...
AttributeError: Can't get attribute 'Identity' on <module 'efficientnet_pytorch.utils' from '/home/nvidia/Pytorch/EfficientNet-PyTorch/efficientnet_pytorch/utils.py'>

Since some libraries are required for the sample.
Would you mind sharing the detailed installation steps and corresponding lib version with us as well?

Thanks.

Hi AastaLLL:
Sorry for your inconvenience.I work with yawei.yang.
The reason of this issue is for lack of model sturcture information.
You can just skip the" python3 pytorch_origin.py" steps to avoid this issue.because when use torch script,the structure of model is not needed.
Compared with " python3 pytorch_jit.py " and " * ./main", we can still get to the results to prove the issue.
We are using pytorch 1.8.0 from " PyTorch for Jetson"

Thank you for your patient again~

Hi,

Thanks for the information.
Confirmed that we can reproduce the performance difference in our environment.
We are checking this with our internal team. Will share more information with you later.

1 Like

Hi,

Since there is a new JetPack and PyTorch package release recently.
Would you mind upgrading your environment to JetPack 4.6.1 + PyTorch v1.11 to see if the same issue occurs?

https://developer.download.nvidia.com/compute/redist/jp/v461/pytorch/

Thanks.

Hi,

As you suggest,I try JetPack 4.6.1 + PyTorch v1.11.But there’s compiled issue.
cmake … -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc
– Found PythonInterp: /usr/bin/python (found version “2.7.17”)
– The C compiler identification is GNU 7.5.0
– The CXX compiler identification is GNU 7.5.0
– Check for working C compiler: /usr/bin/cc
– Check for working C compiler: /usr/bin/cc – works
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Detecting C compile features
– Detecting C compile features - done
– Check for working CXX compiler: /usr/bin/c++
– Check for working CXX compiler: /usr/bin/c++ – works
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Detecting CXX compile features
– Detecting CXX compile features - done
– Looking for pthread.h
– Looking for pthread.h - found
– Looking for pthread_create
– Looking for pthread_create - not found
– Looking for pthread_create in pthreads
– Looking for pthread_create in pthreads - not found
– Looking for pthread_create in pthread
– Looking for pthread_create in pthread - found
– Found Threads: TRUE
– Found CUDA: /usr/local/cuda (found version “10.2”)
– The CUDA compiler identification is NVIDIA 10.2.300
– Check for working CUDA compiler: /usr/local/cuda/bin/nvcc
– Check for working CUDA compiler: /usr/local/cuda/bin/nvcc – works
– Detecting CUDA compiler ABI info
– Detecting CUDA compiler ABI info - done
– Caffe2: CUDA detected: 10.2
– Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
– Caffe2: CUDA toolkit directory: /usr/local/cuda
– Caffe2: Header version is: 10.2
– Found CUDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
– /usr/local/cuda/lib64/libnvrtc.so shorthash is 7d272a04
– Autodetected CUDA architecture(s): 7.2
– Added CUDA NVCC flags for: -gencode;arch=compute_72,code=sm_72
– Found Torch: /home/lc/.local/lib/python3.6/site-packages/torch/lib/libtorch.so
– Found OpenCV: /usr (found version “4.1.1”)
– Configuring done
– Generating done
– Build files have been written to: /home/lc/cc/libtorch_train_slow_in_xavier/build
lc@lc-desktop:~/cc/libtorch_train_slow_in_xavier/build$ make
Scanning dependencies of target main
[ 50%] Building CXX object CMakeFiles/main.dir/main.cpp.o
In file included from /usr/include/c++/7/bits/stl_algobase.h:59:0,
from /usr/include/c++/7/memory:62,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/aarch64-linux-gnu/c++/7/bits/c++config.h:250:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/aarch64-linux-gnu/c++/7/bits/c++config.h:438:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/bits/stl_algobase.h:59:0,
from /usr/include/c++/7/memory:62,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/aarch64-linux-gnu/c++/7/bits/c++config.h:563:3: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_DUAL_ABI && __cpp_transactional_memory >= 201505L
^~
In file included from /usr/include/c++/7/bits/stl_algobase.h:66:0,
from /usr/include/c++/7/memory:62,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/stl_iterator_base_funcs.h:107:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/string:52:0,
from /usr/include/c++/7/stdexcept:39,
from /usr/include/c++/7/array:39,
from /usr/include/c++/7/tuple:39,
from /usr/include/c++/7/bits/unique_ptr.h:37,
from /usr/include/c++/7/memory:80,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/basic_string.h:56:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/string:53:0,
from /usr/include/c++/7/stdexcept:39,
from /usr/include/c++/7/array:39,
from /usr/include/c++/7/tuple:39,
from /usr/include/c++/7/bits/unique_ptr.h:37,
from /usr/include/c++/7/memory:80,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/basic_string.tcc:50:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/array:39:0,
from /usr/include/c++/7/tuple:39,
from /usr/include/c++/7/bits/unique_ptr.h:37,
from /usr/include/c++/7/memory:80,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/stdexcept:46:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/stdexcept:127:28: error: operator ‘||’ has no left operand
#if _GLIBCXX_USE_CXX11_ABI || _GLIBCXX_DEFINE_STDEXCEPT_COPY_OPS
^~
/usr/include/c++/7/stdexcept:211:28: error: operator ‘||’ has no left operand
#if _GLIBCXX_USE_CXX11_ABI || _GLIBCXX_DEFINE_STDEXCEPT_COPY_OPS
^~
In file included from /usr/include/c++/7/bits/ios_base.h:41:0,
from /usr/include/c++/7/ios:42,
from /usr/include/c++/7/istream:38,
from /usr/include/c++/7/sstream:38,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/locale_classes.h:354:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/bits/ios_base.h:46:0,
from /usr/include/c++/7/ios:42,
from /usr/include/c++/7/istream:38,
from /usr/include/c++/7/sstream:38,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/system_error:91:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/ios:42:0,
from /usr/include/c++/7/istream:38,
from /usr/include/c++/7/sstream:38,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/ios_base.h:230:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/ios_base.h:254:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/sstream:297:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/sstream:826:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/macros/Macros.h:228,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/DeviceType.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Device.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/Allocator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/sstream.tcc:92:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/locale:41:0,
from /usr/include/c++/7/iomanip:43,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/logging_is_not_google_glog.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/Logging.h:2,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorImpl.h:19,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/GeneratorImpl.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/Generator.h:22,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/Context.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/locale_facets_nonio.h:719:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/locale_facets_nonio.h:1567:33: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.h:1590:33: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.h:1712:33: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.h:1747:33: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
In file included from /usr/include/c++/7/bits/locale_facets_nonio.h:2013:0,
from /usr/include/c++/7/locale:41,
from /usr/include/c++/7/iomanip:43,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/logging_is_not_google_glog.h:8,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/util/Logging.h:2,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/TensorImpl.h:19,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/c10/core/GeneratorImpl.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/Generator.h:22,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/Context.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/ATen.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:3,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/locale_facets_nonio.tcc:352:33: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
/usr/include/c++/7/bits/locale_facets_nonio.tcc:564:33: error: operator ‘&&’ has no right operand
&& _GLIBCXX_USE_CXX11_ABI == 0
^~
In file included from /usr/include/c++/7/list:63:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/OperatorEntry.h:17,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/Dispatcher.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/runtime/operator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/ir/ir.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/function_impl.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/method.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/object.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/frontend/tracer.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/autograd/generated/variable_factories.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/stl_list.h:326:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/stl_list.h:349:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/stl_list.h:1886:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
/usr/include/c++/7/bits/stl_list.h:1955:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
In file included from /usr/include/c++/7/list:64:0,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/OperatorEntry.h:17,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/ATen/core/dispatch/Dispatcher.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/runtime/operator.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/ir/ir.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/function_impl.h:4,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/method.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/api/object.h:6,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/jit/frontend/tracer.h:9,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/autograd/generated/variable_factories.h:12,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/csrc/api/include/torch/types.h:7,
from /home/lc/.local/lib/python3.6/site-packages/torch/include/torch/script.h:3,
from /home/lc/cc/libtorch_train_slow_in_xavier/main.cpp:1:
/usr/include/c++/7/bits/list.tcc:178:27: error: #if with no expression
#if _GLIBCXX_USE_CXX11_ABI
^
CMakeFiles/main.dir/build.make:62: recipe for target ‘CMakeFiles/main.dir/main.cpp.o’ failed
make[2]: *** [CMakeFiles/main.dir/main.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target ‘CMakeFiles/main.dir/all’ failed
make[1]: *** [CMakeFiles/main.dir/all] Error 2
Makefile:83: recipe for target ‘all’ failed
make: *** [all] Error 2

Hi,

Based on the suggestion online, it seems there are some issues with the g++.
Do you think it is possible to compile your app with clang?

Thanks.

Hi,

As you suggest,I use clang to avoid compiling issue: adding two lines in CMakeLists.txt before “project”.
SET(CMAKE_C_COMPILER /usr/bin/clang)
SET(CMAKE_CXX_COMPILER /usr/bin/clang++)
#project(cpp_server)

But there’s another issue: OOM.
If we reduce input resolution,the training speed is still slower than pytorch’s jit version.
The source code can reproduce OOM issue.
Using jetpack 4.5,cudnn8.0,pytoch 1.8,there is no OOM issue.
Using jetpack 4.6,cudnn8.2,pytoch 1.8,There is OOM issue too.

Thanks

Thanks for the testing.

Not sure why the memory usage increased in JetPack 4.6.
We are still checking this internally.
Will share more information with you later.

H Aasta,

Any updated information? This is a importance feature for our product, could you kindly help to highlight and proceed forwards? Thanks!

Hi,

Sorry that since Torch is a third-party library, the discussion and resources internally are quite limited.

Just want to confirm again.
Do you also see the performance difference between libtorch and PyTorch on a desktop environment?
If yes, have you checked with the torch team about this issue?

Thanks.

Hi,

Just some information.

We test this issue again on Orin with the latest JetPack 5.0.2 and PyTorch v1.13.0+nv22.07.
The performance between libtorch and PyTorch are almost the same.

libtorch

$ ./main
warming up,pls wait...
infer cost 1.35902 s
train cost 7.94993 s

Pytorch

$ python3 pytorch_jit.py
warming up,pls wait...
infer cost 1.661548376083374
train cost 7.868820667266846

Thanks.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.