How to use pycaffe using float16 onTX1

JOO12 · May 16, 2017, 9:19am

Hi,

I have a problem with checking the inference time of faster rcnn using nvcaffe_fp16 branch on TX1.
To check the inference time of faster rcnn using nvcaffe_fp16 branch, I already succeeded caffe build using nvcaffe_fp16 branch. And I modified the _caffe.cpp(python/caffe/) as follows:
typedef float16 Dtype;
typedef float16 Mtype;
const int NPY_DTYPE = NPY_FLOAT16;
After this, I did pycaffe build.
Then I tried to check the inference time on python but the error messages occurs as follows:
ImportError: /home/ubuntu/nvcaffe/python/caffe/_caffe.so: undefined symbol: _ZN5caffe6SolverINS_7float16ES1_E5SolveEPKc

For my case, I used TX1 installed jetpack3.0, cuda8.0 and cudnn5.1.

Can anybody help me?

JOO

AastaLLL · May 17, 2017, 2:35am

Hi,

Thanks for your question.

Faster-rcnn has their own caffe branch since self-implemented roi-pooling layer.
Please find more detail in this topic:
[url]https://devtalk.nvidia.com/default/topic/1004976/jetson-tx1/faster-r-cnn-on-jetson-tx1/post/5131544/#5131544[/url]

Thanks.

JOO12 · May 17, 2017, 5:25am

Hi,

Thanks for your reply but that is not my problem.

To begin with, I intended to ask the way to check the performance of faster rcnn using nvcaffe fp16 branch not their own caffe branch on TX1.

In my case, I already succeeded to implement faster rcnn using nvcaffe with cudnn5.1 by adding things related to roi_pooling layer and modifying thins related to cudnn5.1 so I obtained the performance of faster rcnn using nvcaffe branch whose type is float. But there is a problem with using nvcaffe float16 not float.
That is, in order to check the inference time of faster rcnn using nvcaffe float16, I modified the _caffe.cpp(python/caffe/) as follows:
(originally the type of Dtype and Mtype is float not float16 in _caffe.cpp)
typedef float16 Dtype;
typedef float16 Mtype;
const int NPY_DTYPE = NPY_FLOAT16;
After this, I did pycaffe build.
Then I tried to check the inference time on python but the error messages occurs as follows:
import caffe
File “/home/ubuntu/nvcaffe/python/caffe/__init__py”, line 1, in
from .pycaffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver
File “/home/ubuntu/nvcaffe/python/caffe/pycaffe.py”, line 13. in
from ._caffe import Net, SGDSolver, NesterovSolver, AdaGradSolver, RMSPropSolver, AdaDeltaSolver, AdamSolver,
ImportError: /home/ubuntu/nvcaffe/python/caffe/_caffe.so: undefined symbol: _ZN5caffe6SolverINS_7float16ES1_E5SolveEPKc

If you have the solution to fix this problem, please help me and let me know.

JOO

AastaLLL · May 19, 2017, 4:29am

Hi,

Thanks for your feedback and sorry for any misunderstanding.
We will test nvcaffe with python wrapper to check this issue.

Just want to check:
Do you build nvcaffe with office steps?

github.com

NVIDIA/DIGITS/blob/master/docs/BuildCaffe.md

# Building Caffe

DIGITS requires a build of Caffe.
We officially only support recent releases from [NVIDIA/caffe](https://github.com/NVIDIA/caffe) (NVcaffe), but any recent build of [BVLC/caffe](https://github.com/BVLC/caffe) will probably work too.

## Dependencies

Before Caffe can be build, Protobuf 3 needs to be build. Follow [this guide](BuildProtobuf.md) to build Protobuf 3

For best performance, you'll want:

* One or more NVIDIA GPUs ([details](InstallCuda.md#gpu))
* An NVIDIA driver ([details and installation instructions](InstallCuda.md#driver))
* A CUDA toolkit ([details and installation instructions](InstallCuda.md#cuda-toolkit))
* cuDNN ([download page](https://developer.nvidia.com/cudnn))

Install some dependencies with Deb packages:
```sh
sudo apt-get install --no-install-recommends build-essential cmake git gfortran libatlas-base-dev libboost-filesystem-dev libboost-python-dev libboost-system-dev libboost-thread-dev libgflags-dev libgoogle-glog-dev libhdf5-serial-dev libleveldb-dev liblmdb-dev libopencv-dev libsnappy-dev python-all-dev python-dev python-h5py python-matplotlib python-numpy python-opencv python-pil python-pip python-pydot python-scipy python-skimage python-sklearn
```

This file has been truncated. show original

JOO12 · May 19, 2017, 4:48am

Hi,

Thanks for your reply.

Actually I installed this branch,
https://github.com/nvidia/caffe/tree/experimental/fp16

and I builded nvaffe with office steps.

JOO

AastaLLL · May 22, 2017, 7:23am

Hi,

Sorry for that we can’t reproduce this issue since the branch ‘experimental/fp16’ is not available anymore.
(Branch caffe-0.16 requires cuDNNv6 up which is also not available in current JetPack)

Do you follow this page to build up nvcaffe?

github.com

dusty-nv/jetson-inference/blob/master/docs/building-nvcaffe.md

<img src="https://github.com/dusty-nv/jetson-inference/raw/master/docs/images/deep-vision-header.jpg" width="100%">

# Building nvcaffe

A special branch of caffe is used on TX1 which includes support for FP16.<br />
The code is released in NVIDIA's caffe repo in the experimental/fp16 branch, located here:
> https://github.com/nvidia/caffe/tree/experimental/fp16

#### 1. Installing Dependencies

``` bash
$ sudo apt-get update -y
$ sudo apt-get install cmake -y

# General dependencies
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev \
libhdf5-serial-dev protobuf-compiler -y
$ sudo apt-get install --no-install-recommends libboost-all-dev -y

# BLAS

This file has been truncated. show original

Could you check if this issue only occurs with python wrapper?

JOO12 · May 22, 2017, 8:52am

Hi,

Do you follow this page to build up nvcaffe?

github.com

dusty-nv/jetson-inference/blob/master/docs/building-nvcaffe.md

<img src="https://github.com/dusty-nv/jetson-inference/raw/master/docs/images/deep-vision-header.jpg" width="100%">

# Building nvcaffe

A special branch of caffe is used on TX1 which includes support for FP16.<br />
The code is released in NVIDIA's caffe repo in the experimental/fp16 branch, located here:
> https://github.com/nvidia/caffe/tree/experimental/fp16

#### 1. Installing Dependencies

``` bash
$ sudo apt-get update -y
$ sudo apt-get install cmake -y

# General dependencies
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev \
libhdf5-serial-dev protobuf-compiler -y
$ sudo apt-get install --no-install-recommends libboost-all-dev -y

# BLAS

This file has been truncated. show original

→ [Answer] Exactly right. I followed this page to build up nvcaffe.

Could you check if this issue only occurs with python wrapper?
→ [Answer] Yes. as I checked, this issue only occurs with python wrapper using float16. When I used float32 with python wrapper, there is no problem and I obtained the result.

JOO

AastaLLL · May 24, 2017, 10:13am

Hi,

Could you try if can run nvcaffe successfully with c++ interface in float16 mode?
Since the error looks like some missing implementation of flaot16 solver in _caffe.so

ImportError: /home/ubuntu/nvcaffe/python/caffe/_caffe.so: undefined symbol: _ZN5caffe6SolverINS_7float16ES1_E5SolveEPKc

Thanks.

JOO12 · May 29, 2017, 5:18am

Hi,

Sorry for late response.
As I checked what you asked for me, I successfully run nvcaffe with c++ inferface in float16 mode.
(Originally, I intended to ask you that nvcaffe float 16 mode with python interface since I already succeeded to run nvcaffe float16 mode with c++ interface.)

Now, What can I do to run nvcaffe float16 mode with python mode?
If you have the solution, please help me and let me know.

JOO

AastaLLL · May 31, 2017, 5:03am

Hi,

Thanks for your checking.
Guess that error may cause by some libraries for python interface.
Could you share source of this branch for us debugging? (Sorry since it is not available)

JOO12 · June 2, 2017, 12:57am

Hi,

I can share source of this branch but I don’t know how to upload the file.
So can you explain the way to share source?

JOO

AastaLLL · June 2, 2017, 2:10am

Hi,

Please click the “Add attachment” button at the upper right of a posted comment.

Thanks.

JOO12 · June 5, 2017, 5:25am

Hi,

I cannot upload file because of our security policy such as firewall.
so can you tell me how to share the source through another way?

Thanks.
JOO

AastaLLL · June 5, 2017, 7:46am

Hi,

Could you upload to some online space. Ex. google-drive
And share link here.