int8_patch in caffe data_layer.cpp not generating batch files for INT8 inference

Hi,

I tried to follow TensorRT user guide for generating batch files for different datasets(currently MNIST), but it didn’t worked :

Step 1:

cd /samples/data
mkdir -p int8/mnist
cd int8/mnist

Step 2:

Inside caffe root directory:
patch -p1 < //sampleINT8/int8_caffe.patch
This changed my src/caffe/layers/data_layer.cpp file.

Step 3:

Recompiled caffe ‘make all’ by changing DEBUG:=1 in Makefile.config

Step 4:
Inside caffe root directory:

bash data/mnist/get_mnist.sh
bash examples/mnist/create_mnist.sh
cd …
ln -s caffe/examples .

Step 5:

mkdir batches
export TENSORRT_INT8_BATCH_DIRECTORY=
For eg: pwd: If batches folder is in /home/abc/batches, I did exportTENSORRT_INT8_BATCH_DIRECTORY=/home/abc/

Step 6: Ran the command:
caffe/build/tools/caffe test -gpu 0 -iterations 1000 -model
examples/mnist/lenet_train_test.prototxt -weights
/samples/mnist/mnist.caffemodel
ln -s /samples/mnist/mnist.caffemodel .
ln -s /samples/mnist/mnist.prototxt .

Step 7: The output log shows :

no_backup/d1230> caffe-master/build/tools/caffe test -gpu 0 -iterations 1 -model examples/mnist/lenet.prototxt -weights TensorRT-2.1.2/data/mnist/mnist.caffemodel
I0824 16:49:26.099154 28097 caffe.cpp:275] Use GPU with device ID 0
I0824 16:49:26.129475 28097 caffe.cpp:279] GPU device name: Graphics Device
I0824 16:49:26.800251 28097 net.cpp:51] Initializing net from parameters:
name: “LeNet”
state {
phase: TEST
level: 0
stage: “”
}
layer {
name: “data”
type: “Input”
top: “data”
input_param {
shape {
dim: 64
dim: 1
dim: 28
dim: 28
}
}
}
layer {
name: “conv1”
type: “Convolution”
bottom: “data”
top: “conv1”
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 20
kernel_size: 5
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
}
}
}
layer {
name: “pool1”
type: “Pooling”
bottom: “conv1”
top: “pool1”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “conv2”
type: “Convolution”
bottom: “pool1”
top: “conv2”
param {
lr_mult: 1
}
param {
lr_mult: 2
}
convolution_param {
num_output: 50
kernel_size: 5
stride: 1
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
}
}
}
layer {
name: “pool2”
type: “Pooling”
bottom: “conv2”
top: “pool2”
pooling_param {
pool: MAX
kernel_size: 2
stride: 2
}
}
layer {
name: “ip1”
type: “InnerProduct”
bottom: “pool2”
top: “ip1”
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 500
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
}
}
}
layer {
name: “relu1”
type: “ReLU”
bottom: “ip1”
top: “ip1”
}
layer {
name: “ip2”
type: “InnerProduct”
bottom: “ip1”
top: “ip2”
param {
lr_mult: 1
}
param {
lr_mult: 2
}
inner_product_param {
num_output: 10
weight_filler {
type: “xavier”
}
bias_filler {
type: “constant”
}
}
}
layer {
name: “prob”
type: “Softmax”
bottom: “ip2”
top: “prob”
}
I0824 16:49:26.800843 28097 layer_factory.hpp:77] Creating layer data
I0824 16:49:26.800900 28097 net.cpp:84] Creating Layer data
I0824 16:49:26.800923 28097 net.cpp:380] data -> data
I0824 16:49:26.843833 28097 net.cpp:122] Setting up data
I0824 16:49:26.843933 28097 net.cpp:129] Top shape: 64 1 28 28 (50176)
I0824 16:49:26.843941 28097 net.cpp:137] Memory required for data: 200704
I0824 16:49:26.843981 28097 layer_factory.hpp:77] Creating layer conv1
I0824 16:49:26.844084 28097 net.cpp:84] Creating Layer conv1
I0824 16:49:26.844115 28097 net.cpp:406] conv1 <- data
I0824 16:49:26.844172 28097 net.cpp:380] conv1 -> conv1
I0824 16:49:27.320663 28097 cudnn_conv_layer.cpp:194] Reallocating workspace storage: 20664
I0824 16:49:27.320716 28097 net.cpp:122] Setting up conv1
I0824 16:49:27.320744 28097 net.cpp:129] Top shape: 64 20 24 24 (737280)
I0824 16:49:27.320749 28097 net.cpp:137] Memory required for data: 3149824
I0824 16:49:27.320863 28097 layer_factory.hpp:77] Creating layer pool1
I0824 16:49:27.320924 28097 net.cpp:84] Creating Layer pool1
I0824 16:49:27.320937 28097 net.cpp:406] pool1 <- conv1
I0824 16:49:27.320978 28097 net.cpp:380] pool1 -> pool1
I0824 16:49:27.321074 28097 net.cpp:122] Setting up pool1
I0824 16:49:27.321087 28097 net.cpp:129] Top shape: 64 20 12 12 (184320)
I0824 16:49:27.321092 28097 net.cpp:137] Memory required for data: 3887104
I0824 16:49:27.321101 28097 layer_factory.hpp:77] Creating layer conv2
I0824 16:49:27.321143 28097 net.cpp:84] Creating Layer conv2
I0824 16:49:27.321151 28097 net.cpp:406] conv2 <- pool1
I0824 16:49:27.321187 28097 net.cpp:380] conv2 -> conv2
I0824 16:49:27.325822 28097 cudnn_conv_layer.cpp:194] Reallocating workspace storage: 10272
I0824 16:49:27.326429 28097 net.cpp:122] Setting up conv2
I0824 16:49:27.326473 28097 net.cpp:129] Top shape: 64 50 8 8 (204800)
I0824 16:49:27.326478 28097 net.cpp:137] Memory required for data: 4706304
I0824 16:49:27.326519 28097 layer_factory.hpp:77] Creating layer pool2
I0824 16:49:27.326575 28097 net.cpp:84] Creating Layer pool2
I0824 16:49:27.326586 28097 net.cpp:406] pool2 <- conv2
I0824 16:49:27.326607 28097 net.cpp:380] pool2 -> pool2
I0824 16:49:27.326721 28097 net.cpp:122] Setting up pool2
I0824 16:49:27.326736 28097 net.cpp:129] Top shape: 64 50 4 4 (51200)
I0824 16:49:27.326741 28097 net.cpp:137] Memory required for data: 4911104
I0824 16:49:27.326750 28097 layer_factory.hpp:77] Creating layer ip1
I0824 16:49:27.326781 28097 net.cpp:84] Creating Layer ip1
I0824 16:49:27.326791 28097 net.cpp:406] ip1 <- pool2
I0824 16:49:27.326810 28097 net.cpp:380] ip1 -> ip1
I0824 16:49:27.356739 28097 net.cpp:122] Setting up ip1
I0824 16:49:27.356787 28097 net.cpp:129] Top shape: 64 500 (32000)
I0824 16:49:27.356793 28097 net.cpp:137] Memory required for data: 5039104
I0824 16:49:27.356853 28097 layer_factory.hpp:77] Creating layer relu1
I0824 16:49:27.356897 28097 net.cpp:84] Creating Layer relu1
I0824 16:49:27.356910 28097 net.cpp:406] relu1 <- ip1
I0824 16:49:27.356935 28097 net.cpp:367] relu1 -> ip1 (in-place)
I0824 16:49:27.357224 28097 net.cpp:122] Setting up relu1
I0824 16:49:27.357236 28097 net.cpp:129] Top shape: 64 500 (32000)
I0824 16:49:27.357241 28097 net.cpp:137] Memory required for data: 5167104
I0824 16:49:27.357250 28097 layer_factory.hpp:77] Creating layer ip2
I0824 16:49:27.357276 28097 net.cpp:84] Creating Layer ip2
I0824 16:49:27.357285 28097 net.cpp:406] ip2 <- ip1
I0824 16:49:27.357305 28097 net.cpp:380] ip2 -> ip2
I0824 16:49:27.362922 28097 net.cpp:122] Setting up ip2
I0824 16:49:27.362957 28097 net.cpp:129] Top shape: 64 10 (640)
I0824 16:49:27.362979 28097 net.cpp:137] Memory required for data: 5169664
I0824 16:49:27.363004 28097 layer_factory.hpp:77] Creating layer prob
I0824 16:49:27.363059 28097 net.cpp:84] Creating Layer prob
I0824 16:49:27.363070 28097 net.cpp:406] prob <- ip2
I0824 16:49:27.363092 28097 net.cpp:380] prob -> prob
I0824 16:49:27.363414 28097 net.cpp:122] Setting up prob
I0824 16:49:27.363430 28097 net.cpp:129] Top shape: 64 10 (640)
I0824 16:49:27.363433 28097 net.cpp:137] Memory required for data: 5172224
I0824 16:49:27.363445 28097 net.cpp:200] prob does not need backward computation.
I0824 16:49:27.363452 28097 net.cpp:200] ip2 does not need backward computation.
I0824 16:49:27.363459 28097 net.cpp:200] relu1 does not need backward computation.
I0824 16:49:27.363464 28097 net.cpp:200] ip1 does not need backward computation.
I0824 16:49:27.363471 28097 net.cpp:200] pool2 does not need backward computation.
I0824 16:49:27.363476 28097 net.cpp:200] conv2 does not need backward computation.
I0824 16:49:27.363484 28097 net.cpp:200] pool1 does not need backward computation.
I0824 16:49:27.363490 28097 net.cpp:200] conv1 does not need backward computation.
I0824 16:49:27.363497 28097 net.cpp:200] data does not need backward computation.
I0824 16:49:27.363504 28097 net.cpp:242] This network produces output prob
I0824 16:49:27.363538 28097 net.cpp:255] Network initialization done.
I0824 16:49:27.379578 28097 net.cpp:744] Ignoring source layer mnist
I0824 16:49:27.379611 28097 net.cpp:744] Ignoring source layer scale
I0824 16:49:27.379617 28097 net.cpp:747] Copying source layer conv1
I0824 16:49:27.379711 28097 net.cpp:747] Copying source layer pool1
I0824 16:49:27.379716 28097 net.cpp:747] Copying source layer conv2
I0824 16:49:27.380110 28097 net.cpp:747] Copying source layer pool2
I0824 16:49:27.380115 28097 net.cpp:747] Copying source layer ip1
I0824 16:49:27.385720 28097 net.cpp:747] Copying source layer relu1
I0824 16:49:27.385735 28097 net.cpp:747] Copying source layer ip2
I0824 16:49:27.385818 28097 net.cpp:744] Ignoring source layer loss
I0824 16:49:27.385880 28097 caffe.cpp:290] Running for 1 iterations.
I0824 16:49:27.390357 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390403 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390411 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390415 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390439 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390444 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390449 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390453 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390486 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390492 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390496 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390501 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390506 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390511 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390516 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390537 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390544 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390549 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390554 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390559 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390565 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390570 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390574 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390579 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390584 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390589 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390594 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390599 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390604 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390609 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390614 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390619 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390625 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390630 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390635 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390640 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390645 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390650 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390655 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390658 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390673 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390681 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390688 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390697 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390702 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390708 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390712 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390718 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390724 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390728 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390733 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390738 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390743 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390748 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390753 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390758 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390763 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390768 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390772 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390776 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390782 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390787 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390792 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390796 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390805 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390810 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390813 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390826 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390832 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390837 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390842 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390847 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390852 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390857 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390862 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390866 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390872 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390877 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390882 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390887 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390892 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390897 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390902 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390905 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390911 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390916 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390920 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390925 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390930 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390935 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390940 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390944 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390950 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.390954 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.390959 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.390964 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.390969 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.390974 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.390980 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.390983 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.390990 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.390993 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.390998 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391003 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391010 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391013 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391018 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391022 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391028 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391032 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391037 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391042 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391048 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391052 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391057 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391062 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391067 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391072 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391077 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391082 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391086 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391091 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391096 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391101 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391106 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391111 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391115 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391126 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391134 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391140 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391144 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391149 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391155 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391160 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391165 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391170 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391175 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391180 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391185 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391188 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391194 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391198 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391202 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391206 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391212 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391217 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391222 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391227 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391232 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391237 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391242 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391247 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391252 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391258 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391261 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391266 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391273 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391276 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391281 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391286 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391291 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391296 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391301 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391305 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391311 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391316 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391320 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391325 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391330 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391335 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391340 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391345 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391350 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391355 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391360 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391364 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391369 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391374 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391379 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391383 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391389 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391394 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391399 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391403 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391409 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391414 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391418 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391429 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391435 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391440 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391445 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391450 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391455 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391460 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391464 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391469 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391475 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391479 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391484 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391489 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391494 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391499 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391505 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391510 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391515 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391520 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391525 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391530 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391535 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391540 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391544 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391548 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391554 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391558 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391563 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391568 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391573 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391578 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391583 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391588 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391593 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391598 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391603 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391608 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391613 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391618 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391623 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391628 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391633 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391638 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391644 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391649 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391654 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391659 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391664 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391667 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391674 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391680 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391685 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391688 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391695 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391700 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391703 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391708 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391715 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391719 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391729 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391736 28097 caffe.cpp:313] Batch 0, prob = 0.133425
I0824 16:49:27.391741 28097 caffe.cpp:313] Batch 0, prob = 0.0338381
I0824 16:49:27.391746 28097 caffe.cpp:313] Batch 0, prob = 0.0664772
I0824 16:49:27.391751 28097 caffe.cpp:313] Batch 0, prob = 0.0654982
I0824 16:49:27.391754 28097 caffe.cpp:313] Batch 0, prob = 0.228821
I0824 16:49:27.391760 28097 caffe.cpp:313] Batch 0, prob = 0.0896867
I0824 16:49:27.391765 28097 caffe.cpp:313] Batch 0, prob = 0.0834918
I0824 16:49:27.391769 28097 caffe.cpp:313] Batch 0, prob = 0.102563
I0824 16:49:27.391774 28097 caffe.cpp:313] Batch 0, prob = 0.114155
I0824 16:49:27.391782 28097 caffe.cpp:313] Batch 0, prob = 0.0820439
I0824 16:49:27.391788 28097 caffe.cpp:313] Batch 0, prob = 0

Step 8: I cannot see the batch0, batch1 etc in my batches folder. How to get these batch files ?

Step 9: I put gdb to see if my function load_batch() in data_layer.cpp which is patched gets called or not upon step 7, IT DOESN’T.

The code says that function is called when Caffe does prefetch. I dont understand how does caffe does prefetch and why my function doesnt gets called.

Any suggestions ? Please help !!!

Thanks a lot !
Have a nice day !

i meet the same problem as you ! could you find the solution to this issue?

Hi,

I resolved the problem by cloning the caffe at particular commit given in the TensorRT guide and then patching the int_8_patch file given in the samplesINT8 folder.

Make sure you checkout and clone the same caffe given in the Tensorrt user guide.

Thanks