In case it is helpful, I did re-run the last few steps of the notebook to show that the model should be ok:
!ls -rlt $USER_EXPERIMENT_DIR/experiment_dir_retrain/weights
total 43960
-rw-r–r-- 1 lx root 45012680 Mar 15 20:20 resnet18_detector_pruned.tlt
! rm -rf "$USER_EXPERIMENT_DIR/experiment_dir_final"
!mkdir -p $USER_EXPERIMENT_DIR/experiment_dir_final
# Removing a pre-existing copy of the etlt if there has been any.
import os
output_file=os.path.join(os.environ['USER_EXPERIMENT_DIR'],
"experiment_dir_final/resnet18_detector.etlt")
if os.path.exists(output_file):
os.system("rm {}".format(output_file))
!tlt-export detectnet_v2 \
-m $USER_EXPERIMENT_DIR/experiment_dir_retrain/weights/resnet18_detector_pruned.tlt \
-o $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.etlt \
-k $KEY`
Using TensorFlow backend.
NOTE: UFF has been tested with TensorFlow 1.14.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
DEBUG [/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py:96] Marking ['output_cov/Sigmoid', 'output_bbox/BiasAdd'] as outputs
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 2 output network tensors.
print(‘Exported model:’)
print(‘------------’)
!ls -lh $USER_EXPERIMENT_DIR/experiment_dir_final
Exported model:
total 43M
-rw-r–r-- 1 lx root 43M Mar 18 13:57 resnet18_detector.etlt
!tlt-int8-tensorfile detectnet_v2 -e $SPECS_DIR/detectnet_v2_retrain_resnet18_kitti.txt
-m 10
-o $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.tensor
Using TensorFlow backend.
2023-03-18 13:58:56,679 [INFO] iva.detectnet_v2.spec_handler.spec_loader: Merging specification from /workspace/specs/detectnet_v2_retrain_resnet18_kitti.txt
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
target/truncation is not updated to match the crop areaif the dataset contains target/truncation.
Writing calibration tensorfile: 100%|███████████| 10/10 [00:04<00:00, 2.32it/s]
Time taken to run iva.detectnet_v2.scripts.calibration_tensorfile:main: 0:00:06.858156.
!rm -rf $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.etlt
!tlt-export detectnet_v2
-m $USER_EXPERIMENT_DIR/experiment_dir_retrain/weights/resnet18_detector_pruned.tlt
-o $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.etlt
-k $KEY
–cal_data_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.tensor
–data_type int8
–batches 10
–batch_size 4
–max_batch_size 4
–engine_file $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.trt.int8
–cal_cache_file $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.bin
–verbose
Using TensorFlow backend.
2023-03-18 13:59:18,789 [DEBUG] iva.common.export.base_exporter: Saving etlt model file at: /workspace/tlt-experiments/experiment_dir_final/resnet18_detector.etlt.
2023-03-18 13:59:20,047 [DEBUG] modulus.export._uff: Patching keras BatchNormalization…
2023-03-18 13:59:20,047 [DEBUG] modulus.export._uff: Patching keras Dropout…
2023-03-18 13:59:20,047 [DEBUG] modulus.export._uff: Patching UFF TensorFlow converter apply_fused_padding…
2023-03-18 13:59:20,696 [DEBUG] modulus.export._uff: Unpatching keras BatchNormalization layer…
2023-03-18 13:59:20,696 [DEBUG] modulus.export._uff: Unpatching keras Dropout layer…
NOTE: UFF has been tested with TensorFlow 1.14.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
DEBUG [/usr/lib/python2.7/dist-packages/uff/converters/tensorflow/converter.py:96] Marking [‘output_cov/Sigmoid’, ‘output_bbox/BiasAdd’] as outputs
2023-03-18 13:59:21,884 [DEBUG] iva.common.export.base_exporter: Reading input dims from tensorfile.
2023-03-18 13:59:21,885 [DEBUG] modulus.export.data: Opening /workspace/tlt-experiments/experiment_dir_final/calibration.tensor with mode=r
2023-03-18 13:59:21,997 [DEBUG] iva.common.export.base_exporter: Input dims: (3, 368, 640)
2023-03-18 13:59:21,997 [DEBUG] modulus.export.data: Opening /workspace/tlt-experiments/experiment_dir_final/calibration.tensor with mode=r
2023-03-18 13:59:22,130 [DEBUG] modulus.export._tensorrt: read_calibration_cache - no-op
[TensorRT] INFO: Detected 1 inputs and 2 output network tensors.
[TensorRT] WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
[TensorRT] INFO: Starting Calibration with batch size 4.
DEPRECATED: This variant of get_batch is deprecated. Please use the single argument variant described in the documentation instead.
[TensorRT] INFO: Calibrated batch 0 in 0.0717277 seconds.
[TensorRT] INFO: Calibrated batch 1 in 0.0784402 seconds.
[TensorRT] INFO: Calibrated batch 2 in 0.0767087 seconds.
[TensorRT] INFO: Calibrated batch 3 in 0.0837051 seconds.
[TensorRT] INFO: Calibrated batch 4 in 0.0745355 seconds.
[TensorRT] INFO: Calibrated batch 5 in 0.0787708 seconds.
[TensorRT] INFO: Calibrated batch 6 in 0.0739311 seconds.
[TensorRT] INFO: Calibrated batch 7 in 0.0755091 seconds.
[TensorRT] INFO: Calibrated batch 8 in 0.0787474 seconds.
[TensorRT] INFO: Calibrated batch 9 in 0.0769921 seconds.
[TensorRT] INFO: Post Processing Calibration data in 1.134 seconds.
[TensorRT] INFO: Calibration completed in 11.0105 seconds.
2023-03-18 13:59:33,160 [DEBUG] modulus.export._tensorrt: read_calibration_cache - no-op
[TensorRT] INFO: Writing Calibration Cache for calibrator: TRT-7000-EntropyCalibration2
2023-03-18 13:59:33,160 [INFO] modulus.export._tensorrt: Saving calibration cache (size 4144) to /workspace/tlt-experiments/experiment_dir_final/calibration.bin
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INFO: Detected 1 inputs and 2 output network tensors.
!tlt-converter $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.etlt
-k "object-detection-from-sim-pipeline"
-c $USER_EXPERIMENT_DIR/experiment_dir_final/calibration.bin
-o output_cov/Sigmoid,output_bbox/BiasAdd
-d 3,384,1248
-i nchw
-m 64
-t int8
-e $USER_EXPERIMENT_DIR/experiment_dir_final/resnet18_detector.trt
-b 4
[INFO] Reading Calibration Cache for calibrator: EntropyCalibration2
[INFO] Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
[INFO] To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
[INFO] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[INFO] Detected 1 inputs and 2 output network tensors.
So all the above was on the PC with the Notebook in the docker image. Then I copy this ETLT file and BIN file to the AGX Xavier as follows:
rcp *.etlt carter@192.168.1.101:/home/carter/Downloads
rcp *.bin carter@192.168.1.101:/home/carter/Downloads
And on the Carter’s AGX Xavier I run these commands (remotely with SSH):
carter@carter-001:~/Downloads$ ls -l
total 129600
-rw-r–r-- 1 carter carter 4144 Mar 18 17:20 calibration.bin
-rw-r–r-- 1 carter carter 87573224 Mar 18 17:20 calibration.tensor
-rw-r–r-- 1 carter carter 44874341 Mar 18 17:20 resnet18_detector.etlt
-rwxrwxr-x 1 carter carter 120936 Mar 18 15:45 tao-converter
-rwxrwxr-x 1 carter carter 125136 Mar 16 21:19 tlt-converter
./tao-converter resnet18_detector.etlt
-k "object-detection-from-sim-pipeline"
-c calibration.bin
-o output_cov/Sigmoid,output_bbox/BiasAdd
-d 3,384,1248
-i nchw
-m 64
-t int8
-e resnet18_detector.trt
-b 4
and then I just get this message before it crashes:
[INFO] Reading Calibration Cache for calibrator: EntropyCalibration2
[INFO] Generated calibration scales using calibration cache. Make sure that calibration cache has latest scales.
[INFO] To regenerate calibration cache, please delete the existing one. TensorRT will generate a new calibration cache.
[INFO]
[INFO] --------------- Layers running on DLA:
[INFO]
[INFO] --------------- Layers running on GPU:
[INFO] conv1/convolution + activation_1/Relu, block_1a_conv_1/convolution + block_1a_relu_1/Relu, block_1a_conv_2/convolution, block_1a_conv_shortcut/convolution + add_1/add + block_1a_relu/Relu, block_1b_conv_1/convolution + block_1b_relu_1/Relu, block_1b_conv_2/convolution + add_2/add + block_1b_relu/Relu, block_2a_conv_1/convolution + block_2a_relu_1/Relu, block_2a_conv_2/convolution, block_2a_conv_shortcut/convolution + add_3/add + block_2a_relu/Relu, block_2b_conv_1/convolution + block_2b_relu_1/Relu, block_2b_conv_2/convolution + add_4/add + block_2b_relu/Relu, block_3a_conv_1/convolution + block_3a_relu_1/Relu, block_3a_conv_2/convolution, block_3a_conv_shortcut/convolution + add_5/add + block_3a_relu/Relu, block_3b_conv_1/convolution + block_3b_relu_1/Relu, block_3b_conv_2/convolution + add_6/add + block_3b_relu/Relu, block_4a_conv_1/convolution + block_4a_relu_1/Relu, block_4a_conv_2/convolution, block_4a_conv_shortcut/convolution + add_7/add + block_4a_relu/Relu, block_4b_conv_1/convolution + block_4b_relu_1/Relu, block_4b_conv_2/convolution + add_8/add + block_4b_relu/Relu, output_cov/convolution, output_cov/Sigmoid, output_bbox/convolution,
[INFO] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
Sometimes I don’t even get that last message