my environment as follow:
centos7
Tensorrt 5.1.5.0
Cuda 10.0
Cudnn 7.5.1
GPU Tesla v100
python 3.6.5
tensorflow 1.14.0
model is bilstm-crf. I want to quantize it, but error occured when calibration.
code:
converter = trt.TrtGraphConverter(
input_saved_model_dir=‘models/ner’,
input_saved_model_tags=[‘serve’],
input_saved_model_signature_key=‘ner’,
nodes_blacklist=[‘result’],
max_batch_size=512,
max_workspace_size_bytes=2<<32,
precision_mode=‘INT8’,
is_dynamic_op=False)
trt_graph = converter.convert()
dataset = tf.data.Dataset.from_generator(…)
iter=dataset.make_one_shot_iterator()
seqs=iter.get_next()
result= tf.import_graph_def(trt_graph, return_elements=[‘result:0’], input_map={‘seqs:0’:=seqs}, name=‘’)
sess.run(result)
Traceback:
InternalError Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, *args)
1355 try:
→ 1356 return fn(*args)
1357 except errors.OpError as e:
~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1340 return self._call_tf_sessionrun(
→ 1341 options, feed_dict, fetch_list, target_list, run_metadata)
1342
~/anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1428 self._session, options, feed_dict, fetch_list, target_list,
→ 1429 run_metadata)
1430
InternalError: 2 root error(s) found.
(0) Internal: Failed to feed calibration data
[[{{node TRTEngineOp_3}}]]
[[ner/cond_2/Merge/_177]]
(1) Internal: Failed to feed calibration data
[[{{node TRTEngineOp_3}}]]
0 successful operations.
0 derived errors ignored.