How to use Deep Stream with multiple model in single project ? help me

I am working with project use two model show in image link
https://drive.google.com/file/d/14_2Z4szmNaKo9_KNjw0i-FpnYRh1S_ro/view?usp=sharing
How to can i deploy with DeepStream? Help me

I had refered https://github.com/NVIDIA-AI-IOT/deepstream_reference_apps/tree/master/back-to-back-detectors some problem had sloved, but i have a question. How to config secondary_detector_config.txt file to deploy CRNN LSTM customize model?

Hi,

May I know more information about your model? Which format do you use?

Please noticed that not all kind of LSTM are supported.
Would you mind to check our support matrix here first?
https://docs.nvidia.com/deeplearning/sdk/tensorrt-archived/tensorrt-601/tensorrt-support-matrix/index.html

Thanks.

Hi
I use layer Conv2D, MaxPooling2D, Input, Dense, Activation, Bidirectional…

from keras import backend as K
from keras import regularizers
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Input, Dense, Activation, Bidirectional, Dropout
from keras.layers import Reshape, Lambda, BatchNormalization
from keras.models import Model
from keras.layers.recurrent import LSTM

Hi,

It’s recommended to verify your model is supported by the TensorRT first.

The general steps are:
1. Freeze your model into .pb file
2. Convert your model into uff format with convert_to_uff.py
3. Create a TensorRT engine with trtexec application.

Please follow this sample for the detail procedure.
/usr/src/tensorrt/samples/sampleUffSSD/

Thanks.

waoo
Thanks so much.

When i convert model.pb into model.uff
Use:
import uff
path_model = ‘weight_db/Model_S.pb’
uff.from_tensorflow(graphdef=path_model,
output_filename=“ab.uff”,
output_nodes=[“softmax/truediv”],
text=True)

NOTE: UFF has been tested with TensorFlow 1.15.0.
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
UFF Version 0.6.9
=== Automatically deduced input nodes ===
[name: “the_input”
op: “Placeholder”
attr {
key: “dtype”
value {
type: DT_FLOAT
}
}
attr {
key: “shape”
value {
shape {
dim {
size: -1
}
dim {
size: 110
}
dim {
size: 47
}
dim {
size: 1
}
}
}
}
]
=========================================

Using output node softmax/truediv
Converting to UFF graph
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: No conversion function registered for layer: Unpack yet.
Converting dense2/unstack as custom op: Unpack
WARNING:tensorflow:From /usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py:226: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.

Warning: No conversion function registered for layer: ReverseV2 yet.
Converting bidirectional_1/ReverseV2_1 as custom op: ReverseV2
Warning: No conversion function registered for layer: TensorArrayGatherV3 yet.
Converting bidirectional_1/TensorArrayStack_1/TensorArrayGatherV3 as custom op: TensorArrayGatherV3
Warning: No conversion function registered for layer: Exit yet.
Converting bidirectional_1/while_1/Exit_2 as custom op: Exit
Warning: No conversion function registered for layer: Switch yet.
Converting bidirectional_1/while_1/Switch_2 as custom op: Switch
Warning: No conversion function registered for layer: LoopCond yet.
Converting bidirectional_1/while_1/LoopCond as custom op: LoopCond
Warning: No conversion function registered for layer: LogicalAnd yet.
Converting bidirectional_1/while_1/LogicalAnd as custom op: LogicalAnd
Warning: No conversion function registered for layer: Less yet.
Converting bidirectional_1/while_1/Less_1 as custom op: Less
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/Less_1/Enter as custom op: Enter
Warning: No conversion function registered for layer: ReverseV2 yet.
Converting bidirectional_1/ReverseV2 as custom op: ReverseV2
Warning: No conversion function registered for layer: Unpack yet.
Converting dense1/unstack as custom op: Unpack
Warning: No conversion function registered for layer: Merge yet.
Converting bidirectional_1/while_1/Merge_1 as custom op: Merge
Warning: No conversion function registered for layer: NextIteration yet.
Converting bidirectional_1/while_1/NextIteration_1 as custom op: NextIteration
Warning: No conversion function registered for layer: Switch yet.
Converting bidirectional_1/while_1/Switch as custom op: Switch
Warning: No conversion function registered for layer: Merge yet.
Converting bidirectional_1/while_1/Merge as custom op: Merge
Warning: No conversion function registered for layer: NextIteration yet.
Converting bidirectional_1/while_1/NextIteration as custom op: NextIteration
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/Enter as custom op: Enter
Warning: No conversion function registered for layer: Switch yet.
Converting bidirectional_1/while_1/Switch_1 as custom op: Switch
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/Enter_1 as custom op: Enter
Warning: No conversion function registered for layer: Less yet.
Converting bidirectional_1/while_1/Less as custom op: Less
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/Less/Enter as custom op: Enter
Warning: No conversion function registered for layer: Merge yet.
Converting bidirectional_1/while_1/Merge_2 as custom op: Merge
Warning: No conversion function registered for layer: NextIteration yet.
Converting bidirectional_1/while_1/NextIteration_2 as custom op: NextIteration
Warning: No conversion function registered for layer: TensorArrayWriteV3 yet.
Converting bidirectional_1/while_1/TensorArrayWrite/TensorArrayWriteV3 as custom op: TensorArrayWriteV3
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/MatMul_6/Enter as custom op: Enter
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/mul_4/Enter as custom op: Enter
Warning: No conversion function registered for layer: Fill yet.
Converting bidirectional_1/ones_like_3 as custom op: Fill
Warning: No conversion function registered for layer: Tile yet.
Converting bidirectional_1/Tile_2 as custom op: Tile
Warning: keepdims is ignored by the UFF Parser and defaults to True
Warning: No conversion function registered for layer: ZerosLike yet.
Converting bidirectional_1/zeros_like_1 as custom op: ZerosLike
Warning: No conversion function registered for layer: Switch yet.
Converting bidirectional_1/while_1/Switch_3 as custom op: Switch
Warning: No conversion function registered for layer: Merge yet.
Converting bidirectional_1/while_1/Merge_3 as custom op: Merge
Warning: No conversion function registered for layer: NextIteration yet.
Converting bidirectional_1/while_1/NextIteration_3 as custom op: NextIteration
Warning: No conversion function registered for layer: Enter yet.
Converting bidirectional_1/while_1/Enter_3 as custom op: Enter
Traceback (most recent call last):
File “/home/phongtsv/Myproject/crnn_small/uffconvert.py”, line 8, in
text=True)
File “/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/conversion_helpers.py”, line 225, in from_tensorflow
debug_mode=debug_mode)
File “/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py”, line 141, in convert_tf2uff_graph
uff_graph, input_replacements, debug_mode=debug_mode)
File “/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py”, line 126, in convert_tf2uff_node
op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes, debug_mode=debug_mode)
File “/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py”, line 94, in convert_layer
return cls.registry_[op](name, tf_node, inputs, uff_graph, **kwargs)
File “/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter_functions.py”, line 586, in convert_bias_add
kwargs[“tf_nodes”][biases_name])
File “/usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter.py”, line 169, in convert_tf2numpy_const_node
raise UffException(“Const node conversion requested, but node is not Const\n” + str(tf_node))
uff.model.exceptions.UffException: Const node conversion requested, but node is not Const
name: “bidirectional_1/while_1/BiasAdd_2/Enter”
op: “Enter”
input: “bidirectional_1/backward_lstm1/strided_slice_10”
attr {
key: “T”
value {
type: DT_FLOAT
}
}
attr {
key: “frame_name”
value {
s: “bidirectional_1/while_1/while_context”
}
}
attr {
key: “is_constant”
value {
b: true
}
}
attr {
key: “parallel_iterations”
value {
i: 32
}
}

Hi,

It looks like there are some training operation within your model.
Would you mind to export your model into onnx format and check it again?

Onnx model is native supported via the TensorRT.
You can run it with the following command directly:

/usr/src/tensorrt/bin/trtexec --onnx=[your/model/format]

Thanks.

1 Like

Thank so much 😍😍

Hi @PhongTN,

So how did you change your secondary_detector_config.txt config file?

Thanks!