TF-TRT demo for Jetson Nano (TF2)

Autonomous Machines Jetson & Embedded Systems Jetson Nano
,
1

Hi all,

I’ve made a small demo example that showcases TF-TRT optimization of a small toy model in Keras TF2. You can run it on JetPack 4.4 using three simple commands:

git clone https://github.com/pascal-hwky/tftrt-jetson.git
cd tftrt-jetson
./arm.sh

Please find the repository here. Currently, I’m still receiving a strange error in the optimization stage:

Assertion Error in fromSymbolic: 0 (x must be build-time constant)

Unfortunately, this prevents the conversion of this segment to a fully optimized TRT engine operation. If anybody has seen this error before and knows what’s causing it, I would greatly appreciate any help!

1 Like
3

Hi,

We are going to reproduce this issue.
Will share more information with you later.

Thanks.

4

Hi,

We can reproduce this in our environment.
Will let you know once we got a progress.

Thanks.

1 Like
6

HI @AastaLLL, have you made any progress on this issue?

Thanks.

7

Hi,

Sorry for keeping you waiting.

This issue is caused from the restrictions of dynamic shapes.
Based on the document below, you will need to specify the constant channel input:

This indicates that you will need to have at least 4 dimension (including batch) input tensor on NHWC format.
After checking, your sample can work correctly when expand the input tensor to the 2nd or 4th dimension.

    inputs = keras.Input(shape=(1,1,32,))
    ...

def my_input_fn():
  for _ in range(100):
    inp1 = np.random.normal(size=(8,1,1,32)).astype(np.float32)
    yield inp1,

Or

    inputs = keras.Input(shape=(32,1,1,))
    ...
def my_input_fn():
  for _ in range(100):
    inp1 = np.random.normal(size=(8,32,1,1)).astype(np.float32)
    yield inp1,

Thanks.

8

Hi @AastaLLL, thanks!

In order to have a network segment eligible for TensorRT optimization, I additionally had to change the network architecture to use convolutional layers.

I have updated the repository.