Tensorflow Serving on Xavier NX?

[patrick.waibel]

Hi,
I would like to use Tensorflow Serving inside a Docker container on the Xavier NX. (The NX is intended to be used as a TF model server for other devices via REST or grpc)
Since the jetson arch seems to be not supported yet, is there a workaround or will there be an official support in the future?
Or is there another recommended way for preparing the NX as model server (maybe via triton inference server)?
Thanks in advance!

[AastaLLL]

Hi,

Sorry that we don’t officially provide TensorFlow Serving for Jetson.
But you can build it from source on your own.

Please find the detailed steps here:

TensorFlow Serving on Jetson TX2

@dxyjob et al: See https://github.com/helmuthva/jetson/tree/master/workflow/deploy/tensorflow-serving-base/src for a Dockerfile and .bazelrc file showing how to get TensorFlow Serving running on Jetson edge devices. The image is based on https://github.com/helmuthva/jetson/tree/master/workflow/deploy/ml-base which provides prerequisites. See https://github.com/helmuthva/jetson for the bigger picture of this project.

Thanks.