Tao-Toolkit AWS SageMaker Deployment

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Please provide the following information when requesting support.

• Hardware (T4/V100/Xavier/Nano/etc): T4 AWS SageMaker Studio
• Network Type (Detectnet_v2/Faster_rcnn/Yolo_v4/LPRnet/Mask_rcnn/Classification/etc): TrafficCamNet, LPD, LPR, VehicleMake, VehicleType

Hi NV team,

Following in from the successful implementation of TAO Toolkit in SageMaker Studio, helped by @Morganh here - NVIDIA-TAO-Deploy -pycocotools-fix issue - Python Wheels TAO implementation porting from Collab to AWS SageMaker Studio - #10 by riyang1

I aim to deploy the model to AWS SageMaker and seek support in developing a REST API endpoint for operational use. Is there any guidance/examples to help with this?

3

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

For AWS EKS, please refer to AWS EKS setup. Setup - NVIDIA Docs

For AWS SageMaker, please check if above guide works. Also, you can try to use bare-metal-setup Setup - NVIDIA Docs. The steps are as below.

Steps:

local-morganh@ipp1-1080:~/getting_started_v5.0.0/setup/quickstart_api_bare_metal$ history
    1  . .bashrc
    2  nvidia-smi
    3  cat /etc/lsb-release
    4  nvidia-smi
    5  sudo apt install nvidia-driver-535
    6  nvidia-smi
    7  sudo reboot
    8  . .bashrc
    9  nvidia-smi
   10  nvidia-smi -L
   11  wget --content-disposition https://ngc.nvidia.com/downloads/ngccli_linux.zip && unzip ngccli_linux.zip && chmod u+x ngc-cli/ngc
   12  ls
   13  find ngc-cli/ -type f -exec md5sum {} + | LC_ALL=C sort | md5sum -c ngc-cli.md5
   14  echo "export PATH=\"\$PATH:$(pwd)/ngc-cli\"" >> ~/.bash_profile && source ~/.bash_profile
   15  ln -s $(pwd)/ngc-cli/ngc ./ngc
   16  ngc config set
   17  wget --content-disposition https://api.ngc.nvidia.com/v2/resources/nvidia/tao/tao-getting-started/versions/5.0.0/zip -O getting_started_v5.0.0.zip
   18  unzip -u getting_started_v5.0.0.zip -d ./getting_started_v5.0.0 && rm -rf getting_started_v5.0.0.zip && cd ./getting_started_v5.0.0
   19  cd setup/quickstart_api_bare_metal/
   20  vim gpu-operator-values.yml
   21  cat gpu-operator-values.yml
   22  vim hosts
   23  sudo nano /etc/sudoers
   24  vim tao-toolkit-api-ansible-values.yml
   25  cat gpu-operator-values.yml
$ cat gpu-operator-values.yml
enable_mig: no
mig_profile: all-disabled
mig_strategy: single
nvidia_driver_version: "525.85.12"
install_driver: false #true

   26  bash setup.sh check-inventory
   27  bash setup.sh install
   28  kubectl get pods
   29  history

After installation,

local-morganh@ipp1-1080:~/getting_started_v5.0.0/setup/quickstart_api_bare_metal$ kubectl get pods
NAME                                              READY   STATUS    RESTARTS   AGE
ingress-nginx-controller-55794ccb68-sldjn         1/1     Running   0          2m48s
nfs-subdir-external-provisioner-cb9798d4-tp5hx    1/1     Running   0          2m38s
nvidia-smi-ipp1-1080                              1/1     Running   0          2m32s
tao-toolkit-api-app-pod-55c5d88d86-79vpj          1/1     Running   0          2m29s
tao-toolkit-api-jupyterlab-pod-5db94dd6cc-7w4lg   1/1     Running   0          2m29s
tao-toolkit-api-workflow-pod-55db5b9bf9-747jf     1/1     Running   0          2m29s