Pruning of Customized trained ONNX model via TAO

Please provide the following information when requesting support.

• Hardware (JETSON NANO)
• Network Type (ResNET 18/Classification/etc)

Dear team.
I have re-trained a model on jetson nano via transfer learning on my customized data, using resnet 18 and PyTorch as per the instruction given on GitHub - dusty-nv/jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson..

The accuracy is almost 75 to 80% of my trained model.
Now I want to use the TAO toolkit for pruning purpose (Only pruning, not optimization or any other thing).
Please provide a detailed video or complete guide on how to prune my customized developed ONNX model via the TAO tool kit.
If there is any other info needed please let me know
Thanks.

Is it a classification onnx model? If yes, you can use TAO BYOM feature to convert it to .tltb model. Then train with very low learning rate to get the .tlt model and further prune it.

Thanks for the reply,
I gave it a try and the following error is happening.

ERROR: Command errored out with exit status 1:
     command: 'C:\Users\wasee\miniconda3\envs\byom_dev\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rjprvti6\\nvidia-tensorflow_23ae8ad94a254207ab6c8c5c4608a653\\setup.py'"'"'; __file__='"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rjprvti6\\nvidia-tensorflow_23ae8ad94a254207ab6c8c5c4608a653\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\wasee\AppData\Local\Temp\pip-pip-egg-info-o3xo5cxa'
         cwd: C:\Users\wasee\AppData\Local\Temp\pip-install-rjprvti6\nvidia-tensorflow_23ae8ad94a254207ab6c8c5c4608a653\
    Complete output (18 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\wasee\AppData\Local\Temp\pip-install-rjprvti6\nvidia-tensorflow_23ae8ad94a254207ab6c8c5c4608a653\setup.py", line 150, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.

    This package can be installed as:
    ```
    $ pip install nvidia-pyindex
    $ pip install nvidia-tensorflow
    ```

    Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
    ###########################################################################################

    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/c7/b8/b96bd09cc7c9cf9dbd8b82313c4f9860fe615b3ce7f3f6efc962d166178b/nvidia-tensorflow-0.0.1.dev5.tar.gz#sha256=65f44f1b583b950925a3617a75fd299c29975f1f14327bb2d946cb52e84baef5 (from https://pypi.org/simple/nvidia-tensorflow/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Downloading nvidia-tensorflow-0.0.1.dev4.tar.gz (3.8 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\wasee\miniconda3\envs\byom_dev\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rjprvti6\\nvidia-tensorflow_14cae1d2da3345a18768265786b44465\\setup.py'"'"'; __file__='"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rjprvti6\\nvidia-tensorflow_14cae1d2da3345a18768265786b44465\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\wasee\AppData\Local\Temp\pip-pip-egg-info-4uu05pme'
         cwd: C:\Users\wasee\AppData\Local\Temp\pip-install-rjprvti6\nvidia-tensorflow_14cae1d2da3345a18768265786b44465\
    Complete output (17 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\wasee\AppData\Local\Temp\pip-install-rjprvti6\nvidia-tensorflow_14cae1d2da3345a18768265786b44465\setup.py", line 150, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.

    This package can be installed as:
    ```
    $ pip install nvidia-pyindex
    $ pip install nvidia-tensorflow
    ```

    Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
    ###########################################################################################
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/97/39/b1aa7fe84e2ad1f183f5fc851bea78bb46a73ebb9a91ffed0878f656f2f6/nvidia-tensorflow-0.0.1.dev4.tar.gz#sha256=1be0b140aadedcd7a05f736566587bb1aacc6baa1797d7fda7d8f23e59f35142 (from https://pypi.org/simple/nvidia-tensorflow/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement nvidia-tensorflow (from versions: 0.0.1.dev4, 0.0.1.dev5)
ERROR: No matching distribution found for nvidia-tensorflow

And please provide the command to run pruning after conversion.

What command you have run to meet above error?

BTW, it is expected to run TAO training/pruning in dgpu devices instead of Nano.

Dear @Morganh

I have tried on both laptop and the jetson error is the same.
I am following GitHub - NVIDIA-AI-IOT/tao_byom_examples: Examples of converting different open-source deep learning models to TAO compatible format through TAO BYOM package.

on this command:
pip3 install nvidia-tensorflow
it shows the above-mentioned error.

Following are my all step by step commands

(base) PS C:\WINDOWS\system32> conda activate byom_dev

(byom_dev) PS C:\WINDOWS\system32> pip3 install -r requirements.txt --no-deps

Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: certifi==2021.5.30 in c:\users\wasee\miniconda3\envs\byom_dev\lib\site-packages (from -r requirements.txt (line 1)) (2021.5.30)
Requirement already satisfied: wincertstore==0.2 in c:\users\wasee\miniconda3\envs\byom_dev\lib\site-packages (from -r requirements.txt (line 2)) (0.2)

(byom_dev) PS C:\WINDOWS\system32> pip3 install nvidia-pyindex
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: nvidia-pyindex in c:\users\wasee\miniconda3\envs\byom_dev\lib\site-packages (1.0.9)

(byom_dev) PS C:\WINDOWS\system32> pip3 install nvidia-tensorflow
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting nvidia-tensorflow
  Downloading nvidia-tensorflow-0.0.1.dev5.tar.gz (7.9 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\wasee\miniconda3\envs\byom_dev\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rekg24ab\\nvidia-tensorflow_b9d2c40b03d044819554dee20bde33d3\\setup.py'"'"'; __file__='"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rekg24ab\\nvidia-tensorflow_b9d2c40b03d044819554dee20bde33d3\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\wasee\AppData\Local\Temp\pip-pip-egg-info-bhzc0mpt'
         cwd: C:\Users\wasee\AppData\Local\Temp\pip-install-rekg24ab\nvidia-tensorflow_b9d2c40b03d044819554dee20bde33d3\
    Complete output (18 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\wasee\AppData\Local\Temp\pip-install-rekg24ab\nvidia-tensorflow_b9d2c40b03d044819554dee20bde33d3\setup.py", line 150, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.

    This package can be installed as:
    ```
    $ pip install nvidia-pyindex
    $ pip install nvidia-tensorflow
    ```

    Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
    ###########################################################################################

    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/c7/b8/b96bd09cc7c9cf9dbd8b82313c4f9860fe615b3ce7f3f6efc962d166178b/nvidia-tensorflow-0.0.1.dev5.tar.gz#sha256=65f44f1b583b950925a3617a75fd299c29975f1f14327bb2d946cb52e84baef5 (from https://pypi.org/simple/nvidia-tensorflow/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Downloading nvidia-tensorflow-0.0.1.dev4.tar.gz (3.8 kB)
    ERROR: Command errored out with exit status 1:
     command: 'C:\Users\wasee\miniconda3\envs\byom_dev\python.exe' -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rekg24ab\\nvidia-tensorflow_3a0b6a4b59cf4af4bb3b028e3904613a\\setup.py'"'"'; __file__='"'"'C:\\Users\\wasee\\AppData\\Local\\Temp\\pip-install-rekg24ab\\nvidia-tensorflow_3a0b6a4b59cf4af4bb3b028e3904613a\\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\wasee\AppData\Local\Temp\pip-pip-egg-info-vnzymsa5'
         cwd: C:\Users\wasee\AppData\Local\Temp\pip-install-rekg24ab\nvidia-tensorflow_3a0b6a4b59cf4af4bb3b028e3904613a\
    Complete output (17 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\wasee\AppData\Local\Temp\pip-install-rekg24ab\nvidia-tensorflow_3a0b6a4b59cf4af4bb3b028e3904613a\setup.py", line 150, in <module>
        raise RuntimeError(open("ERROR.txt", "r").read())
    RuntimeError:
    ###########################################################################################
    The package you are trying to install is only a placeholder project on PyPI.org repository.
    This package is hosted on NVIDIA Python Package Index.

    This package can be installed as:
    ```
    $ pip install nvidia-pyindex
    $ pip install nvidia-tensorflow
    ```

    Please refer to NVIDIA instructions: https://github.com/NVIDIA/tensorflow#install.
    ###########################################################################################
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/97/39/b1aa7fe84e2ad1f183f5fc851bea78bb46a73ebb9a91ffed0878f656f2f6/nvidia-tensorflow-0.0.1.dev4.tar.gz#sha256=1be0b140aadedcd7a05f736566587bb1aacc6baa1797d7fda7d8f23e59f35142 (from https://pypi.org/simple/nvidia-tensorflow/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement nvidia-tensorflow (from versions: 0.0.1.dev4, 0.0.1.dev5)
ERROR: No matching distribution found for nvidia-tensorflow
(byom_dev) PS C:\WINDOWS\system32> pip3 install nvidia-tao-byom
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting nvidia-tao-byom
  Downloading nvidia_tao_byom-0.0.9-py3-none-any.whl (183 kB)
     |████████████████████████████████| 183 kB 312 kB/s
Collecting pillow<9.0.0,>=8.3.2
  Downloading Pillow-8.4.0-cp36-cp36m-win_amd64.whl (3.2 MB)
     |████████████████████████████████| 3.2 MB 501 kB/s
Collecting keras==2.2.4
  Downloading Keras-2.2.4-py2.py3-none-any.whl (312 kB)
     |████████████████████████████████| 312 kB 1.6 MB/s
Collecting h5py<3.0.0
  Downloading h5py-2.10.0-cp36-cp36m-win_amd64.whl (2.4 MB)
     |████████████████████████████████| 2.4 MB 2.2 MB/s
Collecting netron<=6.0.2,>=6.0.0
  Downloading netron-6.0.2-py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 2.2 MB/s
Collecting skl2onnx==1.10.4
  Downloading skl2onnx-1.10.4-py2.py3-none-any.whl (273 kB)
     |████████████████████████████████| 273 kB 1.6 MB/s
Collecting nvidia-tao-byom
  Downloading nvidia_tao_byom-0.0.7-py3-none-any.whl (201 kB)
     |████████████████████████████████| 201 kB 3.2 MB/s
Collecting numpy==1.18.5
  Downloading numpy-1.18.5-cp36-cp36m-win_amd64.whl (12.7 MB)
     |████████████████████████████████| 12.7 MB 819 kB/s
ERROR: Cannot install nvidia-tao-byom==0.0.7 and nvidia-tao-byom==0.0.9 because these package versions have conflicting dependencies.

The conflict is caused by:
    nvidia-tao-byom 0.0.9 depends on nvidia-eff==0.5.6
    nvidia-tao-byom 0.0.7 depends on nvidia-eff==0.5.6

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
(byom_dev) PS C:\WINDOWS\system32>

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

Could you use Ubuntu instead or wsl ?
I am asking this according to GitHub - NVIDIA/tensorflow: An Open Source Machine Learning Framework for Everyone

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.