I checked the following other posts for this error, but they recommend an update.
Failed to Build TensorRT Engine 1
Failed to Build TensorRT Engine 2
Failed to Build TensorRT Engine 3
Failed to Build TensorRT Engine 4
Failed to Build TensorRT Engine 5
Even on the newest version, this error appears.
The Audio2Face version is 2023.2.0, the error appears in
demo_fullface_mark.usda on an Nvidia RTX A4000 (16 GB), Ubuntu 23.04, Linux 6.2.0-39-generic, Python 3.11.4
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.audio2face.utility.scripts.utils] Waiting for A2E Network: TASK DELETED
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated key:'P (On Press)'
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.audio2face.wizard-105.9.0] omni.audio2face.wizard-105.9.0::audio2face__player__play_pause.P
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P, error code: Duplicated key definition
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated key:'P (On Press)'
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.audio2face.wizard-105.9.0] omni.audio2face.wizard-105.9.0::audio2face__player__play_pause.P
2024-02-25 21:39:12 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P, error code: Duplicated key definition
2024-02-25 21:39:12 [Warning] [omni.usd] Warning (secondary thread): in _ReportErrors at line 2890 of /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp -- Could not load sublayer @metrics:UnitsAdjust-160680d357780c2b3e46f33c2c8dc7d3.metricsAssembler@ of layer @/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.wizard/deps/audio2face-assets/mark/mesh/mark_fullface_model.usd@; skipping. (instantiating stage on stage @/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.wizard/assets/demo_fullface_mark.usda@ <0x7fd314064920>)
2024-02-25 21:39:12 [Warning] [omni.usd]
2024-02-25 21:43:42 [Warning] [omni.audio2face.utility.scripts.utils] Unable to read TRT info: /home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/trt_info.json ; Details: [Errno 2] No such file or directory: '/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/trt_info.json'
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] Unable to build TensorRT network for: /home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] Async Task Failed: <Task finished name='Task-901' coro=<A2FCoreFullfaceInstance._init_a2e_processor_async() done, defined at /home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.core/omni/audio2face/core/scripts/instance_fullface.py:592> exception=RuntimeError("Error during running command: ['/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference_deps/deps/audio2face-inference-deps/bin/trtexec', '--onnx=/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx', '--saveEngine=/home/base_admin/.cache/ov/Kit/105.1/c677816d/audio2emotion-nets/a2e_v1.4.2/network_device0_fp32.trt', '--device=0', '--tacticSources=-CUDNN', '--explicitBatch']")>
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] Error: Error during running command: ['/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference_deps/deps/audio2face-inference-deps/bin/trtexec', '--onnx=/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx', '--saveEngine=/home/base_admin/.cache/ov/Kit/105.1/c677816d/audio2emotion-nets/a2e_v1.4.2/network_device0_fp32.trt', '--device=0', '--tacticSources=-CUDNN', '--explicitBatch']
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] Traceback (most recent call last):
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 134, in _done_callback
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] task.result()
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.core/omni/audio2face/core/scripts/instance_fullface.py", line 608, in _init_a2e_processor_async
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await self._a2e_processor.set_network(network_fpath, network_name)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion/omni/audio2face/emotion/scripts/processor.py", line 286, in set_network
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await self._build_and_load_network(network_path, network_name, self._trt_info)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion/omni/audio2face/emotion/scripts/processor.py", line 479, in _build_and_load_network
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await self._trt_builder.prepare_and_load_trt_network(
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 54, in prepare_and_load_trt_network
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await self._build_or_rebuild_and_load_trt_network(
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 77, in _build_or_rebuild_and_load_trt_network
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await self._build_and_load_trt_network(
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 87, in _build_and_load_trt_network
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await self._build_trt_network(input_network_fpath, trt_network_fpath, optimizer_params)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 116, in _build_trt_network
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await run_cmd(cmd, cmd_name="TRTEXEC", progress_fn=self._build_trt_network_progress)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/cmd_runner.py", line 76, in run_cmd_in_app
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] await run_cmd(cmd, cmd_name, progress)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] File "/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/cmd_runner.py", line 62, in run_cmd
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] raise RuntimeError("Error during running command: {}".format(cmd))
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils] RuntimeError: Error during running command: ['/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference_deps/deps/audio2face-inference-deps/bin/trtexec', '--onnx=/home/base_admin/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx', '--saveEngine=/home/base_admin/.cache/ov/Kit/105.1/c677816d/audio2emotion-nets/a2e_v1.4.2/network_device0_fp32.trt', '--device=0', '--tacticSources=-CUDNN', '--explicitBatch']
2024-02-25 21:43:49 [Error] [omni.audio2face.utility.scripts.utils]