Thank you for your answer!
I tried uninstalling with the launcher cleanup-tool but after reinstallation of Omniverse and A2F the TensorRT build still crashes at 4% when building the auto-emotion TensorRT engine for the A2F example with Mark v2.
For some reason the log file is not created in .local/share/ov/data/Kit/Audio2Face/2023.0/ but the following is printed in the console:
2024-04-08 12:09:36 [Warning] [omni.ext.plugin] [ext: omni.kit.window.modifier.titlebar-105.1.1+105] Built using kit version: 105. Current version: 105.1. It is considered compatible, but building with a newer version is recommended.
2024-04-08 12:09:36 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Content *********************
2024-04-08 12:09:36 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Content" from omni.kit.window.content_browser.extension uses onclick_fn
2024-04-08 12:09:36 [Warning] [carb] Acquiring non optional plugin interface which is not listed as dependency: [omni::fabric::IFabricUsd v0.2] (plugin: <default plugin>), by client: omni.graph.io.plugin. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client.
2024-04-08 12:09:36 [Warning] [omni.kit.widget.cache_indicator.cache_state_menu] Unable to detect Omniverse Cache Server. Consider installing it for better IO performance.
2024-04-08 12:09:36 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Console *********************
2024-04-08 12:09:36 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Console" from omni.kit.window.console.scripts.console_extension uses onclick_fn
2024-04-08 12:09:37 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Stage *********************
2024-04-08 12:09:37 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Stage" from omni.kit.window.stage.stage_extension uses onclick_fn
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Action Graph *********************
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Action Graph" from functools uses onclick_fn
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Examples *********************
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Examples" from omni.kit.browser.sample.extension uses onclick_fn
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Push Graph *********************
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Push Graph" from functools uses onclick_fn
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] ********************* MenuItemDescription Lazy Graph *********************
2024-04-08 12:09:39 [Warning] [omni.kit.menu.utils.scripts.builder_utils] Menu item "Lazy Graph" from functools uses onclick_fn
2024-04-08 12:09:40 [Warning] [omni.kit.profiler.window] remove _SpanInstance.__lt__ and use insort 'key' arg instead
2024-04-08 12:09:46 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2024-04-08 12:09:46 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
2024-04-08 12:10:08 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated action as SPACE (On Press) with omni.kit.widget.toolbar.omni.kit.widget.toolbar::toolbar::play!
2024-04-08 12:10:08 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::play.SPACE, error code: Duplicated action definition
2024-04-08 12:12:09 [Warning] [omni.audio2face.wizard.scripts.extension] Removing a hotkey: "P" Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P
2024-04-08 12:12:09 [Warning] [omni.audio2face.wizard.scripts.extension] Removing a hotkey: "R" Global [omni.kit.widget.toolbar] omni.kit.widget.toolbar::toolbar::scale.R
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated key:'P (On Press)'
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.audio2face.wizard-105.9.0] omni.audio2face.wizard-105.9.0::audio2face__player__play_pause.P
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P, error code: Duplicated key definition
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] duplicated key:'P (On Press)'
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] -- Global [omni.audio2face.wizard-105.9.0] omni.audio2face.wizard-105.9.0::audio2face__player__play_pause.P
2024-04-08 12:14:54 [Warning] [omni.kit.hotkeys.core.registry] [Hotkey] Cannot register Global [omni.kit.menu.utils] omni.kit.menu.edit::parent_prims.P, error code: Duplicated key definition
2024-04-08 12:14:55 [Warning] [omni.usd] Warning (secondary thread): in _ReportErrors at line 2890 of /buildAgent/work/ac88d7d902b57417/USD/pxr/usd/usd/stage.cpp -- Could not load sublayer @metrics:UnitsAdjust-160680d357780c2b3e46f33c2c8dc7d3.metricsAssembler@ of layer @/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.wizard/deps/audio2face-assets/mark/mesh/mark_fullface_model.usd@; skipping. (instantiating stage on stage @/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.wizard/assets/demo_fullface_mark.usda@ <0x7f213c00c170>)
2024-04-08 12:14:55 [Warning] [omni.usd]
2024-04-08 12:14:56 [Warning] [omni.fabric.plugin] No source has valid data array=0x3e7df6c0 usdValid=0 cpuValid=0
2024-04-08 12:16:40 [Warning] [omni.fabric.plugin] No source has valid data array=0x3e7e02b8 usdValid=0 cpuValid=0
2024-04-08 12:16:40 [Warning] [omni.audio2face.utility.scripts.utils] Unable to read TRT info: /home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/trt_info.json ; Details: [Errno 2] No such file or directory: '/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/trt_info.json'
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] Unable to build TensorRT network for: /home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] Async Task Failed: <Task finished name='Task-161' coro=<A2FCoreFullfaceInstance._init_a2e_processor_async() done, defined at /home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.core/omni/audio2face/core/scripts/instance_fullface.py:592> exception=RuntimeError("Error during running command: ['/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference_deps/deps/audio2face-inference-deps/bin/trtexec', '--onnx=/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx', '--saveEngine=/home/ux/.cache/ov/Kit/105.1/c677816d/audio2emotion-nets/a2e_v1.4.2/network_device0_fp32.trt', '--device=0', '--tacticSources=-CUDNN', '--explicitBatch']")>
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] Error: Error during running command: ['/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference_deps/deps/audio2face-inference-deps/bin/trtexec', '--onnx=/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx', '--saveEngine=/home/ux/.cache/ov/Kit/105.1/c677816d/audio2emotion-nets/a2e_v1.4.2/network_device0_fp32.trt', '--device=0', '--tacticSources=-CUDNN', '--explicitBatch']
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] Traceback (most recent call last):
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 134, in _done_callback
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] task.result()
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.core/omni/audio2face/core/scripts/instance_fullface.py", line 608, in _init_a2e_processor_async
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await self._a2e_processor.set_network(network_fpath, network_name)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion/omni/audio2face/emotion/scripts/processor.py", line 286, in set_network
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await self._build_and_load_network(network_path, network_name, self._trt_info)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion/omni/audio2face/emotion/scripts/processor.py", line 479, in _build_and_load_network
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await self._trt_builder.prepare_and_load_trt_network(
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 54, in prepare_and_load_trt_network
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await self._build_or_rebuild_and_load_trt_network(
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 77, in _build_or_rebuild_and_load_trt_network
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await self._build_and_load_trt_network(
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 87, in _build_and_load_trt_network
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await self._build_trt_network(input_network_fpath, trt_network_fpath, optimizer_params)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference/omni/audio2face/inference/scripts/trt_builder.py", line 116, in _build_trt_network
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await run_cmd(cmd, cmd_name="TRTEXEC", progress_fn=self._build_trt_network_progress)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/cmd_runner.py", line 76, in run_cmd_in_app
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] await run_cmd(cmd, cmd_name, progress)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/utils.py", line 55, in wrapper
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] return await func(*args, **kwargs)
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] File "/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.utility/omni/audio2face/utility/scripts/cmd_runner.py", line 62, in run_cmd
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] raise RuntimeError("Error during running command: {}".format(cmd))
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils] RuntimeError: Error during running command: ['/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.inference_deps/deps/audio2face-inference-deps/bin/trtexec', '--onnx=/home/ux/.local/share/ov/pkg/deps/321b626abba810c3f8d1dd4d247d2967/exts/omni.audio2face.emotion_deps/deps/audio2emotion-nets/nets/a2e_v1.4.2/network.onnx', '--saveEngine=/home/ux/.cache/ov/Kit/105.1/c677816d/audio2emotion-nets/a2e_v1.4.2/network_device0_fp32.trt', '--device=0', '--tacticSources=-CUDNN', '--explicitBatch']
2024-04-08 12:16:46 [Error] [omni.audio2face.utility.scripts.utils]