2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205486] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205486] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 15:36:07 [0ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 15:36:07 [1ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 15:36:07 [1ms] [Info] [omni.server] [1205498] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 15:36:07 [2ms] [Info] [omni.server] [1205498] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-16 15:36:07 [2ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 15:36:07 [2ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 15:36:07 [2ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 15:36:07 [3ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 15:36:07 [3ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x2381530 for './omni-lft-service/omni-lft-service'. 2022-12-16 15:36:07 [3ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 1205498. 2022-12-16 15:36:07 [3ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-16 15:36:07 [3ms] [Info] [carb] [1205498] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-16 15:36:07 [3ms] [Info] [carb.launcher.plugin] [1205510] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1205511 from the parent 1205498. 2022-12-16 15:36:07 [3ms] [Info] [carb.launcher.plugin] [1205510] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1205511. 2022-12-16 15:36:07 [4ms] [Info] [carb.launcher.plugin] [1205510] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1205511. 2022-12-16 15:36:07 [4ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1205511 from the parent 1205498. 2022-12-16 15:36:07 [4ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1205511 {process = 0x2381530} 2022-12-16 15:36:07 [4ms] [Info] [omni.core] [1205498] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-16 15:36:07 [4ms] [Info] [omni.core] [1205498] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-16 15:36:07 [4ms] [Info] [carb.launcher.plugin] [1205513] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1205511. 2022-12-16 15:36:07 [4ms] [Info] [carb.launcher.plugin] [1205512] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1205511. 2022-12-16 15:36:07 [260ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-16 15:36:07 [264ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 10:36:07,348 - system_log - INFO - reconnecting ... 2022-12-16 15:36:07 [272ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 10:36:07,355 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 15:36:08 [1,274ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 10:36:08,357 - system_log - INFO - reconnecting ... 2022-12-16 15:36:08 [1,275ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 10:36:08,358 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 15:36:09 [2,004ms] [Info] [omni.core] [1205502] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():308: discovery client: starting up 2022-12-16 15:36:09 [2,004ms] [Info] [omni.core] [1205502] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():351: discovery client: connecting to ws://localhost:3333 2022-12-16 15:36:09 [2,004ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 15:36:09 [2,005ms] [Info] [omni.core] [1205502] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():386: discovery client: registering 'external' with transport 'connlib' and transport parameters '{ "url": "ws://127.0.0.1:3009" }' 2022-12-16 15:36:09 [2,005ms] [Info] [omni.core] [1205498] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:getOmniResolverInstance():87: Successfully created OmniResolver instance of type(omniverse_resolver_composite) 2022-12-16 15:36:09 [2,005ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 15:36:09 [2,056ms] [Warning] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IObjects v1.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 15:36:09 [2,056ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 15:36:09 [2,081ms] [Warning] [carb] [1205498] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IMetaUpgrade v2.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 15:36:09 [2,081ms] [Info] [carb] [1205498] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 15:36:09 [2,081ms] [Info] [omni.meta-upgrade] [1205498] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:getCurrentDbVersion():574: Database version record not found, key: service://metaDatabaseVersion 2022-12-16 15:36:09 [2,081ms] [Info] [omni.meta-upgrade] [1205498] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:upgrade():2882: Fresh database, nothing to upgrade 2022-12-16 15:36:09 [2,081ms] [Info] [omni.meta-upgrade] [1205498] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:setVersion():552: Meta database upgraded 2022-12-16 15:36:09 [2,082ms] [Info] [omni.core] [1205498] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:initialize():585: Version: 114.4-84b6cd59 2022-12-16 15:36:09 [2,083ms] [Info] [omni.core] [1205498] ../../../source/plugins/omni.core/impl/Mounts.cpp:createMount():97: Successfully created resolver of type: omniverse_resolver_s3 2022-12-16 15:36:09 [2,085ms] [Info] [omni.core] [1205498] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:processPendingDeltaManagerRecords():1515: Checking for any pending delta manager records. 2022-12-16 15:36:09 [2,085ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():123: waiting for Core to startup ... 2022-12-16 15:36:09 [2,085ms] [Info] [omni.core] [1205505] ../../../source/plugins/omni.core/impl/requests/DataStoreBase.cpp:runContentRemove():351: Started content remove task 2022-12-16 15:36:09 [2,086ms] [Info] [omni.core] [1205503] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:pingTask():599: Starting delta applier process. 2022-12-16 15:36:09 [2,086ms] [Info] [carb.launcher.plugin] [1205503] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x7f8c28003650 for './omni.delta-applier.app'. 2022-12-16 15:36:09 [2,086ms] [Info] [carb.launcher.plugin] [1205503] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni.delta-applier.app' from process 1205498. 2022-12-16 15:36:09 [2,087ms] [Info] [carb.launcher.plugin] [1205510] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1205562 from the parent 1205498. 2022-12-16 15:36:09 [2,087ms] [Info] [carb.launcher.plugin] [1205510] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1205562. 2022-12-16 15:36:09 [2,088ms] [Info] [carb.launcher.plugin] [1205510] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1205562. 2022-12-16 15:36:09 [2,088ms] [Info] [carb.launcher.plugin] [1205503] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1205562 from the parent 1205498. 2022-12-16 15:36:09 [2,088ms] [Info] [carb.launcher.plugin] [1205503] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1205562 {process = 0x7f8c28003650} 2022-12-16 15:36:09 [2,088ms] [Info] [carb.launcher.plugin] [1205563] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1205562. 2022-12-16 15:36:09 [2,088ms] [Info] [carb.launcher.plugin] [1205564] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1205562. 2022-12-16 15:36:09 [2,276ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 10:36:09,360 - system_log - INFO - reconnecting ... 2022-12-16 15:36:09 [2,277ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 10:36:09,361 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 15:36:09 [2,290ms] [Info] [omni.core] [1205502] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:pingTask():629: Successfully started delta applier process. 2022-12-16 15:36:10 [3,279ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 10:36:10,362 - system_log - INFO - reconnecting ... 2022-12-16 15:36:10 [3,280ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 10:36:10,363 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 15:36:11 [4,085ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():126: waiting for Core to startup (done) 2022-12-16 15:36:11 [4,087ms] [Error] [carb] [1205498] ../../../source/framework/FileSystem.cpp:move():964: File move failed: '/home/baxelrod/.nvidia-omniverse/logs/Nucleus/Access.log' -> '/home/baxelrod/.nvidia-omniverse/logs/Nucleus/Access_OLD.log' (No such file or directory) 2022-12-16 15:36:11 [4,087ms] [Error] [omni.server] [1205498] ../../../source/plugins/omni.server/apilogging/deprecated/ApiLogging.cpp:resetApiLogFile():210: Failed to rename access log file '/home/baxelrod/.nvidia-omniverse/logs/Nucleus/Access.log' to '/home/baxelrod/.nvidia-omniverse/logs/Nucleus/Access_OLD.log' 2022-12-16 15:36:11 [4,087ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():137: Omniverse C++ ApiServer server started. 2022-12-16 15:36:11 [4,087ms] [Info] [omni.signalhandler] [1205498] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():150: Installed signal handler for SIGINT 2022-12-16 15:36:11 [4,087ms] [Info] [omni.signalhandler] [1205498] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():159: Installed signal handler for SIGTERM 2022-12-16 15:36:11 [4,087ms] [Info] [omni.signalhandler] [1205498] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():168: Ignoring SIGPIPE 2022-12-16 15:36:11 [4,281ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 10:36:11,365 - system_log - INFO - reconnecting ... 2022-12-16 15:36:11 [4,283ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 10:36:11,367 - system_log - INFO - connected 2022-12-16 15:36:11 [4,283ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Server started 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.server.app] [1205498] ../../../source/apps/omni.server/main.cpp:main():106: Got signal to exit the application. 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:beginShutdown():391: OmniverseServer shutdown process starting. 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:beginShutdown():392: Waiting for any outstanding transactions to complete. 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:beginShutdown():396: Releasing WebSocket listen ports 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.server] [1212130] ../../../source/plugins/omni.server/ApiConnection.cpp:shutdownCore():70: Shutdown Core 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.core] [1205553] ../../../source/plugins/omni.core/impl/Scheduler.cpp:scheduler():122: Scheduling task is shutted down... 2022-12-16 16:06:12 [1,805,195ms] [Info] [omni.core] [1205504] ../../../source/plugins/omni.core/impl/requests/DataStoreBase.cpp:runContentRemove():422: Finished content remove task 2022-12-16 16:06:12 [1,805,195ms] [Error] [omni.core] [1205513] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Shutting down server...wait 2022-12-16 16:06:12 [1,805,196ms] [Warning] [carb] [1212130] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IObjects v1.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.core] [1212130] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:operator()():849: Content resolver statistics: Total: flush: 0 write: 0 read: 0 stat: 0 remove: 0 list: 0|Temporal: flush: 0 write: 0 read: 0 stat: 0 remove: 0 list: 0|Persistent: flush: 0 write: 0 read: 0 stat: 0 remove: 0 list: 0 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.server.app] [1205498] ../../../source/apps/omni.server/main.cpp:main():120: Releasing the OmniverseServer instance. 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():310: OmniverseServerImpl destructor. 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():312: Releasing ClientConnectionManager 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():315: Releasing ServiceConnectionManager 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():318: Releasing LftConnectionManager 2022-12-16 16:06:12 [1,805,196ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():321: Releasing ConnectionManager 2022-12-16 16:06:12 [1,805,196ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:kill():327: killing the child process 1205562 {exitCode = 137, flags = 0x00000002, timeout = 5000} 2022-12-16 16:06:12 [1,805,197ms] [Info] [carb.launcher.plugin] [1205564] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:06:12 [1,805,197ms] [Info] [carb.launcher.plugin] [1205563] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:06:12 [1,805,197ms] [Info] [carb.launcher.plugin] [1205564] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1205562, bytes = 0} 2022-12-16 16:06:12 [1,805,197ms] [Info] [carb.launcher.plugin] [1205563] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1205562, bytes = 0} 2022-12-16 16:06:12 [1,805,197ms] [Info] [carb.launcher.plugin] [1205564] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1205562. 2022-12-16 16:06:12 [1,805,197ms] [Info] [carb.launcher.plugin] [1205563] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1205562. 2022-12-16 16:06:12 [1,805,199ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():254: waiting for the process 0x7f8c28003650 (1205562) to exit {timeout = 18446744073709551615} 2022-12-16 16:06:12 [1,805,199ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():259: the process has already exited with code 137. 2022-12-16 16:06:12 [1,805,199ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():247: destroying the process object 0x7f8c28003650 {pid = 1205562} 2022-12-16 16:06:12 [1,805,199ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():249: done destroying the process object 0x7f8c28003650 {pid = 1205562} 2022-12-16 16:06:12 [1,805,199ms] [Info] [omni.core] [1205504] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():457: discovery client: exiting 2022-12-16 16:06:12 [1,805,199ms] [Info] [omni.core] [1205498] ../../../source/plugins/common/ChildProcess.cpp:kill():125: Killing child server process ./omni-lft-service/omni-lft-service 2022-12-16 16:06:12 [1,805,199ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:kill():327: killing the child process 1205511 {exitCode = 137, flags = 0x00000003, timeout = 5000} 2022-12-16 16:06:12 [1,805,202ms] [Info] [carb.launcher.plugin] [1205513] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:06:12 [1,805,202ms] [Info] [carb.launcher.plugin] [1205512] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:06:12 [1,805,202ms] [Info] [carb.launcher.plugin] [1205513] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1205511, bytes = 0} 2022-12-16 16:06:12 [1,805,202ms] [Info] [carb.launcher.plugin] [1205512] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1205511, bytes = 0} 2022-12-16 16:06:12 [1,805,202ms] [Info] [carb.launcher.plugin] [1205513] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1205511. 2022-12-16 16:06:12 [1,805,202ms] [Info] [carb.launcher.plugin] [1205512] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1205511. 2022-12-16 16:06:12 [1,805,203ms] [Info] [omni.core] [1205498] ../../../source/plugins/common/ChildProcess.cpp:kill():127: Waiting for child server process ./omni-lft-service/omni-lft-service to exit 2022-12-16 16:06:12 [1,805,203ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():254: waiting for the process 0x2381530 (1205511) to exit {timeout = 18446744073709551615} 2022-12-16 16:06:12 [1,805,203ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():259: the process has already exited with code 137. 2022-12-16 16:06:12 [1,805,203ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():247: destroying the process object 0x2381530 {pid = 1205511} 2022-12-16 16:06:12 [1,805,203ms] [Info] [carb.launcher.plugin] [1205498] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():249: done destroying the process object 0x2381530 {pid = 1205511} 2022-12-16 16:06:12 [1,805,203ms] [Info] [omni.server] [1205498] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():327: Destructor complete. 2022-12-16 16:06:12 [0ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:06:12 [0ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:06:12 [0ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 16:06:12 [0ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:06:12 [0ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:06:12 [0ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:06:12 [1ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:06:12 [1ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 16:06:12 [1ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 16:06:12 [2ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:06:12 [2ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:06:12 [2ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:06:12 [2ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:06:12 [2ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:06:12 [2ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:06:12 [3ms] [Info] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:06:12 [3ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:06:12 [3ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:06:12 [3ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:06:12 [3ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:06:12 [3ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 16:06:12 [3ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 16:06:12 [4ms] [Info] [omni.server] [1212133] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 16:06:12 [4ms] [Info] [omni.server] [1212133] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-16 16:06:12 [4ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:06:12 [4ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:06:12 [4ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:06:12 [5ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:06:12 [5ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0xcfe6c0 for './omni-lft-service/omni-lft-service'. 2022-12-16 16:06:12 [5ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 1212133. 2022-12-16 16:06:12 [5ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-16 16:06:12 [5ms] [Info] [carb] [1212133] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-16 16:06:12 [5ms] [Info] [carb.launcher.plugin] [1212146] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1212147 from the parent 1212133. 2022-12-16 16:06:12 [5ms] [Info] [carb.launcher.plugin] [1212146] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1212147. 2022-12-16 16:06:12 [6ms] [Info] [carb.launcher.plugin] [1212146] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1212147. 2022-12-16 16:06:12 [6ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1212147 from the parent 1212133. 2022-12-16 16:06:12 [6ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1212147 {process = 0xcfe6c0} 2022-12-16 16:06:12 [6ms] [Info] [carb.launcher.plugin] [1212148] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1212147. 2022-12-16 16:06:12 [6ms] [Info] [omni.core] [1212133] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-16 16:06:12 [6ms] [Info] [omni.core] [1212133] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-16 16:06:12 [6ms] [Info] [carb.launcher.plugin] [1212149] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1212147. 2022-12-16 16:06:12 [262ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-16 16:06:12 [266ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:06:12,661 - system_log - INFO - reconnecting ... 2022-12-16 16:06:12 [274ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:06:12,668 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:06:13 [1,275ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:06:13,669 - system_log - INFO - reconnecting ... 2022-12-16 16:06:13 [1,275ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:06:13,670 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:06:14 [2,006ms] [Info] [omni.core] [1212138] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():308: discovery client: starting up 2022-12-16 16:06:14 [2,006ms] [Info] [omni.core] [1212138] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():351: discovery client: connecting to ws://localhost:3333 2022-12-16 16:06:14 [2,006ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:06:14 [2,007ms] [Info] [omni.core] [1212138] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():386: discovery client: registering 'external' with transport 'connlib' and transport parameters '{ "url": "ws://127.0.0.1:3009" }' 2022-12-16 16:06:14 [2,008ms] [Info] [omni.core] [1212133] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:getOmniResolverInstance():87: Successfully created OmniResolver instance of type(omniverse_resolver_composite) 2022-12-16 16:06:14 [2,008ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:06:14 [2,066ms] [Warning] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IObjects v1.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 16:06:14 [2,066ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:06:14 [2,085ms] [Warning] [carb] [1212133] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IMetaUpgrade v2.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 16:06:14 [2,085ms] [Info] [carb] [1212133] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:upgrade():2915: Meta database doesn't require upgrade 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1463: Fixing folder children counters .... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1471: Processed 1 / 6 records 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1497: Checking for orphaned folder children counter records .... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1508: Processed 1 / 6 records 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1532: Fixing folder children counters done 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1304: Fixing content refcounts .... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1311: Collecting content -> refcount mapping ... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1315: Processed 1 / 6 records 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1350: Collecting content -> refcount mapping done 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1351: Setting refcount ... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1380: Setting refcount done 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1382: Checking content table for unused content 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1550: Repairing versioning data... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1563: Reading meta records... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1671: Checking branch lists... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1716: Fixed branch lists count: 0 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1718: Check for orphaned branch lists... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1741: Checking version reference counters... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1796: Fixed version reference counters: 0 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1797: Deleted orphaned version reference counters 0 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1799: Checking checkpoint messages... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1860: Verifying checkpoints... 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():2090: Deleted checkpoints without HEAD: 0 2022-12-16 16:06:14 [2,085ms] [Info] [omni.meta-upgrade] [1212133] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():2091: Updated checkpoints ACLS: 0 2022-12-16 16:06:14 [2,085ms] [Info] [omni.core] [1212133] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:initialize():585: Version: 114.4-84b6cd59 2022-12-16 16:06:14 [2,086ms] [Info] [omni.core] [1212133] ../../../source/plugins/omni.core/impl/Mounts.cpp:createMount():97: Successfully created resolver of type: omniverse_resolver_s3 2022-12-16 16:06:14 [2,088ms] [Info] [omni.core] [1212133] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:processPendingDeltaManagerRecords():1515: Checking for any pending delta manager records. 2022-12-16 16:06:14 [2,088ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():123: waiting for Core to startup ... 2022-12-16 16:06:14 [2,088ms] [Info] [omni.core] [1212140] ../../../source/plugins/omni.core/impl/requests/DataStoreBase.cpp:runContentRemove():351: Started content remove task 2022-12-16 16:06:14 [2,088ms] [Info] [omni.core] [1212141] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:pingTask():599: Starting delta applier process. 2022-12-16 16:06:14 [2,089ms] [Info] [carb.launcher.plugin] [1212141] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x7ff7e0003e10 for './omni.delta-applier.app'. 2022-12-16 16:06:14 [2,089ms] [Info] [carb.launcher.plugin] [1212141] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni.delta-applier.app' from process 1212133. 2022-12-16 16:06:14 [2,089ms] [Info] [carb.launcher.plugin] [1212146] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1212168 from the parent 1212133. 2022-12-16 16:06:14 [2,089ms] [Info] [carb.launcher.plugin] [1212146] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1212168. 2022-12-16 16:06:14 [2,091ms] [Info] [carb.launcher.plugin] [1212146] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1212168. 2022-12-16 16:06:14 [2,091ms] [Info] [carb.launcher.plugin] [1212141] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1212168 from the parent 1212133. 2022-12-16 16:06:14 [2,091ms] [Info] [carb.launcher.plugin] [1212141] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1212168 {process = 0x7ff7e0003e10} 2022-12-16 16:06:14 [2,091ms] [Info] [carb.launcher.plugin] [1212169] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1212168. 2022-12-16 16:06:14 [2,091ms] [Info] [carb.launcher.plugin] [1212170] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1212168. 2022-12-16 16:06:14 [2,277ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:06:14,671 - system_log - INFO - reconnecting ... 2022-12-16 16:06:14 [2,277ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:06:14,672 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:06:14 [2,292ms] [Info] [omni.core] [1212138] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:pingTask():629: Successfully started delta applier process. 2022-12-16 16:06:15 [3,279ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:06:15,674 - system_log - INFO - reconnecting ... 2022-12-16 16:06:15 [3,280ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:06:15,675 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:06:16 [4,088ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():126: waiting for Core to startup (done) 2022-12-16 16:06:16 [4,089ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():137: Omniverse C++ ApiServer server started. 2022-12-16 16:06:16 [4,089ms] [Info] [omni.signalhandler] [1212133] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():150: Installed signal handler for SIGINT 2022-12-16 16:06:16 [4,089ms] [Info] [omni.signalhandler] [1212133] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():159: Installed signal handler for SIGTERM 2022-12-16 16:06:16 [4,089ms] [Info] [omni.signalhandler] [1212133] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():168: Ignoring SIGPIPE 2022-12-16 16:06:16 [4,281ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:06:16,676 - system_log - INFO - reconnecting ... 2022-12-16 16:06:16 [4,283ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:06:16,678 - system_log - INFO - connected 2022-12-16 16:06:16 [4,284ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Server started 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215573] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215573] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 16:18:13 [0ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:18:13 [1ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 16:18:13 [1ms] [Info] [omni.server] [1215584] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 16:18:13 [2ms] [Info] [omni.server] [1215584] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-16 16:18:13 [2ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:18:13 [2ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:18:13 [2ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:18:13 [2ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:18:13 [3ms] [Info] [carb.launcher.plugin] [1215584] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x260c5b0 for './omni-lft-service/omni-lft-service'. 2022-12-16 16:18:13 [3ms] [Info] [carb.launcher.plugin] [1215584] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 1215584. 2022-12-16 16:18:13 [3ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-16 16:18:13 [3ms] [Info] [carb] [1215584] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-16 16:18:13 [3ms] [Info] [carb.launcher.plugin] [1215596] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1215597 from the parent 1215584. 2022-12-16 16:18:13 [3ms] [Info] [carb.launcher.plugin] [1215596] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1215597. 2022-12-16 16:18:13 [3ms] [Info] [carb.launcher.plugin] [1215596] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1215597. 2022-12-16 16:18:13 [4ms] [Info] [carb.launcher.plugin] [1215584] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1215597 from the parent 1215584. 2022-12-16 16:18:13 [4ms] [Info] [carb.launcher.plugin] [1215584] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1215597 {process = 0x260c5b0} 2022-12-16 16:18:13 [4ms] [Info] [carb.launcher.plugin] [1215598] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1215597. 2022-12-16 16:18:13 [4ms] [Info] [omni.core] [1215584] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-16 16:18:13 [4ms] [Info] [omni.core] [1215584] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-16 16:18:13 [4ms] [Info] [carb.launcher.plugin] [1215599] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1215597. 2022-12-16 16:18:13 [269ms] [Error] [omni.core] [1215599] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-16 16:18:13 [273ms] [Error] [omni.core] [1215599] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:18:13,644 - system_log - INFO - reconnecting ... 2022-12-16 16:18:13 [282ms] [Error] [omni.core] [1215599] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:18:13,653 - system_log - INFO - connected 2022-12-16 16:18:13 [283ms] [Error] [omni.core] [1215599] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "run_server.py", line 3, in File "lft/server.py", line 801, in run_application File "asyncio/base_events.py", line 616, in run_until_complete File "lft/server.py", line 715, in startup_server File "aiohttp/web_runner.py", line 121, in start File "asyncio/base_events.py", line 1463, in create_server OSError: [Errno 98] error while attempting to bind on address ('127.0.0.1', 3030): address already in use [1215597] Failed to execute script 'run_server' due to unhandled exception! 2022-12-16 16:18:13 [308ms] [Info] [carb.launcher.plugin] [1215599] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:18:13 [308ms] [Info] [carb.launcher.plugin] [1215598] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:18:13 [308ms] [Info] [carb.launcher.plugin] [1215599] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1215597, bytes = 0} 2022-12-16 16:18:13 [308ms] [Info] [carb.launcher.plugin] [1215598] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1215597, bytes = 0} 2022-12-16 16:18:13 [308ms] [Info] [carb.launcher.plugin] [1215599] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1215597. 2022-12-16 16:18:13 [308ms] [Info] [carb.launcher.plugin] [1215598] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1215597. 2022-12-16 16:18:15 [2,004ms] [Info] [omni.core] [1215589] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():308: discovery client: starting up 2022-12-16 16:18:15 [2,004ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:18:15 [2,004ms] [Info] [omni.core] [1215589] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():351: discovery client: connecting to ws://localhost:3333 2022-12-16 16:18:15 [2,005ms] [Info] [omni.core] [1215589] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():386: discovery client: registering 'external' with transport 'connlib' and transport parameters '{ "url": "ws://127.0.0.1:3009" }' 2022-12-16 16:18:15 [2,005ms] [Info] [omni.core] [1215584] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:getOmniResolverInstance():87: Successfully created OmniResolver instance of type(omniverse_resolver_composite) 2022-12-16 16:18:15 [2,005ms] [Info] [carb] [1215584] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:18:15 [2,005ms] [Warning] [omni.keyvaluedatabase.leveldb] [1215584] ../../../source/plugins/omni.keyvaluedatabase.leveldb/leveldb.cpp:LevelDBImpl():74: wasn't able to load the meta database, trying to repair it ... 2022-12-16 16:18:15 [2,026ms] [Warning] [omni.keyvaluedatabase.leveldb] [1215584] ../../../source/plugins/omni.keyvaluedatabase.leveldb/leveldb.cpp:LevelDBImpl():79: repair failed 2022-12-16 16:18:15 [2,027ms] [Fatal] [omni.core] [1215584] ../../../source/plugins/common/KvDbAsync.h:Instance():36: error while initializing database 'meta' 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 16:18:26 [0ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:18:26 [1ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:18:26 [1ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 16:18:26 [1ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 16:18:26 [1ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 16:18:26 [2ms] [Info] [omni.server] [1216573] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-16 16:18:26 [2ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:18:26 [2ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:18:26 [2ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:18:26 [3ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:18:26 [3ms] [Info] [carb.launcher.plugin] [1216573] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x1bb85b0 for './omni-lft-service/omni-lft-service'. 2022-12-16 16:18:26 [3ms] [Info] [carb.launcher.plugin] [1216573] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 1216573. 2022-12-16 16:18:26 [3ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-16 16:18:26 [3ms] [Info] [carb] [1216573] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-16 16:18:26 [3ms] [Info] [carb.launcher.plugin] [1216585] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1216586 from the parent 1216573. 2022-12-16 16:18:26 [3ms] [Info] [carb.launcher.plugin] [1216585] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1216586. 2022-12-16 16:18:26 [4ms] [Info] [carb.launcher.plugin] [1216585] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1216586. 2022-12-16 16:18:26 [4ms] [Info] [carb.launcher.plugin] [1216573] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1216586 from the parent 1216573. 2022-12-16 16:18:26 [4ms] [Info] [carb.launcher.plugin] [1216573] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1216586 {process = 0x1bb85b0} 2022-12-16 16:18:26 [4ms] [Info] [omni.core] [1216573] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-16 16:18:26 [4ms] [Info] [omni.core] [1216573] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-16 16:18:26 [4ms] [Info] [carb.launcher.plugin] [1216587] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1216586. 2022-12-16 16:18:26 [4ms] [Info] [carb.launcher.plugin] [1216588] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1216586. 2022-12-16 16:18:26 [246ms] [Error] [omni.core] [1216588] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-16 16:18:26 [250ms] [Error] [omni.core] [1216588] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:18:26,659 - system_log - INFO - reconnecting ... 2022-12-16 16:18:26 [260ms] [Error] [omni.core] [1216588] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:18:26,669 - system_log - INFO - connected 2022-12-16 16:18:26 [260ms] [Error] [omni.core] [1216588] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "run_server.py", line 3, in File "lft/server.py", line 801, in run_application File "asyncio/base_events.py", line 616, in run_until_complete File "lft/server.py", line 715, in startup_server File "aiohttp/web_runner.py", line 121, in start File "asyncio/base_events.py", line 1463, in create_server OSError: [Errno 98] error while attempting to bind on address ('127.0.0.1', 3030): address already in use [1216586] Failed to execute script 'run_server' due to unhandled exception! 2022-12-16 16:18:26 [285ms] [Info] [carb.launcher.plugin] [1216587] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:18:26 [285ms] [Info] [carb.launcher.plugin] [1216588] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:18:26 [285ms] [Info] [carb.launcher.plugin] [1216588] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1216586, bytes = 0} 2022-12-16 16:18:26 [285ms] [Info] [carb.launcher.plugin] [1216587] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1216586, bytes = 0} 2022-12-16 16:18:26 [285ms] [Info] [carb.launcher.plugin] [1216588] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1216586. 2022-12-16 16:18:26 [285ms] [Info] [carb.launcher.plugin] [1216587] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1216586. 2022-12-16 16:18:28 [2,004ms] [Info] [omni.core] [1216577] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():308: discovery client: starting up 2022-12-16 16:18:28 [2,004ms] [Info] [omni.core] [1216577] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():351: discovery client: connecting to ws://localhost:3333 2022-12-16 16:18:28 [2,004ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:18:28 [2,005ms] [Info] [omni.core] [1216577] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():386: discovery client: registering 'external' with transport 'connlib' and transport parameters '{ "url": "ws://127.0.0.1:3009" }' 2022-12-16 16:18:28 [2,005ms] [Info] [omni.core] [1216573] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:getOmniResolverInstance():87: Successfully created OmniResolver instance of type(omniverse_resolver_composite) 2022-12-16 16:18:28 [2,005ms] [Info] [carb] [1216573] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:18:28 [2,006ms] [Warning] [omni.keyvaluedatabase.leveldb] [1216573] ../../../source/plugins/omni.keyvaluedatabase.leveldb/leveldb.cpp:LevelDBImpl():74: wasn't able to load the meta database, trying to repair it ... 2022-12-16 16:18:28 [2,012ms] [Warning] [omni.keyvaluedatabase.leveldb] [1216573] ../../../source/plugins/omni.keyvaluedatabase.leveldb/leveldb.cpp:LevelDBImpl():79: repair failed 2022-12-16 16:18:28 [2,013ms] [Fatal] [omni.core] [1216573] ../../../source/plugins/common/KvDbAsync.h:Instance():36: error while initializing database 'meta' 2022-12-16 16:21:33 [920,861ms] [Info] [omni.server.app] [1212133] ../../../source/apps/omni.server/main.cpp:main():106: Got signal to exit the application. 2022-12-16 16:21:33 [920,861ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:beginShutdown():391: OmniverseServer shutdown process starting. 2022-12-16 16:21:33 [920,861ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:beginShutdown():392: Waiting for any outstanding transactions to complete. 2022-12-16 16:21:33 [920,861ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:beginShutdown():396: Releasing WebSocket listen ports 2022-12-16 16:21:33 [920,861ms] [Info] [omni.server] [1217756] ../../../source/plugins/omni.server/ApiConnection.cpp:shutdownCore():70: Shutdown Core 2022-12-16 16:21:33 [920,861ms] [Info] [omni.core] [1212159] ../../../source/plugins/omni.core/impl/Scheduler.cpp:scheduler():122: Scheduling task is shutted down... 2022-12-16 16:21:33 [920,862ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Shutting down server...wait 2022-12-16 16:21:33 [920,862ms] [Info] [omni.core] [1212138] ../../../source/plugins/omni.core/impl/requests/DataStoreBase.cpp:runContentRemove():422: Finished content remove task 2022-12-16 16:21:33 [920,862ms] [Warning] [carb] [1217756] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IObjects v1.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 16:21:33 [920,863ms] [Info] [omni.core] [1217756] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:operator()():849: Content resolver statistics: Total: flush: 0 write: 0 read: 0 stat: 0 remove: 0 list: 0|Temporal: flush: 0 write: 0 read: 0 stat: 0 remove: 0 list: 0|Persistent: flush: 0 write: 0 read: 0 stat: 0 remove: 0 list: 0 2022-12-16 16:21:33 [920,863ms] [Info] [omni.server.app] [1212133] ../../../source/apps/omni.server/main.cpp:main():120: Releasing the OmniverseServer instance. 2022-12-16 16:21:33 [920,863ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():310: OmniverseServerImpl destructor. 2022-12-16 16:21:33 [920,863ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():312: Releasing ClientConnectionManager 2022-12-16 16:21:33 [920,863ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():315: Releasing ServiceConnectionManager 2022-12-16 16:21:33 [920,863ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():318: Releasing LftConnectionManager 2022-12-16 16:21:33 [920,863ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():321: Releasing ConnectionManager 2022-12-16 16:21:33 [920,865ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:kill():327: killing the child process 1212168 {exitCode = 137, flags = 0x00000002, timeout = 5000} 2022-12-16 16:21:33 [920,865ms] [Error] [omni.core] [1212149] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Server is stopped 2022-12-16 16:21:33 [920,867ms] [Info] [carb.launcher.plugin] [1212170] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:21:33 [920,867ms] [Info] [carb.launcher.plugin] [1212169] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212170] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1212168, bytes = 0} 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212169] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1212168, bytes = 0} 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212169] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1212168. 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212170] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1212168. 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():254: waiting for the process 0x7ff7e0003e10 (1212168) to exit {timeout = 18446744073709551615} 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():259: the process has already exited with code 137. 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():247: destroying the process object 0x7ff7e0003e10 {pid = 1212168} 2022-12-16 16:21:33 [920,868ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():249: done destroying the process object 0x7ff7e0003e10 {pid = 1212168} 2022-12-16 16:21:33 [920,869ms] [Info] [omni.core] [1212140] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():457: discovery client: exiting 2022-12-16 16:21:33 [920,869ms] [Info] [omni.core] [1212133] ../../../source/plugins/common/ChildProcess.cpp:kill():125: Killing child server process ./omni-lft-service/omni-lft-service 2022-12-16 16:21:33 [920,869ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:kill():327: killing the child process 1212147 {exitCode = 137, flags = 0x00000003, timeout = 5000} 2022-12-16 16:21:33 [920,871ms] [Info] [carb.launcher.plugin] [1212149] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:21:33 [920,871ms] [Info] [carb.launcher.plugin] [1212148] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-16 16:21:33 [920,871ms] [Info] [carb.launcher.plugin] [1212149] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1212147, bytes = 0} 2022-12-16 16:21:33 [920,871ms] [Info] [carb.launcher.plugin] [1212148] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1212147, bytes = 0} 2022-12-16 16:21:33 [920,871ms] [Info] [carb.launcher.plugin] [1212148] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1212147. 2022-12-16 16:21:33 [920,871ms] [Info] [carb.launcher.plugin] [1212149] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1212147. 2022-12-16 16:21:33 [920,872ms] [Info] [omni.core] [1212133] ../../../source/plugins/common/ChildProcess.cpp:kill():127: Waiting for child server process ./omni-lft-service/omni-lft-service to exit 2022-12-16 16:21:33 [920,872ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():254: waiting for the process 0xcfe6c0 (1212147) to exit {timeout = 18446744073709551615} 2022-12-16 16:21:33 [920,872ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:wait():259: the process has already exited with code 137. 2022-12-16 16:21:33 [920,872ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():247: destroying the process object 0xcfe6c0 {pid = 1212147} 2022-12-16 16:21:33 [920,872ms] [Info] [carb.launcher.plugin] [1212133] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:~ProcessImpl():249: done destroying the process object 0xcfe6c0 {pid = 1212147} 2022-12-16 16:21:33 [920,872ms] [Info] [omni.server] [1212133] ../../../source/plugins/omni.server/OmniverseServer.cpp:destroy():327: Destructor complete. 2022-12-16 16:24:00 [0ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:24:00 [0ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:24:00 [3ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 16:24:00 [3ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:24:00 [4ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:24:00 [4ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:24:00 [7ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:24:00 [8ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 16:24:00 [10ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 16:24:00 [12ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:24:00 [12ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:24:00 [13ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:24:00 [13ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:24:00 [15ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:24:00 [15ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:24:00 [19ms] [Info] [carb] [2784] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:24:00 [19ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:24:00 [19ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:24:00 [19ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:24:00 [19ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:24:00 [19ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:24:00 [19ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 16:24:00 [19ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 16:24:00 [19ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 16:24:00 [20ms] [Info] [omni.server] [2784] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-16 16:24:00 [20ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:24:00 [21ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:24:00 [21ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:24:00 [23ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:24:00 [23ms] [Info] [carb.launcher.plugin] [2784] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x12aa370 for './omni-lft-service/omni-lft-service'. 2022-12-16 16:24:00 [23ms] [Info] [carb.launcher.plugin] [2784] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 2784. 2022-12-16 16:24:00 [23ms] [Info] [carb] [2784] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-16 16:24:00 [23ms] [Info] [carb] [2784] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-16 16:24:00 [24ms] [Info] [carb.launcher.plugin] [2893] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 2895 from the parent 2784. 2022-12-16 16:24:00 [24ms] [Info] [carb.launcher.plugin] [2893] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 2895. 2022-12-16 16:24:00 [25ms] [Info] [carb.launcher.plugin] [2893] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 2895. 2022-12-16 16:24:00 [25ms] [Info] [carb.launcher.plugin] [2784] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 2895 from the parent 2784. 2022-12-16 16:24:00 [25ms] [Info] [carb.launcher.plugin] [2784] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 2895 {process = 0x12aa370} 2022-12-16 16:24:00 [25ms] [Info] [omni.core] [2784] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-16 16:24:00 [25ms] [Info] [omni.core] [2784] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-16 16:24:00 [25ms] [Info] [carb.launcher.plugin] [2901] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 2895. 2022-12-16 16:24:00 [25ms] [Info] [carb.launcher.plugin] [2900] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 2895. 2022-12-16 16:24:01 [352ms] [Error] [omni.core] [2901] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-16 16:24:01 [358ms] [Error] [omni.core] [2901] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:01,283 - system_log - INFO - reconnecting ... 2022-12-16 16:24:01 [366ms] [Error] [omni.core] [2901] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:01,291 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-16 16:24:04 [0ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-16 16:24:04 [1ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-16 16:24:04 [1ms] [Info] [omni.server] [3220] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-16 16:24:04 [2ms] [Info] [omni.server] [3220] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-16 16:24:04 [2ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-16 16:24:04 [2ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-16 16:24:04 [2ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-16 16:24:04 [3ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-16 16:24:04 [3ms] [Info] [carb.launcher.plugin] [3220] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x1686430 for './omni-lft-service/omni-lft-service'. 2022-12-16 16:24:04 [3ms] [Info] [carb.launcher.plugin] [3220] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 3220. 2022-12-16 16:24:04 [3ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-16 16:24:04 [3ms] [Info] [carb] [3220] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-16 16:24:04 [3ms] [Info] [carb.launcher.plugin] [3233] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 3234 from the parent 3220. 2022-12-16 16:24:04 [3ms] [Info] [carb.launcher.plugin] [3233] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 3234. 2022-12-16 16:24:04 [4ms] [Info] [carb.launcher.plugin] [3233] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 3234. 2022-12-16 16:24:04 [4ms] [Info] [carb.launcher.plugin] [3220] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 3234 from the parent 3220. 2022-12-16 16:24:04 [4ms] [Info] [carb.launcher.plugin] [3220] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 3234 {process = 0x1686430} 2022-12-16 16:24:04 [4ms] [Info] [omni.core] [3220] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-16 16:24:04 [4ms] [Info] [omni.core] [3220] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-16 16:24:04 [4ms] [Info] [carb.launcher.plugin] [3235] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 3234. 2022-12-16 16:24:04 [4ms] [Info] [carb.launcher.plugin] [3236] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 3234. 2022-12-16 16:24:05 [240ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-16 16:24:05 [244ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:05,135 - system_log - INFO - reconnecting ... 2022-12-16 16:24:05 [251ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:05,143 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:24:06 [1,253ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:24:06,144 - system_log - INFO - reconnecting ... 2022-12-16 16:24:06 [1,253ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:06,145 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:24:06 [2,004ms] [Info] [omni.core] [3224] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():308: discovery client: starting up 2022-12-16 16:24:06 [2,004ms] [Info] [omni.core] [3224] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():351: discovery client: connecting to ws://localhost:3333 2022-12-16 16:24:06 [2,004ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-16 16:24:06 [2,005ms] [Info] [omni.core] [3224] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():386: discovery client: registering 'external' with transport 'connlib' and transport parameters '{ "url": "ws://127.0.0.1:3009" }' 2022-12-16 16:24:06 [2,009ms] [Info] [omni.core] [3220] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:getOmniResolverInstance():87: Successfully created OmniResolver instance of type(omniverse_resolver_composite) 2022-12-16 16:24:06 [2,009ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-16 16:24:06 [2,053ms] [Warning] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IObjects v1.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 16:24:06 [2,053ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-16 16:24:06 [2,072ms] [Warning] [carb] [3220] ../../../source/framework/FrameworkImpl.cpp:acquireInterface():747: Acquiring non optional plugin interface which is not listed as dependency: [omni::server::IMetaUpgrade v2.0] (plugin: (null)), by client: omni.core. Add it to CARB_PLUGIN_IMPL_DEPS() macro of a client. 2022-12-16 16:24:06 [2,072ms] [Info] [carb] [3220] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:upgrade():2915: Meta database doesn't require upgrade 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1463: Fixing folder children counters .... 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1471: Processed 1 / 6 records 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1497: Checking for orphaned folder children counter records .... 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1508: Processed 1 / 6 records 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixFolderChildrenCounters():1532: Fixing folder children counters done 2022-12-16 16:24:06 [2,072ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1304: Fixing content refcounts .... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1311: Collecting content -> refcount mapping ... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1315: Processed 1 / 6 records 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1350: Collecting content -> refcount mapping done 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1351: Setting refcount ... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1380: Setting refcount done 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixContentRefcounts():1382: Checking content table for unused content 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1550: Repairing versioning data... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1563: Reading meta records... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1671: Checking branch lists... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1716: Fixed branch lists count: 0 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1718: Check for orphaned branch lists... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1741: Checking version reference counters... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1796: Fixed version reference counters: 0 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1797: Deleted orphaned version reference counters 0 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1799: Checking checkpoint messages... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():1860: Verifying checkpoints... 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():2090: Deleted checkpoints without HEAD: 0 2022-12-16 16:24:06 [2,073ms] [Info] [omni.meta-upgrade] [3220] ../../../source/plugins/omni.meta-upgrade/MetaUpgrade.cpp:fixVersioning():2091: Updated checkpoints ACLS: 0 2022-12-16 16:24:06 [2,073ms] [Info] [omni.core] [3220] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:initialize():585: Version: 114.4-84b6cd59 2022-12-16 16:24:06 [2,076ms] [Info] [omni.core] [3220] ../../../source/plugins/omni.core/impl/Mounts.cpp:createMount():97: Successfully created resolver of type: omniverse_resolver_s3 2022-12-16 16:24:06 [2,078ms] [Info] [omni.core] [3220] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:processPendingDeltaManagerRecords():1515: Checking for any pending delta manager records. 2022-12-16 16:24:06 [2,078ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():123: waiting for Core to startup ... 2022-12-16 16:24:06 [2,078ms] [Info] [omni.core] [3225] ../../../source/plugins/omni.core/impl/requests/DataStoreBase.cpp:runContentRemove():351: Started content remove task 2022-12-16 16:24:06 [2,078ms] [Info] [omni.core] [3226] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:pingTask():599: Starting delta applier process. 2022-12-16 16:24:06 [2,078ms] [Info] [carb.launcher.plugin] [3226] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x7f8e14002f40 for './omni.delta-applier.app'. 2022-12-16 16:24:06 [2,078ms] [Info] [carb.launcher.plugin] [3226] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni.delta-applier.app' from process 3220. 2022-12-16 16:24:06 [2,079ms] [Info] [carb.launcher.plugin] [3233] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 3263 from the parent 3220. 2022-12-16 16:24:06 [2,079ms] [Info] [carb.launcher.plugin] [3233] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 3263. 2022-12-16 16:24:06 [2,081ms] [Info] [carb.launcher.plugin] [3233] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 3263. 2022-12-16 16:24:06 [2,081ms] [Info] [carb.launcher.plugin] [3226] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 3263 from the parent 3220. 2022-12-16 16:24:06 [2,081ms] [Info] [carb.launcher.plugin] [3226] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 3263 {process = 0x7f8e14002f40} 2022-12-16 16:24:06 [2,081ms] [Info] [carb.launcher.plugin] [3264] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 3263. 2022-12-16 16:24:06 [2,081ms] [Info] [carb.launcher.plugin] [3265] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 3263. 2022-12-16 16:24:07 [2,255ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:24:07,147 - system_log - INFO - reconnecting ... 2022-12-16 16:24:07 [2,256ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:07,147 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:24:07 [2,292ms] [Info] [omni.core] [3227] ../../../source/plugins/omni.core/impl/DeltaManager.cpp:pingTask():629: Successfully started delta applier process. 2022-12-16 16:24:08 [3,258ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:24:08,149 - system_log - INFO - reconnecting ... 2022-12-16 16:24:08 [3,259ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:08,150 - system_log - INFO - [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 16:24:08 [4,078ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():126: waiting for Core to startup (done) 2022-12-16 16:24:08 [4,080ms] [Info] [omni.server] [3220] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():137: Omniverse C++ ApiServer server started. 2022-12-16 16:24:08 [4,080ms] [Info] [omni.signalhandler] [3220] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():150: Installed signal handler for SIGINT 2022-12-16 16:24:08 [4,080ms] [Info] [omni.signalhandler] [3220] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():159: Installed signal handler for SIGTERM 2022-12-16 16:24:08 [4,080ms] [Info] [omni.signalhandler] [3220] ../../../source/plugins/omni.signalhandler/SignalHandler.cpp:SignalHandler():168: Ignoring SIGPIPE 2022-12-16 16:24:09 [4,260ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "idl/connection/transport/omni.py", line 110, in prepare File "websockets/legacy/client.py", line 650, in __await_impl_timeout__ File "asyncio/tasks.py", line 494, in wait_for File "websockets/legacy/client.py", line 654, in __await_impl__ File "asyncio/base_events.py", line 1025, in create_connection File "asyncio/base_events.py", line 1010, in create_connection File "asyncio/base_events.py", line 924, in _connect_sock File "asyncio/selector_events.py", line 496, in sock_connect File "asyncio/selector_events.py", line 528, in _sock_connect_cb ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 3007) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "lft/server.py", line 560, in await_connection File "idl/connection/transport/omni.py", line 114, in prepare idl.connection.transport.TransportError: [Errno 111] Connect call failed ('127.0.0.1', 3007) 2022-12-16 11:24:09,152 - system_log - INFO - reconnecting ... 2022-12-16 16:24:09 [4,272ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-16 11:24:09,164 - system_log - INFO - connected 2022-12-16 16:24:09 [4,274ms] [Error] [omni.core] [3236] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Server started 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.objects (interfaces: [omni::server::IObjects v1.0]) (impl: omni.objects) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.messaging (interfaces: [omni::server::IMessaging v1.0]) (impl: omni.messaging) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.verifier (interfaces: [omni::server::IVerifier v1.1]) (impl: omni.verifier) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventlogger (interfaces: [omni::server::IEventLogger v1.0]) (impl: omni.eventlogger) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.meta-upgrade (interfaces: [omni::server::IMetaUpgrade v2.0]) (impl: omni.meta-upgrade) 2022-12-19 14:59:13 [0ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/FrameworkImpl.cpp:registerPlugin():391: Registered plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.signalhandler (interfaces: [omni::server::ISignalHandler v2.0]) (impl: omni.signalhandler) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.wakeupthread (interfaces: [omni::server::IWakeupThread v1.0]) (impl: omni.wakeupthread) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.eventsystem (interfaces: [omni::server::IEventSystem v1.0]) (impl: omni.eventsystem) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.server (interfaces: [omni::server::IDeltaContentServer v1.0],[omni::server::IServer v1.1]) (impl: omni.server) 2022-12-19 14:59:13 [1ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.tasking.plugin (interfaces: [carb::tasking::ITasking v2.2],[carb::tasking::IThreadPool v1.0]) (impl: carb.tasking.plugin) 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():151: Launching C++ Omniverse API Server version (1.0.84b6cd59) API version(1.20) 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/OmniverseServer.cpp:create():153: Build from commit 84b6cd59 on Tue Dec 6 07:57:56 2022 -0800 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/OmniverseServer.cpp:OmniverseServer():98: Starting up Omniverse Server. 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():71: ============================================================================== 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():72: Default Data Directory: /home/baxelrod/.local/share/ov/data/server 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():73: Default Log Directory : /home/baxelrod/.cache/ov/server 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/Globals.cpp:initDefaultDirs():74: ============================================================================== 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/omni.server/OmniverseServer.cpp:loadSettings():611: Setting Log directory to: /home/baxelrod/.nvidia-omniverse/logs/Nucleus 2022-12-19 14:59:13 [1ms] [Info] [omni.server] [1180653] ../../../source/plugins/common/ResolvedConfig.h:dumpResolvedConfig():46: Loaded settings: 2022-12-19 14:59:13 [2ms] [Info] [omni.server] [1180653] ../../../source/plugins/common/ResolvedConfig.h:operator()():42: { "log": { "fileLogLevel": "verbose", "detail": false, "includeChannel": true, "includeProcessId": false, "includeSource": true, "processGroupId": 0, "debugConsoleLevel": "verbose", "enableColorOutput": true, "enableDebugConsoleOutput": true, "fileFlushLevel": "verbose", "sources": { "omni.server.API_CLIENT": { "enabled": false }, "omni.server.API_SERVER": { "enabled": false }, "omni.server.DELTA_CONTENT": { "enabled": false }, "omni.server.DBCONTENT": { "enabled": false }, "omni.server.SHUTDOWN": { "enabled": false }, "omni.server.FRAMING": { "enabled": false }, "omni.server.EVENT_SYSTEM": { "enabled": false }, "omni.server.CONNECTIONS": { "enabled": false }, "omni.server.OMNI_OBJECTS": { "enabled": false } }, "outputStreamLevel": "verbose", "flushStandardStreamOutput": false, "includeThreadId": true, "fullDetail": false, "includeLineNumber": true, "enabled": true, "enableStandardStreamOutput": true, "includeFilename": true, "fileAppend": true, "outputStream": "stderr", "setElapsedTimeUnits": "milliseconds", "includeTimeStamp": true, "includeFunctionName": true, "file": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus/omni.server.app.log", "level": "Info", "async": false }, "pluginsLoaded": [ "carb.dictionary*.plugin", "carb.settings.plugin", "carb.launcher.plugin", "carb.tasking.plugin", "carb.crashreporter-breakpad.plugin", "omni.metrics-null", "omni.signalhandler", "omni.wakeupthread", "omni.eventsystem", "omni.keyvaluedatabase.leveldb", "omni.keyvaluedatabase.redis", "omni.server", "omni.websocketslib", "omni.resolver", "omni.core", "carb.omnitrace.plugin", "omni.objects", "omni.meta-upgrade", "omni.verifier", "omni.messaging-redis", "omni.eventlogger" ], "pluginSearchPaths": [ "." ], "crashreporter": { "retryCount": 10, "compressDumpFiles": false, "overrideDefaultDumpFlags": false, "data": { "buildVersion": "114.4.0", "buildMajor": "114", "buildNumber": "21980", "buildMinor": "4", "buildHash": "84b6cd59", "buildBranch": "master", "buildCi": "gitlab", "buildPatch": "0", "buildMr": "0" }, "version": "114.4+master.gitlab.21980.84b6cd59", "url": "https://services.nvidia.com/submit", "dumpFlags": "", "log": "true", "preserveDump": false, "dumpDir": ".", "product": "OmniverseNucleus", "alwaysUpload": false, "debuggerAttachTimeoutMs": 0, "enabled": "true", "skipOldDumpUpload": false, "uploadTimeoutMs": 7200000 }, "omni.common.settings": { "env.var.prefix": "OMNI_" }, "omni.server.app": {}, "privacy": { "performance": "false" }, "plugins": { "omni.websocketslib": { "tracing": false, "defaults": { "http_max_header_length": "61440", "tcp_timeout_seconds": 10, "tcp_keepalive_time": 10, "tcp_keepalive_interval": 2, "tcp_keepalive_probes": 2 } }, "omni.meta-upgrade": { "enabled": "true", "repair": { "enabled": "true", "only_after_upgrade": "true" }, "exit_on_any_failure": false, "interactive": false, "report_progress_every": 1000, "content_resolver": { "resolver": "omniverse_resolver_file", "options": { "async": true, "root": "" } }, "remove_deduplicated_content": true, "convertors": { "meta": { "dry_run_since_version": -1 } } }, "omni.messaging-redis": { "host": "kube-msk-redis.nvidia.com", "port": "6379", "password": "yBGfkZ0iRT", "max_reconnects": 5, "reconnect_interval_ms": 1000, "timeout_ms": 500 }, "carb.tasking.plugin": { "debugLocks": false, "debugTaskBacktrace": false, "debugWaitingTasks": false, "stuckCheckSeconds": 1, "resetFiberStack": true, "fiberCount": 4095, "threadCount": "4" }, "omni.metrics-prometheus": { "model": "pull", "pull": { "host": "127.0.0.1", "port": 3010, "endpoint": "/" }, "push": { "host": "", "port": 9091, "flushTimeInSeconds": 5 }, "metricsPrefix": "omni_server_api", "predefined_labels": { "omni_instance": "" } }, "omni.server": { "delta_content_hang_timeout": 2.0, "default_branch_name": "default", "auth": { "connection_id_salt": "7b6a5f4e0a1b2c3d", "jwt_rs256_public_key": "-----BEGIN PUBLIC KEY-----\nMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBAJFEw8/EqME3ZB1dGk/fvIP6UTzrWA7d\ny3dLlSYkTI0IO7rhia55RVsuxVLuQ4QO1jsC6YF/jw8hqMdyXQkCetUCAwEAAQ==\n-----END PUBLIC KEY-----" }, "metadata": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "meta.1.1" } }, "max_full_path_length": "16384", "concurrent_mount_commands": true, "disable_passwords": false, "metrics": { "enabled": "false", "request": { "duration": { "buckets": [ 100, 250, 500, 750, 1000, 2500, 5000, 7500, 10000.0, 50000.0, 100000.0, 1000000.0 ] } } }, "file_system_table": true, "multipart_chunk_size_mb": "5", "mount_management_requires_admin": true, "omniobjects.values": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "record_integrity_check": false, "delta_manager": { "delta_apply_throttle_ms": 500, "delta_apply_limit_mb": 128, "bw_estimate_bps": 256, "num_retries_until_ignored": 10, "ignorelist_retry_delay_ms": 5000, "reconnect_timeout_s": 20, "delta_apply_timeout_ms": 5000, "ping_interval_ms": 1000, "ping_interval_on_startup_ms": 100, "ping_timeout_ms": 1000 }, "lft": { "launch": "true", "check_alive_interval": 10, "threshold": "65536" }, "mounts": { "default_mounts": [ { "path": "/NVIDIA", "resolver": "omniverse_resolver_s3", "options": "{\"timeout\": 600, \"host\":\"dcb18d6mfegct.cloudfront.net\",\"service\":\"s3\",\"secure\":false,\"redirection\":\"http://dcb18d6mfegct.cloudfront.net/\"}", "override": true } ], "allowed_mount_resolvers": [ "omniverse_resolver_file", "omniverse_resolver_s3" ], "allowed_file_resolver_roots": {}, "logging_enabled": "false" }, "password_recovery": true, "crash_record_recovery": true, "max_path_piece_length": "255", "token_expire_time": 86400, "delta_content_ping_interval": 0.0, "content_flush_enabled": true, "master_password": "omniverse", "lft_content_expiration_time": "600", "content_resolver": { "delete_on_startup": "false", "resolver": "omniverse_resolver_composite", "options": { "elem_size_max": 1048576, "number_elems_low": 100000, "number_elems_high": 120000, "persistent_options": { "async": true, "root": "" }, "total_size_high": 6442450944, "temporal_kind": "omniverse_resolver_memory", "total_size_low": 4294967296, "temporal_options": {}, "persistent_kind": "omniverse_resolver_file" }, "logging_enabled": "true" }, "non_utf8_paths_allowed": "false", "allowed_users": {}, "omniobjects": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "omniobjects.1.0" }, "max_objects_in_cache": 1024, "lft_read_timeout_seconds": 3600, "max_content_mem_cache_size": 104857600, "lft_multi_content_threshold": 0, "unpack_object_buffer_size": 33554432, "ext_value_min_size": 262144, "mem_vs_file_threshold": 262144, "content_mem_limit": 268435456, "max_content_file_cache_size": 1048576000 }, "default_s3_proxy_cache": "", "default_acls": { "owner": "rwa", "rest": { "gm": "rwa" } }, "endpoints": { "api": [ { "interface": "127.0.0.1", "port": "3009", "secure": false, "lft_address": "http://*:3030" } ], "service": [ { "interface": "127.0.0.1", "port": "3006", "secure": false } ], "lft": [ { "interface": "127.0.0.1", "port": "3007", "secure": false } ] }, "store_crash_repro": true, "dirs": { "content_dir": "/home/baxelrod/.local/share/ov/data/server", "meta_dir": "/home/baxelrod/.local/share/ov/data/server", "omniobjects_dir": "/home/baxelrod/.local/share/ov/data/server", "metric_dir": "/home/baxelrod/.local/share/ov/data/server", "record_dir": "", "usergroups_dir": "/home/baxelrod/.local/share/ov/data/server", "log_dir": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "list_batch_size_limit": 100, "run": { "versioning": "true" }, "admin_folders": [ "/Users/omniverse/", "/Users/deeptag_service/", "/Users/search_service/", "/Users/tags_service/", "/Users/thumbnails_service/", "/Users/index/", "/.system/" ], "peer_messaging": { "enable": "false", "exit_on_connection_failure": "false", "channel": "omni_server_messages", "application_name": "Nucleus API", "payloads": { "create_event": { "name": "create" }, "list_event": { "name": "list" }, "servernotification_event": { "name": "servernotification" } } }, "omniobjects.cache": { "resolver_kind": "omniverse_resolver_file", "resolver_options": { "async": true, "root": "" }, "logging_enabled": "true" }, "default_folders": [ [ "/", { "users": "r", "gm": "rwa" } ], [ "/Users/", { "users": "r", "gm": "rwa" } ], [ "/Projects/", { "users": "rw", "gm": "rwa" } ], [ "/Library/", { "users": "rw", "gm": "rwa" } ] ], "quotas": { "list_subscriptions": "0", "read_subscriptions": "0" }, "home_folder_default_acls": { "owner": "rwa", "rest": { "users": "", "gm": "rwa" } }, "api_acls": { "list": "r", "unlock": "w", "rename": "a", "lock": "w", "delete": "a", "get_acl_resolved": "r", "get_checkpoints": "r", "checkpoint_version": "w", "unmount": "w", "create_asset": "w", "create_directory": "w", "mount": "w", "update": "w", "create_object": "w", "copy": "w", "get_acl_v2": "r", "set_acl_v2": "a", "set_path_options": "w", "read": "r" }, "allowed_users_legacy": {}, "test_token_prefix": "", "metric_period": 10.0, "content_type": "", "auto_create_users": true, "delta_content_bw_estimate": 0.5, "discovery": { "enabled": "true", "host": "localhost", "token": "test", "port_based_default_port": 3333, "deployment_string": "{ \"reg\": [ { \"name\": \"external\", \"transport\": { \"type\": \"connlib\", \"params\": { \"url\": \"ws://127.0.0.1:3009\" } } } ] }" }, "content_flush_time_sec": 10, "removed_content_pending_time_sec": 5, "auth_service_client": { "enabled": true, "deployment_lookup": "internal", "override_service_address": "" }, "metacache_flush_count": 1000, "notifications": {}, "usergroups": { "plugin": "omni.keyvaluedatabase.leveldb", "options": { "async": true, "delete_on_startup": "false", "root": "", "subdir": "usergroups.1.0" } }, "file_and_username_filter_exclude_regex": "", "api_logging": { "json_outputs": {}, "omit_event_types": {}, "event_buffer_max_size_bytes": 1048576, "event_buffer_min_size_bytes": 1024, "enable": "false", "event_buffer_try_shrink_interval_sec": 10, "enable_legacy_system": "1", "source_base_name": "api-server", "file_settings": [ { "filename": "eventlog.tsv", "flush_frequency": 1 } ], "source_instance_name": "", "tsv_outputs": [ { "omit_columns": {}, "files": [ "eventlog.tsv" ], "column_order": [ "source", "time", "id", "type", "requestname", "status", "reqid", "username", "connectionid", "clientip", "uri", "eventsize", "optionalinfo", "branch", "checkpoint", "multiplepath" ], "print_header_on_new_field": true, "omit_unknown_columns": true } ] } }, "omni.delta-applier": { "ports": { "server_port": 3001 } }, "omni.resolver": { "directory": ".", "libraries": [ "omniverse_resolver_file", "omniverse_resolver_memory", "omniverse_resolver_composite", "omniverse_resolver_leveldb", "omniverse_resolver_s3" ] } }, "APILOGGING": 1, "LFT": { "INTERFACE": "127.0.0.1" }, "LOG": { "DIR": "/home/baxelrod/.nvidia-omniverse/logs/Nucleus" }, "NUCLEUS": { "SERVER": { "INTERFACE": "127.0.0.1" } }, "S3": { "RESOLVER": { "CACHE": "localhost:8891" } } } 2022-12-19 14:59:13 [2ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.websocketslib (interfaces: [omni::server::IWebSockets v1.1]) (impl: omni.websocketslib) 2022-12-19 14:59:13 [2ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.metrics-null (interfaces: [omni::server::IMetrics v1.1]) (impl: omni.metrics-null) 2022-12-19 14:59:13 [2ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.core (interfaces: [omni::server::ICore v1.0],[omni::server::ICoreMetaContent v1.0],[omni::server::ICoreService v1.0],[omni::server::IServerFeatures v1.0]) (impl: omni.core) 2022-12-19 14:59:13 [2ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.launcher.plugin (interfaces: [carb::launcher::ILauncher v1.3]) (impl: carb.launcher.plugin) 2022-12-19 14:59:13 [2ms] [Info] [carb.launcher.plugin] [1180653] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:ProcessImpl():230: created the process object 0x238f5c0 for './omni-lft-service/omni-lft-service'. 2022-12-19 14:59:13 [3ms] [Info] [carb.launcher.plugin] [1180653] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():643: launching the process './omni-lft-service/omni-lft-service' from process 1180653. 2022-12-19 14:59:13 [3ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: carb.threadutil-framework.plugin (interfaces: [carb::thread::IThreadUtil v1.0]) (impl: carb.threadutil-framework.plugin) 2022-12-19 14:59:13 [3ms] [Info] [carb] [1180653] ../../../source/framework/ThreadUtil.cpp:runRelayTask():199: starting the relayed task thread. 2022-12-19 14:59:13 [3ms] [Info] [carb.launcher.plugin] [1180665] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():883: [child]: starting the child process 1180666 from the parent 1180653. 2022-12-19 14:59:13 [3ms] [Info] [carb.launcher.plugin] [1180665] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():892: [child]: setting the parent death signal for the child process 1180666. 2022-12-19 14:59:13 [4ms] [Info] [carb.launcher.plugin] [1180665] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:forkProcess():1061: [parent]: successfully launched the child process 1180666. 2022-12-19 14:59:13 [4ms] [Info] [carb.launcher.plugin] [1180653] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():775: [parent]: launched the child process 1180666 from the parent 1180653. 2022-12-19 14:59:13 [4ms] [Info] [carb.launcher.plugin] [1180653] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:launch():798: [parent]: successfully launched the child process 1180666 {process = 0x238f5c0} 2022-12-19 14:59:13 [4ms] [Info] [omni.core] [1180653] ../../../source/plugins/common/ChildProcess.cpp:launch():108: Successfully launched the child server Process (/home/baxelrod/.local/share/ov/pkg/nucleus-workstation-2022.4.1/Nucleus/./omni-lft-service/omni-lft-service) 2022-12-19 14:59:13 [4ms] [Info] [omni.core] [1180653] ../../../source/plugins/common/ChildProcess.cpp:launch():111: Sleeping for 2 seconds to get the child server time to spin up 2022-12-19 14:59:13 [4ms] [Info] [carb.launcher.plugin] [1180667] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stdout reader' for the child process 1180666. 2022-12-19 14:59:13 [4ms] [Info] [carb.launcher.plugin] [1180668] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():546: starting the read thread for 'launcher stderr reader' for the child process 1180666. 2022-12-19 14:59:13 [248ms] [Error] [omni.core] [1180668] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:DATA_DIR: /home/baxelrod/.local/share/ov/data/serverBundled - using PyInstaller's runtime dir with resolvers 2022-12-19 14:59:13 [251ms] [Error] [omni.core] [1180668] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-19 09:59:13,544 - system_log - INFO - reconnecting ... 2022-12-19 14:59:13 [260ms] [Error] [omni.core] [1180668] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:2022-12-19 09:59:13,553 - system_log - INFO - connected 2022-12-19 14:59:13 [261ms] [Error] [omni.core] [1180668] ../../../source/plugins/common/ChildProcess.cpp:stderrReadCallback():30: From process ./omni-lft-service/omni-lft-service:Traceback (most recent call last): File "run_server.py", line 3, in File "lft/server.py", line 801, in run_application File "asyncio/base_events.py", line 616, in run_until_complete File "lft/server.py", line 715, in startup_server File "aiohttp/web_runner.py", line 121, in start File "asyncio/base_events.py", line 1463, in create_server OSError: [Errno 98] error while attempting to bind on address ('127.0.0.1', 3030): address already in use [1180666] Failed to execute script 'run_server' due to unhandled exception! 2022-12-19 14:59:13 [297ms] [Info] [carb.launcher.plugin] [1180668] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-19 14:59:13 [297ms] [Info] [carb.launcher.plugin] [1180668] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1180666, bytes = 0} 2022-12-19 14:59:13 [297ms] [Info] [carb.launcher.plugin] [1180667] ../../../source/plugins/carb.launcher/ProcessLinux.cpp:read():234: read ended {errno = 0/Success} 2022-12-19 14:59:13 [297ms] [Info] [carb.launcher.plugin] [1180668] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stderr reader' for the process 1180666. 2022-12-19 14:59:13 [297ms] [Info] [carb.launcher.plugin] [1180667] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():596: read failed. Exiting the read thread {pid = 1180666, bytes = 0} 2022-12-19 14:59:13 [297ms] [Info] [carb.launcher.plugin] [1180667] ../../../source/plugins/carb.launcher/ProcessCommon.cpp:readFromHandleThread():626: done the read thread for 'launcher stdout reader' for the process 1180666. 2022-12-19 14:59:15 [2,005ms] [Info] [omni.core] [1180658] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():308: discovery client: starting up 2022-12-19 14:59:15 [2,005ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.resolver (interfaces: [omni::IResolver v7.0]) (impl: omni.resolver) 2022-12-19 14:59:15 [2,005ms] [Info] [omni.core] [1180658] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():351: discovery client: connecting to ws://localhost:3333 2022-12-19 14:59:15 [2,005ms] [Info] [omni.core] [1180658] ../../../source/plugins/omni.core/impl/ServicesClients.cpp:run():386: discovery client: registering 'external' with transport 'connlib' and transport parameters '{ "url": "ws://127.0.0.1:3009" }' 2022-12-19 14:59:15 [2,006ms] [Info] [omni.core] [1180653] ../../../source/plugins/omni.core/impl/CoreImpl.cpp:getOmniResolverInstance():87: Successfully created OmniResolver instance of type(omniverse_resolver_composite) 2022-12-19 14:59:15 [2,006ms] [Info] [carb] [1180653] ../../../source/framework/PluginImpl.cpp:initialize():512: Initializing plugin: omni.keyvaluedatabase.leveldb (interfaces: [omni::server::IKeyValueDatabase v1.0]) (impl: omni.keyvaluedatabase.leveldb) 2022-12-19 14:59:15 [2,006ms] [Warning] [omni.keyvaluedatabase.leveldb] [1180653] ../../../source/plugins/omni.keyvaluedatabase.leveldb/leveldb.cpp:LevelDBImpl():74: wasn't able to load the meta database, trying to repair it ... 2022-12-19 14:59:15 [2,028ms] [Warning] [omni.keyvaluedatabase.leveldb] [1180653] ../../../source/plugins/omni.keyvaluedatabase.leveldb/leveldb.cpp:LevelDBImpl():79: repair failed 2022-12-19 14:59:15 [2,028ms] [Fatal] [omni.core] [1180653] ../../../source/plugins/common/KvDbAsync.h:Instance():36: error while initializing database 'meta'