4.2.0
4.1.0
4.0.0
2023.1.1
2023.1.0-hotfix.1
Other (please specify):
Operating System
Ubuntu 22.04
Ubuntu 20.04
Windows 11
Windows 10
Other (please specify):
Topic Description
Detailed Description
After installing Local Nucleus Service 2023.2.5 it starts Search Servicev.3.2.10-1033 as omni-search-service, which eventually will kill SSD because it runs ~5M/s disk IO non stop forever (followed this behavior for a week on a number of local servers 24/7).
I would recommend for all users to disable it from startup using config panel in http://localhost:3080/
Thank you for reporting this issue with the omni-search-service in Isaac Sim. We appreciate your detailed description and the steps to reproduce the problem. I understand this is causing significant disk I/O and potentially impacting SSD lifespan, which is a serious concern.
We’ll update this thread as soon as we have more information for the issue.
We have not seen this issue before. Can we please get your Nucleus and Search Service logs located in~/.nvidia-omniverse/logs/
Also, can we please get some more information about your configuration?
Is the data path for Nucleus on the same system drive as Nucleus is installed?
Had you seen any issues like this on previous installations of Nucleus on your system?
How large is the dataset that exists on your Nucleus server?
Do you have access to AWS s3 in your network?
Nucleus main log is pretty big:
Access.log is 308MB
this one is short omni.server.app.log - omni.server.app.log (162.0 KB)
Search Service is stopped just after I have noticed this, so it doesn’t produce logs anymore. If I start it from web config panel it starts IO writes again.
Is the data path for Nucleus on the same system drive as Nucleus is installed? I did clean install of full system using launcher omniverse-launcher-linux.AppImage, with all defaults. So it should be as default configured in it.
Had you seen any issues like this on previous installations of Nucleus on your system? I noticed this after last installations, I think from time when isaac-sim 4.2.0 appeared
How large is the dataset that exists on your Nucleus server? ~3GB
Do you have access to AWS s3 in your network? It is localhost Nucleus on the same machine with Isaac-sim and Create. But from logs I can see that Search Service spams +/NVIDIA/... something.
If Search Service caches thumbs .png from /NVIDIA… then maybe the file list is so long that it takes weeks?
In addition stopped Thumbnailv.1.5.11-17670007 and Tagging Servicev.3.1.26-17325333 as they continuously load CPU. Interesting that ctrl+s in Isaac-sim and Create still generates thumbnails for my usd files. So maybe this part could be explained more, for what reason they exist and work non-stop in background as services.
So, that /NVIDIA directory is actually a mount to s3. So, if you have inconsistent or low bandwidth to AWS s3, this could cause search to take a long time to populate (You can unmount this by right->unmount).
Thumbnail and tagging will use some small amount of CPU load, but it should be negligible.
To get a bit more information
What OS are you running on, and what is your hardware spec? Please look at our sizing guide to understand what’s needed.
Thumbnail and tagging will use some small amount of CPU load, but it should be negligible. I don't agree. They non stop use sum of ~5% of CPU, which doesn't have any reasonable explanation.
I will do more measurements of system load under each of services and update.
