I provisioned infrastructure on AWS through Terraform and then provisioned the software with Ansible.
I started with a plain Ubuntu Server 22.04 LTS from canonical and installed all the requirements.
It is running DeepSearch and Nucleus through Docker-compose.
However when installing Farm Headless I meet some challenges.
When Creating a Systemd service I get the following:
journalctl -u farm-agent.service
journalctl -u farm-queue.service
Oct 20 21:58:25 ip-10-0-116-66 bash[3907]: 2023-10-20 21:58:25 [2,463ms] [Error] [omni.kit.app.plugin] failed to open file '/home/farm/.local/share/ov/data/Kit/omni.farm.queue.headless/102.11/user.config.json' - unable to save the dictionary
Oct 20 22:08:27 ip-10-0-116-66 bash[4030]: 2023-10-20 22:08:27 [1,651ms] [Error] [omni.kit.app.plugin] failed to open file '/home/farm/.local/share/ov/data/Kit/omni.farm.agent.headless/102.5/user.config.json' - unable to save the dictionary
Also when running the proper curl commands to verify the integrity of Farm queue it is working fine.
As per the step 6 - Deploying Omniverse Farm on Kubernetes — Omniverse Farm latest documentation
However I am unable to submit jobs (curl -X “POST”) and when running:
curl -H "Accept: application/json" -H "Accept: application/json" http://localhost:8222/queue/management/agents/list
It returns null.
I am extremely confused on how to navigate this problem, any help would be appreciated.
Note: it is running on a private vpc and only access is through AWS SSM so only CLI.