Challenges Installing Farm Headless

I provisioned infrastructure on AWS through Terraform and then provisioned the software with Ansible.

I started with a plain Ubuntu Server 22.04 LTS from canonical and installed all the requirements.

It is running DeepSearch and Nucleus through Docker-compose.

However when installing Farm Headless I meet some challenges.

When Creating a Systemd service I get the following:

journalctl -u farm-agent.service
journalctl -u farm-queue.service

Oct 20 21:58:25 ip-10-0-116-66 bash[3907]: 2023-10-20 21:58:25 [2,463ms] [Error] [] failed to open file '/home/farm/.local/share/ov/data/Kit/' - unable to save the dictionary
Oct 20 22:08:27 ip-10-0-116-66 bash[4030]: 2023-10-20 22:08:27 [1,651ms] [Error] [] failed to open file '/home/farm/.local/share/ov/data/Kit/' - unable to save the dictionary

Also when running the proper curl commands to verify the integrity of Farm queue it is working fine.
As per the step 6 - Deploying Omniverse Farm on Kubernetes — Omniverse Farm latest documentation

However I am unable to submit jobs (curl -X “POST”) and when running:

curl -H "Accept: application/json" -H "Accept: application/json" http://localhost:8222/queue/management/agents/list

It returns null.

I am extremely confused on how to navigate this problem, any help would be appreciated.

Note: it is running on a private vpc and only access is through AWS SSM so only CLI.