iac/ansible

Ansible

site.yaml is the desired state of both servers and is applied by the deploy pipeline on every push to main. The playbooks under playbooks/ are host bootstrap and are run by hand.

Running

echo '<vault password>' > .vault_pass && chmod 600 .vault_pass

ansible-playbook site.yaml --check --diff       # dry run, over WireGuard
ansible-playbook site.yaml                      # apply
ansible-playbook site.yaml --tags forgejo       # one role

On NixOS: nix shell nixpkgs#ansible -c ansible-playbook ….

Templates that carry secrets are never shown in --diff output.

Inventories

• inventory/hosts.yaml: default. Connects as root to the WireGuard addresses.
• inventory/ci.yaml: used by the pipeline. Connects as deploy over the private network, reaching asxpio-services through asxpio-ops.

known_hosts pins both servers' SSH host keys for the pipeline.

Variables

inventory/
├── group_vars/all/
│   ├── network.yaml         # VPC and WireGuard addresses
│   ├── paths.yaml           # /opt
│   ├── s3.yaml              # object storage buckets and s3fs mounts
│   ├── wireguard.yaml       # WireGuard peers
│   └── vault.yaml           # encrypted secrets
├── group_vars/ops_servers/
│   ├── traefik.yaml         # traefik_version
│   └── forgejo.yaml         # forgejo_version, domain and aliases
├── group_vars/services_servers/
│   ├── forgejo_runner.yaml  # runner version, labels' nix image
│   └── minecraft.yaml
└── host_vars/               # per-host addresses, SSH and s3fs settings

Compose files without variables are plain files under roles/<role>/files/; edit the image tag there.

Roles in site.yaml

base, traefik, crawler_alert, forgejo, forgejo_runner, sillytavern, minecraft, xmpp, xray, restic_backup. Service roles go through docker_service, which writes the files, runs docker compose up -d, restarts on a config change and waits for the service to answer.

crawler_alert installs a copy of traefik-crawlers-statistics; update roles/crawler_alert/files/ from there.

Bootstrap playbooks

Playbook	Purpose
01-wireguard-setup.yaml	WireGuard hub on asxpio-ops, peer on asxpio-services, client configs
02-ssh-hardening.yaml	restrict SSH to WireGuard and the private network
03-setup-s3fs.yaml	s3fs mounts on asxpio-services (off on asxpio-ops)
04-add-user-zerrtika.yaml	add a user to running servers
05-swap.yaml	2 GB swapfile on asxpio-ops

Always dry-run these first with --check --diff.

• 03-setup-s3fs.yaml rewrites /etc/s3fs/.passwd-s3fs, which the backup script on asxpio-ops also reads.
• On asxpio-services the WireGuard interface is named asxpio-services, not wg0. Check 01-wireguard-setup.yaml against the host before re-running it there.

Runner notes

The nix-latest label runs nixos/nix:<forgejo_runner_nix_version> with a nix-store-<version> volume over /nix. To bump it: change the variable, let the pipeline apply it, then remove the old volume.

Runner and nix image upgrades

forgejo_runner_version and forgejo_runner_nix_version are bumped by Renovate PRs. Applying either one recreates the runner while it runs the very job that applies it:

• that run ends as failed (runner 13; cancelled on older runners) after everything else has been applied, its log stopping in the forgejo_runner role, and its job container stays behind on asxpio-services (docker ps | grep FORGEJO-ACTIONS-TASK); remove it;
• the next run confirms the result. The workflow only triggers on changes under ansible/ (or workflow_dispatch in the web UI), so an empty commit starts nothing;
• after a nix bump, remove the old nix-store-<old version> volume.