From 59415515a155798722f51990e51c707609b5fe7e Mon Sep 17 00:00:00 2001 From: Sergei Poljanski Date: Thu, 12 Nov 2026 21:03:10 +0400 Subject: [PATCH] docs: restic backups, XMPP and Xray on services, Fossils retired Describe the new roles, how to list and restore restic snapshots, and that the restic password must be kept outside the servers. Assisted-by: Claude:opus-5-5 --- README.md | 25 +++++++++++++++++++++++-- ansible/README.md | 3 ++- terraform/README.md | 3 ++- 3 files changed, 27 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 8993dbc..201b18f 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ Ansible. Pushing to `main` deploys: a Forgejo Actions workflow applies | Server | Size | Runs | |--------|------|------| | asxpio-ops | 2 vCPU, 4 GB | Traefik, Forgejo, crawler-alert; app stacks deployed from their own repos | -| asxpio-services | 8 vCPU, 16 GB | Forgejo Actions runner, Minecraft (Paper and Fossils), SillyTavern | +| asxpio-services | 8 vCPU, 16 GB | Forgejo Actions runner, Minecraft (Paper), SillyTavern, XMPP (asxp.io), Xray | Both are reachable for administration only over WireGuard (10.20.0.0/24) or the private network (10.10.0.0/20). @@ -31,7 +31,11 @@ Both are reachable for administration only over WireGuard | `forgejo` | ops | Forgejo at git.asxp.io (git.czsk.it is an alias), SSH passthrough | | `forgejo_runner` | services | Actions runner and its job labels | | `sillytavern` | services | SillyTavern on the WireGuard address | -| `minecraft`, `minecraft_fossils` | services | game servers, daily restart timer | +| `minecraft` | services | Paper server | +| `minecraft_fossils` | services | retired: keeps the Fabric server stopped; world left in `/opt/minecraft-fossils/data` | +| `xmpp` | services | ejabberd, apushd and tackygram for asxp.io; secrets stay in `/opt/xmpp/state` | +| `xray` | services | Xray VLESS Reality on :443, config in the vault | +| `restic_backup` | both | nightly restic backup to object storage, hel1 and fsn1 | Not managed here: the app stacks on asxpio-ops (asxpio, datacenterbingo, blog, czsk, vhpbot, storage). Each deploys from its own repository into @@ -47,6 +51,23 @@ blog, czsk, vhpbot, storage). Each deploys from its own repository into A run with nothing to change touches nothing and restarts nothing. +## Backups + +Hetzner's server backups are off. Each host runs `restic-backup.timer` +nightly (ops 03:30, services 04:15 UTC) into two independent +repositories, `restic/` in `asxpio-backups-main` (hel1) and +`asxpio-backups-repl` (fsn1). Databases go in as dumps or online +copies; see `restic_pre_hooks` in `ansible/inventory/host_vars/`. +A failed run mails its journal. + +```sh +restic-backup hel1 snapshots +restic-backup fsn1 restore latest --target /tmp/restore --include /opt/xmpp +``` + +The password is `vault_restic_password` in the vault. Keep a copy +outside the servers and this repo's CI: without it nothing can be read. + Things to know: - **Game servers** are saved and stopped with a 120 s timeout before a diff --git a/ansible/README.md b/ansible/README.md index e581645..073cd2c 100644 --- a/ansible/README.md +++ b/ansible/README.md @@ -52,7 +52,8 @@ Compose files without variables are plain files under ## Roles in site.yaml `base`, `traefik`, `crawler_alert`, `forgejo`, `forgejo_runner`, -`sillytavern`, `minecraft`, `minecraft_fossils`. Service roles go +`sillytavern`, `minecraft`, `minecraft_fossils` (retired, keeps it +stopped), `xmpp`, `xray`, `restic_backup`. Service roles go through `docker_service`, which writes the files, runs `docker compose up -d`, restarts on a config change and waits for the service to answer. diff --git a/terraform/README.md b/terraform/README.md index b5a4cba..62c5cab 100644 --- a/terraform/README.md +++ b/terraform/README.md @@ -7,7 +7,8 @@ Hetzner Cloud infrastructure managed via Terraform. - **asxpio-ops** (CX23: 2 vCPU, 4GB) - Traefik, Forgejo, app stacks - **asxpio-services** (CX43: 8 vCPU, 16GB) - CI runner, game servers, SillyTavern -Both have Hetzner daily backups plus delete and rebuild protection. +Both have delete and rebuild protection. Hetzner's server backups are +off; restic backs up the data instead (see ../README.md). Both servers carry `prevent_destroy` and ignore changes to `user_data`, `image` and `ssh_keys`: cloud-init only runs at first boot, and hcloud