ansible: monitoring with Prometheus, Loki and Grafana

Browse source

Nothing watched the servers: tonight's asxpio crash loop was found by
hand. On asxpio-ops, Prometheus (30 days), Loki (14 days) and Grafana
at gf.asxp.io (WireGuard only), with blackbox_exporter probing every
service from outside and postgres_exporter. On both hosts node_exporter
and cAdvisor on the VPC address, and Alloy shipping container logs, the
journal and Traefik's access log to Loki over the VPC.

Traefik and Forgejo expose their metrics; restic and the bucket replica
record their last success for node_exporter. Dashboards: an overview as
home, plus hosts, containers, Traefik, probes, Postgres and logs.
Alerts by email to me@asxp.io: host or service down, certificate under
14 days, disk or memory nearly full, a container restart-looping,
Postgres down, backup or replica stale.

Assisted-by: Claude:opus-5-5

...

This commit is contained in:

Sergei Poljanski 2026-11-13 04:59:11 +04:00

parent 5289542dca

commit e20f5a7d07

Signed by: asxpi

GPG key ID: 4F8851660FA4121B

28 changed files with 24705 additions and 161 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

3131

ansible/roles/monitoring/files/dashboards/Services/postgres.json Normal file

View file

File diff suppressed because it is too large Load diff

1071

ansible/roles/monitoring/files/dashboards/Services/probes.json Normal file

View file

File diff suppressed because it is too large Load diff

1527

ansible/roles/monitoring/files/dashboards/Services/traefik.json Normal file

View file

File diff suppressed because it is too large Load diff