ansible: monitoring with Prometheus, Loki and Grafana
All checks were successful
deploy / ansible (push) Successful in 7m23s
All checks were successful
deploy / ansible (push) Successful in 7m23s
Nothing watched the servers: tonight's asxpio crash loop was found by hand. On asxpio-ops, Prometheus (30 days), Loki (14 days) and Grafana at gf.asxp.io (WireGuard only), with blackbox_exporter probing every service from outside and postgres_exporter. On both hosts node_exporter and cAdvisor on the VPC address, and Alloy shipping container logs, the journal and Traefik's access log to Loki over the VPC. Traefik and Forgejo expose their metrics; restic and the bucket replica record their last success for node_exporter. Dashboards: an overview as home, plus hosts, containers, Traefik, probes, Postgres and logs. Alerts by email to me@asxp.io: host or service down, certificate under 14 days, disk or memory nearly full, a container restart-looping, Postgres down, backup or replica stale. Assisted-by: Claude:opus-5-5
This commit is contained in:
parent
5289542dca
commit
e20f5a7d07
28 changed files with 24705 additions and 161 deletions
17
ansible/roles/monitoring/templates/alerting.yaml.j2
Normal file
17
ansible/roles/monitoring/templates/alerting.yaml.j2
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
apiVersion: 1
|
||||
contactPoints:
|
||||
- orgId: 1
|
||||
name: email
|
||||
receivers:
|
||||
- uid: email-me
|
||||
type: email
|
||||
settings:
|
||||
addresses: {{ monitoring_alert_email }}
|
||||
singleEmail: true
|
||||
policies:
|
||||
- orgId: 1
|
||||
receiver: email
|
||||
group_by: [alertname, host]
|
||||
group_wait: 1m
|
||||
group_interval: 5m
|
||||
repeat_interval: 4h
|
||||
Loading…
Add table
Add a link
Reference in a new issue