crawler_alert, monitoring: AI crawler metrics and dashboards
Some checks failed
deploy / ansible (push) Failing after 1m17s
Some checks failed
deploy / ansible (push) Failing after 1m17s
crawler-alert.py --metrics (5-minute timer) reads what the Traefik access log gained since the last run, judges every request that names an AI bot exactly like the weekly report (vendor IP ranges, forward- confirmed rDNS, Meta's ASN) and writes cumulative counters to the node_exporter textfile directory. Verdicts are cached per bot and IP for a week, so DNS only sees new addresses; range files are refetched once a day. It covers every router, while the mails keep watching Forgejo only. New dashboard "AI crawlers", and a row of it on the overview. The overview's load gauge divided node_load1 by a count with fewer labels, which matches nothing; aggregate both sides by host. Assisted-by: Claude:opus-5-5
This commit is contained in:
parent
3b25f4ae25
commit
bc0dac9188
7 changed files with 1924 additions and 6 deletions
|
|
@ -22,6 +22,8 @@
|
|||
- crawler-alert.timer
|
||||
- crawler-weekly.service
|
||||
- crawler-weekly.timer
|
||||
- crawler-metrics.service
|
||||
- crawler-metrics.timer
|
||||
notify: Reload systemd
|
||||
|
||||
- name: Flush handlers
|
||||
|
|
@ -35,3 +37,4 @@
|
|||
loop:
|
||||
- crawler-alert.timer
|
||||
- crawler-weekly.timer
|
||||
- crawler-metrics.timer
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue