NIP: Spaces archive stale

Alert: NipSpacesArchiveStale · Severity: Critical · Host: monitoring-prod

What this means

No successful archive export in 26+ hours. Cron should run daily at 02:00 UTC; something prevented success or cron did not run.

Business impact

Same as archive failed — growing gap in 7-year cold storage. Multiple days may need backfill.

Confirm

cat /var/lib/node_exporter/textfile_collector/nip_archive.prom
# Check nip_archive_last_success_unixtime — convert:
date -u -d @$(grep last_success_unixtime /var/lib/node_exporter/textfile_collector/nip_archive.prom | awk '{print $2}')

grep nip-spaces /etc/cron.d/nip-spaces-archive
tail -200 /var/log/nip-spaces-archive.log

Fix

1. Fix underlying failure (see spaces archive failed).
2. Backfill missing days: bash set -a && . /etc/pup/nip-spaces-archive.env && set +a for d in 2026-08-04 2026-08-05; do ./scripts/spaces-archive/nip-spaces-archive-export.sh --date "$d" done
3. Confirm metrics show fresh last_success_unixtime.
4. Reload Prometheus if needed: curl -X POST http://127.0.0.1:9091/-/reload

Escalate if

• Cron missing or disabled on server.
• Backfill fails for multiple dates.