Skip to content

NIP: transfer failure spike

Alert: NipTransferFailureSpike · Severity: Critical · Requires app /metrics

Status: Alert rule exists; fires when nip_inward_results_total is scraped from app /metrics.

What this means

More than 20% of NIP inward results returned a non-00 response code over 10 minutes (with minimum traffic volume). Customers are seeing failed transfers at scale.

Business impact

  • Money movement failing — not a single bad transaction; systemic issue (CBS, SSM, validation, or app bug).
  • May correlate with dependency slowness or worker wedging.

Confirm

# Prometheus (when metrics live)
curl -sG 'http://127.0.0.1:9091/api/v1/query' \
  --data-urlencode 'query=sum(rate(nip_inward_results_total{response_code!="00"}[10m])) / sum(rate(nip_inward_results_total[10m]))'

Grafana: NIP Service Health → response-code panel (when wired).

Postgres (always available):

SELECT final_response_code, COUNT(*)
FROM nip_inwardtransaction
WHERE received_at > NOW() - INTERVAL '1 hour'
GROUP BY 1 ORDER BY 2 DESC;

Fix

  1. Identify dominant response code (07 invalid account, 51 insufficient funds, 96 system error, etc.).
  2. Check CBS / SSM status — see dependency slow.
  3. Check web health — health probe down.
  4. Review app logs in Loki: {platform="nip-bridge", environment="nip"} |= "error".
  5. If code 96 spike — often upstream CBS; coordinate with core banking team.

Escalate if

  • Failure rate stays >20% after 15 minutes.
  • Code pattern suggests CBS outage or credential/SSM failure.
  • NIP Service Health — response codes, traffic
  • Transaction Explorer — sample failed Session IDs