#03 - Are you tracking these KPIs?

Here's a common issue I see very often:

I'm having a call with a prospective client and they are complaining about fraud starting to be a pain point.

At this point, I will ask something along the lines of "Can you tell me how you measure this pain? What's the current figure?"

I'm expecting the prospect to tell me their chargeback rate has gone up by 50bps or that their approved daily dollar volume went down by 2%.

But what I hear most often is some mumbling, accompanied by a confused look and a shrug for good measure.

It's always surprising to meet teams that build financial products, but have no grip on their own numbers. Surprising, but actually very common.

If it was just about that moment of awkwardness, I wouldn't care much, but the implications are concerning.

If you don't track fraud-related KPIs, you:

1. Identify fraud spikes too late (leaving money on the table)

2. Take too much time to resolve it (leaving money on the table)

3. Block users indiscriminately (leaving money on the table)

Bottom line - if it wasn't clear - you're leaving money on the table.

Assuming that's an outcome you want to avoid, let's talk about the KPIs you should track.

Oh, and good news - it's really not a lot.

Business Health

These are the "bread and butter" KPIs you want to establish as soon as possible. They'll give you a sense of how your business is doing and how fraud impacts your P&L.

Track these KPIs on a weekly basis. Ideally, you track them WoW and YoY to make sure you take seasonality into account.

Additionally, each KPI should be tracked in two versions - count (number of events) and volume (dollar amount, when applicable).

Fraud Rate: Number of fraud cases (by original event date) divided by number of events. You can segment this further by regions, payment methods or other flows.

Approval Rate: Number of approved events divided by number of all events (including failed attempts). Segments as above.

System Health

These metrics will track how your system is doing. It's helpful to track it on a weekly basis as well, or monthly at the very least. Tracking these metrics will ensure your system is behaving as expected.

Introducing faulty logics that harm the business is more common than we'd all like to think.

As with the business health KPIs, make sure you account for seasonality as well as measuring both count and volume dimensions for each metric.

Decline Rates: You're already tracking approval rates, so why track decline rates separately? Answer is - there are almost always multiple actors that block events. These can be external (e.g., your fraud vendor, acquirer, or the customer's issuer) but also internal (e.g., AI model, rules, blocklists, manual reviews, etc.). Have them all tracked separately so you can easily uncover misbehaviors.

Rule Decline Rates: Following the above, make sure you monitor each rule separately, and not just the aggregated results. If you wake up one day to see your approval rates dropping by 40%, this will help you find the culprit in no-time in 90% of the cases.

Error Rates: Let me be clear right away - you don't need to track all possible error rates, there can be thousands of those. But think about possible user-induced technical errors such as invalid card details, login failures, etc. Tracking these can show you when your business is under attack, even if many of the attempts have failed.

Anomaly Detection

The problem with fraud is that until enough of it matures for you to notice it, it might be too late. New fraud trends can easily take a month for most businesses to spot.

Segmenting your fundamental business/product metrics can be very helpful with early fraud detection. It can expose abnormal behavior and allow you to investigate early on.

The best Fintechs I came across had a robust product analytics infrastructure that allowed them to effectively stop fraud, even if their actual fraud prevention acumen was pretty low.

Ideally, you track these on a daily basis. This sounds excessive, but really, these should be core metrics your product team is looking at anyway.

Here are some examples. Try and think how you can apply them to your specific business.

Amount bands: Whether you're processing payments, transactions, or loans - segment your flow by amount bands. Fraud will almost always show up here first.

Events: Aggregate your events daily (count & amount where relevant). This should include contact details changes, password recovery flows, etc. Super useful when your system is used for money laundering, testing and collusion.

Flows: Have different regions? Operating with several partners? Running two applications and a website? Cross-cutting these flows with the above dimensions can help you notice targeted attacks faster.

TL;DR

Tracking some basic metrics early on can save you a world of pain. You want to focus on:

1. Business Health: What is your financial cost of fraud?

2. System Health: Is my system behaving as expected?

3. Anomaly Detection: Can I see early signs of a new attack?

Have these all covered?

Great! Now you can focus on extending granularity, data freshness, and automatic alerting.

That’s all for this week.

See you next Saturday.

P.S. If you feel like you're running out of time and need some expert advice with getting your fraud strategy on track, here's how I can help you:

Fraud Strategy Workshop - are you an early-stage Fintech that needs to move fast and with confidence? Book this 1.5-hours workshop to get instant insight into your vulnerabilities, optimization opportunities, and get clear actionable recommendations that won't burn through your budget.

​Book Your Workshop Now >>​

Fraud Strategy Transformation Program - are you a growth-stage Fintech in need for performance optimization or expansion of your products offering? Sign up to this 6-8 weeks program, culminating in a tailored made, high-ROI roadmap that will unlock world-class performance.

​Schedule a Call Now >>

Enjoyed this and want to read more? Sign up to my newsletter to get fresh, practical insights weekly!