#16 - This one metric predicts if your fraud team is ready to scale

What are the two most contradicting goals of your fraud team?

Mitigating fraud losses while eliminating false positives?

Think again.

It’s how to ensure the best overall performance possible while enabling the organization to scale.

Why are these contradicting?

Simple: being more efficient means spending less time, attention, and resources on each individual case. And this impacts the quality of your decisions.

In a previous issue of TSFS I briefly mentioned the concept of Model Participation.

Today, I’d like to explore this further as a means for managing this conflict.

Automation does not equal scalability

The biggest fallacy I repeatedly come across is that automated decisions enable scalability. No, they just enable real-time operations.

If these decisions are driven by a set of rules, you still need to manually:

• Create new rules all the time

• Monitor your live rules

• Tweak and deprecated degraded rules

If you ever ran an organization that counted on 100 fraud rules or more, you know what a resource sink this quickly turns out to be.

The real leap in scalability happens when your organization transitions to using AI effectively.

Why? Because AI, with all its limitations, is designed to enable decisions on scale and over time.

Degradation still happens, but it’s slower. Monitoring is still needed, but you’re monitoring a handful of models at worst, instead of hundreds of rules.

As such, the utilization level of AI is also the best predictor for handling operations at scale.

The main issue with AI, especially when we acquire it from a vendor, is that it is a black-box. FraudOps teams tend to use it less, and resort instead to rules and other “controllable” means.

But organizations that want to scale, need to actively task, measure, and encourage fraud teams to utilize AI to its fullest.

This is why I find the concept of Model Participation Rate so important. As the saying goes: “What is not measured, is not managed”.

Measuring your AI utilization level naturally leads your team to increase it. Increasing it, naturally leads your org to be better prepared for scaling.

Best of all, you now have the means to make conscious decisions: when to optimize for performance, and when to optimize for scalability.

Calculating Model Participation Rate

The way I calculate Model Participation Rate is straightforward:

Notice that I don’t talk about events (e.g., payments, signups, etc.), I’m talking about decisions.

Let me break it down:

Let’s say I processed 1,000 payments. It’s likely that most of them just went through without any of my controls “touching” them.

But let’s imagine that 100 of these were actively decided upon by one of my logics. It doesn’t matter if they were eventually blocked or approved. We only care if we actively made a decision on that payment.

Next, I will categorize these 100 payments by the source “decision vehicle” - an AI score, a rule, a block/approve list, a manual review, etc.

Now let’s say that 27 of the payments were blocked by my AI score, and 3 more got approved by my AI score. The other 70 cases were processed by other means.

In that case:

Side note: I know. You already have a couple of objections and edge-cases in mind. Please keep your mind open and read forward. All practicalities are solvable, it’s the core principle that matters.

What does your Model Participation Rate reveal about your business?

Try doing a back-of-the-envelope model participation calculation for your business. Chances are it’s quite low. Lower than what you expected at least.

This shouldn’t come as a surprise. After all, if you didn’t measure it so far there’s no reason to believe it to be high.

But now ask yourself this:

What will it take to increase our model participation rate by 20 percentage points? What would be the performance of the business, if we switch to such a strategy tomorrow?

The answers you get represent the trade-off you’re making every day between scaling your business and optimizing your business.

Why? Because it’s likely that switching strategies tomorrow would mean more false positives and/or more losses. After all, if the performance would be better you’d already be doing that.

But it’s not only a question of comparing performance figures. You now know what’s the impact of the AI score you’re paying so much for. Are you still happy with your ROI?

You also know now what will be the impact of removing one-third of your rules and using AI instead.

Sure, the performance might not be so great, but maybe you don’t need one-third of your rule writers any more. Or better yet, you can utilize these resources for when your business grows without needing to hire for more.

Achieving this decoupling - between team size and performance - is exactly what “scale-readiness” means.

And most importantly - this translates back into the performance itself.

Think about it: businesses don’t grow linearly and predictably. Teams do not scale before they have to. And in that period of stretched resources you’re paying with decreased performance.

Teams that streamlined most of their decisions through AI, though, are better prepared and therefore will perform better.

So, do scalability and performance really contradict each-other?

Not if they are measured and managed.

Have questions or feedback? Reply to this email, I read all messages.

In the meantime, that’s all for this week.

See you next Saturday.

P.S. If you feel like you're running out of time and need some expert advice with getting your fraud strategy on track, here's how I can help you:

Fraud Strategy "Power Call" - Book a consultation call with me to get clear, actionable recommendations that fit your budget. Guaranteed.
​Book a Call Now >>​

Fraud Strategy Workshop - are you an early-stage Fintech that needs to move fast and with confidence? Book this 1.5-hours workshop to get instant insight into your vulnerabilities, optimization opportunities, and get clear actionable recommendations that won't burn through your budget.
​Book Your Workshop Now >>​

Fraud Strategy Transformation Program - are you a growth-stage Fintech in need for performance optimization or expansion of your products offering? Sign up to this 6-8 weeks program, culminating in a tailored made, high-ROI roadmap that will unlock world-class performance.
​Schedule a Call Now >>

Enjoyed this and want to read more? Sign up to my newsletter to get fresh, practical insights weekly!