Analytics run amok?

Joe McKendrick pointed me to a story on ABC News today – ‘Good Morning America’ Gets Answers: Some Credit Card Companies Financially Profiling Customers. The story is worrying and has some good reporting in but, as usual, the analytics behind the whole thing are poorly described.

So, first things first. The headline gets us off on the wrong foot – of course credit card companies are financially profiling customers. They have to manage their risk and most banking regulations require decent financial profiling to ensure the bank does not get saddled by bad debt. This is not the story, though it makes a great headline for the analytically unsophisticated. The reporting gets better with the first real fact of the piece:

A new policy being used by at least one major credit card company judges a shopper not necessarily by his credit purchases and payments alone, but also by the fiscal behavior of the fellow shoppers in the stores he visits.

This is the meat of the issue. In the human interest story that, of course, leads the piece AmEx is quoted as saying that they have reduced someone’s credit line because:

“…

A new policy being used by at least one major credit card company judges a shopper not necessarily by his credit purchases and payments alone, but also by the fiscal behavior of the fellow shoppers in the stores he visits.

This is the meat of the issue. In the human interest story that, of course, leads the piece AmEx is quoted as saying that they have reduced someone’s credit line because:

“Other customers who have used their card at establishments where you recently shopped have a poor repayment history with American Express.”

and this is new.

Now let’s get a couple of things straight. First this is “behavioral scoring” as the article asserts but so are lots of other, more appropriate uses of analytics. Behavioral scoring is also not, despite what the article says, new in any way. Behavioral scoring simply means using behavior data – what you do once you become a customer – rather than just the application data you submitted. Behavioral scoring is much more accurate because your actions are a great predictor of your future behavior. These kind of scores are much more accurate than those based only on your application and are used for credit line decisions, early stage collections and much more. Any time your credit card company changes something about your card it is almost certainly driven by a behavior score. And this is fine – after all you are being judged by your actions.

Where the worrying stuff starts, and where I think the analytics may have run amok, is in the kinds of data being used in these new scores.

Credit cards tied to retailers have long used the information they have about your purchase patterns at their store in their behavior scores. For instance, one company I know classified its products into “cheap version”, “regular version” and “deluxe version”. If you always bought the cheap versions that said something different about you than if you always bought the deluxe one. Of course what it says – short of cash or just careful with your money – is complex and so including it in a behavior score was non-trivial. Again this still seems reasonable – after all it is your behavior that is being used.

It seems though that:

banks may now be using data collected by customers to compare them to other shoppers at individual retail locations or by zip code

and this seems to me to be a step too far. This is not judging your risk by your behavior but judging your risk by the risk of others with whom you have an incidental association. As any insurance company will tell you, just living in a “dangerous” zip code does not automatically make you a bad risk nor living in a “safe” one make you a good risk. Similarly, shopping at a store where other shoppers have problems does not mean you do. The correlation here seems weak and the causation non-existent. Bad idea.

I think banks need to stick to “behavioral scoring” and avoid this kind of “associative scoring” as customers can see how their own behavior does and should affect their credit but they don’t see why the behavior of others should affect it. And nor do I.