Introduction to Automated Review Policy
Last week, we received a patent for our Automated Review Policy. Automated Review Policy, or ARP, is a way to look at “risky” data and remove the cases that aren’t, in fact, risky for the business. So, in other words, it’s a way to “Accept” more. We were excited about it when we first deployed it, and are as excited about it now, because it is proven to solve some of the issues that risk and compliance teams have as they try to close doors to bad actors while maximizing their business, and lowering their operational cost.
In general, fraud prevention systems compare data (e.g. payment transactions, account origination information) against models that convey either positive or negative results with the goal of making an automated risk assessment. The models may include deterministic statements (e.g. rule systems), or statistical (e.g. statistical anomaly detection), AI (e.g. supervised machine learning, neural networks), or simple risk scoring (e.g. values assigned to risk conditions). The more specific the model, the more accurate the results, meaning fewer false positives. However, the more specific the model is, the more likely you’ll miss risk conditions or get false negatives. So, as always, the key is to achieve a balance between how much you catch and how much you are willing to let go. No secrets here.
This is no different than how Intrusion Detection Systems or antivirus software work in network and computer security. There is a signature (e.g. model), and by looking at network traffic or computer behavior, the system detects it or not. Small variations of attack vectors are usually undetected until a new signature is available (and thus the reason for keeping the antivirus database up-to-date is vital).
Negative models are also based on past behaviors, in general, these systems are not very effective in catching new risk, so how effective they are is also based on how quickly they can intake new risk models.
Positive models are based on expected behavior; this can be prescriptive based on how an application is supposed to work, or they can be based on historically normal behavior, presenting some very similar practical issues as negative-model based systems.
To no one’s surprise, experienced risk managers understand that a combination of these approaches is actually more effective in practice. The most common and simple implementation of this combined model is to have a “white list” of users or users who are “good” or whose “risky” behavior is understood. A more advanced implementation is embedding good indicators within the risk scoring model, such that it counterbalances the bad risk indicators. And yet, a far more advanced option is to process all data (good and bad) using AI. However, because of the complexities of good feature engineering in data science, usually the scope is reduced to only bad data.
Let’s take a look at how IdentityMind implements a combination of these approaches into its fraud prevention framework.
The IdentityMind Fraud Prevention Framework
IdentityMind’s fraud prevention framework is divided into three stages:
1) Identity Risk. Data is analyzed to assess the risk of the user behind the transaction. It includes historical analysis of the identity parameters within the IdentityMind platform, identity link analysis to uncover synthetic identities, Account Takeover (ATO) detection, the likelihood of stolen identities, etc. This stage includes both positive (e.g. trusted reputation) and negative aspects of identities (e.g. bad, suspicious reputations, and likelihood of identity fraud). We have described at length our eDNA™ technology, and you can find more information in the following resources:
2) Risk Detection System. It runs a set of prescriptive models looking to highlight specific risk patterns and or business policy patterns. Within the rule system, you can build your own risk scoring to complement the model system and catch those cases that, although not specific enough for a model, gobble up enough risk conditions to be highlighted. This is a pure “negative” modeling stage. You can learn more about the rule system and its capabilities in the following resource:
3) Automated Review Policy (ARP). ARP runs a set of models that are specifically aimed to analyze data identified as risky (flagged in the previous stage) and to try to remove those cases that are likely acceptable to the business. This stage is a pure “positive” modeling stage.
The purpose of this article is to understand more about our Automated Review Policy.
Automated Review Policy (ARP)
The Reason Behind ARP
Let’s start by discussing why we separate the “negative” from the “positive” models.
Fraud models degrade over time. Performance degradation can come from many angles but, in general, fraudsters adapt, risk evolves, and so forth. So, in order to maintain a model, these need to be updated periodically (within practical reason) to deal with new and old cases. Some models will become irrelevant with time. For example, in e-commerce, certain products become hot items for a period of time, but then that risk dissipates in favor of other hot products. However there is always the notion of “hot products,” so you can update the “hot products” list and apply some version of the same model; the key is to make sure your list of hot products is updated constantly. There was a time when looking at differences between shipping and billing addresses was a good risk indicator, now, not so much. Unfortunately, many companies have a legacy of fraud models in their system that are likely irrelevant, and that are only there for historical purposes.
So imagine that you have multiple models running at the same time; large enterprise clients may have hundreds or thousands running at the same time. And you want to apply positive aspects to your negative models. Do you need to update all models? It may sound crazy, but yes, sometimes, in order to combine positive and negative, you’d need to update, at least those that are meaningful to the day-to-day operations. This may already be too many.
Then there is the issue of performance. The more models, and the more complex the models are, the longer they take to run. So, yes, in the world of cheap computing, one can argue you throw computing power at it. But any fraud manager and risk officer will agree that going to management and asking for more computers (computing power) isn’t necessarily the path they’d like to take. Vendors, including ourselves, will try to work with clients to find the optimal set of models for the risk they are trying to address, because there are always operational limitations. So, both clients and vendors will try to minimize the number of models that are running at any given time.
By separating the negative and the positive models you are likely, at least to end up with less complex models, removing the performance issue… but wait! Wouldn’t this create more models?
It certainly could. In software there is this argument between models and abstractions. In simple terms, models are specific and abstractions are general. So, the more abstract a definition is, the more examples exist that meet that abstraction. Models are the opposite: you are trying to be as specific as possible, so there are fewer examples of that model. These are two ends of the spectrum. Imagine that you are trying to define a chair, an abstraction is “An object you use to sit on.” If you define it like that, then suddenly a couch meets the definition, and so it does a table, so does a bucket, etc. You get the point. A model would be “An object you can sit on,” “it has legs,” “It can support over 300 lbs,” “it fits one person,” etc. You get the point. The more specific the model of the chair is, the less examples of a chair you can think about.
In fraud terms, you can say, for example, that an abstraction of a stolen credit card can be uncovered by having a shipping address different from the billing address associated with the card. Well, yes, that is certainly one component, but no one in their right mind would write a fraud model that is only looking at this difference to analyze a stolen credit card. This would be an abstraction. A model would need to include many other aspects for it to be remotely practical. You may include, for example, billing and shipping distance, IP geolocation distance to both billing and shipping, transaction history, device fingerprint, bank’s response, etc. However, the more items you include to the fraud model, the more specific it is, and thus, the more susceptible to false negatives.
So, in risk, as in software, the more specific the model is, the fewer examples you are going to catch, and that is the reason why you need to find a balance in your models. So, the trick is finding the right level of specificity, such that you have a better balance between false positives and false negatives. And here is where the other benefit of separating the positive and the negative is: the level of specificity you need for each one is ostensibly less, and therefore your models, theoretically, tend to be simpler, and as a corollary they are (should be) fewer in numbers.
The implementation of this thought process is actually unique to IdentityMind for the purposes of risk mitigation and fraud detection. As such, we have recently been awarded patent No 10,356,099 by the USPTO.
ARP in the IdentityMind Platform
The ARP functionality is available within the configuration of the risk profile, and each profile has its own ARP configuration.
Models can be complex or very simple (in their simplest form, you can think of them as filters). These are some examples of models available within the ARP configuration:
User’s Consistent Good History
Many times, we get asked what is in our reputation concept for trusted identities. And, well, it is a bit of a secret sauce. Identities with trusted reputation are indeed very accurate — only 0.07% of the time, an identity with a trusted reputation is associated with some form of risk. We tend to be very strict in this concept and some of our clients wanted to be a bit more relaxed and define their own concept of good behavior. So, we exposed a model that allows our clients have their own definition (with certain boundaries).
This model is usually better leveraged by a client in the e-commerce world. It basically represents clients that should be considered historically good, and therefore represent less of a risk to you. The current model looks at:
- Chargeback count. Number of chargebacks associated with the identity.
- Payment Days. The number of days the payment instrument has been seen in the system.
- Payment Transaction Count. Number of transactions associated with the payment instrument (e.g. credit card, bank account, etc) in the transaction being evaluated.
- User Account Days. How long has this identity has been in the system.
- User Account Transaction Count. How many transactions have been associated with the user.
E-commerce clients have found this model adequate to express the “goodness” of their customers, and therefore remove certain risk conditions when associated with these users.
The graph score is a supervised learning machine learning model that scores the identity graph based on a set of features that measure the complexity of the graph. In general terms, graphs above 90 are considered very low risk, scores between 70 and 90 are considered neutral, and below 70 are high risk. So, for every transaction considered for manual review by the fraud models, if the score is higher than the configured value (90 by default), then those transactions would be accepted. In other words, you may want to add an ARP configuration to accept every transaction with a graph score above 90.
The risk score is a framework that allows every client to basically develop a scoring system based on the risk indicators. For example, you can assign a “20” point value to a discrepancy between shipping and billing address, “20” points when the IP fraud score is “High,” and so forth. This rule would allow you to say that if the overall risk score is less than, say, “30” then the risk of the transaction is low, and therefore, should be accepted. The platform is very flexible in how to use these models.
KYC Previously Accepted
This rule takes a previous decision on a client’s onboarding process, specifically related to sanctions screening, and reuses the results of the previous analysis when flagging for sanctions again. We have discussed this at length in our “Reducing False Positives in Sanctions & PEP Screening” video.
At the moment, models for ARP are written only by us. Within the user interface, you can configure the ones available. If you are in need of other models that you’d like to implement, please reach out to your Client Success Manager, and we can certainly get it into the process.
In summary, ARP:
- Focuses on accepting transactions that likely pose no negative impact to the business, but could be considered risky otherwise
- Complements IdentityMind’s risk mitigation technology, allowing for a combination of positive and negative risk evaluation approaches that reduce the overall complexity of risk modeling
- Reduces the overall percentage of manual reviews by making the overall process more accurate
In a follow up post, we will discuss performance data coming from our clients’ implementation and usage of ARP.