Скачать или смотреть Fraud and Financial Crime Update (Financial Data Model Training)

Thanks for watching our fraud and financial crime update! For more info on financial data model training, machine learning in banking, and fraud risk management, please subscribe to our channel:    / @feedzai.riskops  

CHAPTERS:
0:00 Fraud Detection Model Best Practices: Stratified Sampling
0:53 Separate Data Sets & Time Gaps
1:56 Causes of Performance Drops & Low Model Performance

TRANSCRIPT:
This week, we continue our discussion around best practices when training a fraud detection model.

My name is Roxana and this is your Feedzai Fraud and Financial Crime Update.

The first thing is sampling. Your model is as good as the sample you train the model on. A good sample is representative of the whole data set, so the model can generalize and learn all the patterns and all the behavior.

Since the financial data is very unbalanced, we mostly use some techniques to get more fraud labels.

So stratified sampling is basically keeping all the fraud transactions and then adding non-fraud transactions as needed. So for example, if we want to have 5% fraud transactions in our data set, we keep all the fraud, which is going to be 5% of our training set, and then the other 95% will be gathered from the non-fraud transactions.

The other thing is to make sure to have separate data sets. So we should have a training set, but we should also have one validation set and our test set.

The other thing is that we need to make sure that these training sets, validation sets, and test sets are chronologically ordered. The time is very important in financial data when you want to create a model.

It would be also good to have a gap between each of the data sets that you train, validate, and test the model on, and that's because it helps the model to generalize itself better. If you have a gap between your training set and the test set, you make sure that the model is not biased to the training set and it's learning from other time ranges of the data.

For example, if your training set is from January to March, then we always say, let's keep one month out, and then we use May to June for validation, and then again one month out, and then the next two or three months as the test set.

If we evaluate the model on the validation set and test set and we see that the performance has dropped drastically, it means that the model is biased and is overfitted to the training set.

The other problem is that sometimes you see that the performance is suspiciously low. Suppose you deploy your model in production and everything looks fine, but then suddenly you see a huge drop in the model performance. There are different reasons for that. The first one is the data drift, as we mentioned. It's normal that you see changes in your data values, in your payment behaviors. You can review your sampling process and make sure that you have created a representative sample so that the model can actually generalize itself very well.

Next time we're going to talk about measures and metrics. Thanks for watching. This was your Feedzai Fraud and Financial Crime Update.