Fraud detection is one of the classic problems in applied machine learning. Financial transactions are massive in volume, but only a tiny fraction of them are fraudulent — often less than 1%. This makes the problem highly imbalanced, and it means that a model that always predicts "not fraud" would still be correct 99% of the time. That is clearly useless, so we need better metrics and better models.

In this short study, we compare three models that are widely used in this domain: Logistic Regression, Random Forest, and XGBoost. The dataset is a standard credit card fraud dataset (284,807 transactions, 492 frauds).

Why these three models?

Each one represents a different family of approaches:

• Logistic Regression — a simple linear model. Fast, interpretable, and a good baseline. It works well when the relationship between features and target is roughly linear.
• Random Forest — an ensemble of decision trees. It handles non-linear relationships, is robust to outliers, and needs very little tuning.
• XGBoost — gradient boosting on trees. Usually the strongest performer on tabular data, but with more hyperparameters to tune.

How we handle imbalance

Instead of training on the raw data, we use two common techniques:

• SMOTE — creates synthetic samples of the minority class.
• Class weights — penalizes mistakes on the minority class more.

We also use stratified k-fold cross-validation to make sure that each fold has the same proportion of frauds, and we evaluate with metrics that are appropriate for imbalance:

Results

The table below shows the average performance of each model on the test set (5-fold cross-validation):

Chart: F1-Score comparison

Chart: Precision vs Recall trade-off

Fraud detection often cares about recall more than precision — missing a fraud is usually more expensive than a false alarm. But if recall is too high, the system flags too many legitimate transactions, which annoys customers.

What this tells us

• Logistic Regression is not useless. It reaches F1 = 0.88 in 2 seconds, with almost no tuning. For many production systems, that's already good enough as a first model.
• Random Forest is the safe middle ground. It handles non-linear patterns, needs almost no feature engineering, and gets close to XGBoost.
• XGBoost is often the best on tabular data, but the gain over Random Forest is modest (F1 +0.02). The cost is more tuning, longer training, and harder interpretability.

Final thoughts

If I had to pick one model to start with, I'd go with Logistic Regression as a baseline, then move to Random Forest if it's not enough, and only reach for XGBoost when I have time to tune it properly. In practice, the biggest gains usually come from better features and better data, not from a fancier model.

---

References

• Dal Pozzolo, A., Caelen, O., Le Borgne, Y. A., Waterschoot, S., & Bontempi, G. (2014). Learned lessons in credit card fraud detection from a practitioner perspective. Expert Systems with Applications.
• Chen, T., & Guestrin, C. (2016). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD.
• Breiman, L. (2001). Random Forests. Machine Learning, 45(1), 5–32.
• Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over-sampling Technique. Journal of Artificial Intelligence Research.
• Credit Card Fraud Detection dataset — ULB Machine Learning Group, Kaggle.