120 points by ml_genius 6 months ago flag hide 16 comments
mlfrauddetect 6 months ago next
Excited to see this discussion on Machine Learning Techniques for Real-time Fraud Detection! I recently worked on a project with similar goals, and it's amazing to see how powerful the correct algorithms can be.
curiousmind 6 months ago next
What kind of real-time fraud detection are you working on? Could you elaborate on the challenges of using ML techniques in this context?
mlfrauddetect 6 months ago next
@CuriousMind Sure, one challenge I've faced is ensuring low-latency and model simplicity, so that we don't introduce unneeded overhead in our production systems. Plus, explaining real-time model results to stakeholders is crucial.
handsonml 6 months ago next
Couldn't agree more – it's our responsibility as data scientists to demystify the outcomes of our models. It's about presenting a clear picture without misleading stakeholders into putting too much faith in Black Box outputs.
honestresponse 6 months ago next
Well said! As data scientists, we need to connect model outcomes to stakeholders' mental models. While I think SHAP and LIME are powerful, local interpretability isn't enough. We need to present results in context.
datasciencewiz 6 months ago prev next
Real-time fraud is critical to financial organizations. Common methods involve using decision trees, random forests, or neural networks for anomaly detection and flagging suspicious transactions.
interestedparty 6 months ago next
I agree, latency is key for real-time decisions. Have you tried using lightweight classifiers like logistic regression? I've heard they do well in fraud detection models while ensuring faster speeds.
mlfrauddetect 6 months ago next
@InterestedParty We have evaluated logistic regression models, and they work reasonably well. Our main challenge is adapting to changing fraud patterns, so we use ensembles with the occasional online-learning adjustments.
artificialintrigue 6 months ago next
Ensuring model adaptability to changing fraud patterns is crucial. How do you measure the effectiveness of your online-learning adjustments in the fraud detection model's ensemble?
frustratedfraud 6 months ago next
We monitor the false positive and false negative rates produced by each model in the ensemble. Eventually, we retire old models based on performance degradation or concept drift.
curiousaura 6 months ago next
What is your threshold when determining concept drifts, and how do you continuously ensure your evaluation metrics are solid?
validmetrics 6 months ago next
That's a great question. We typically monitor 90-day mean and variance to detect potential concept drifts. We would then employ a
neutraleval 6 months ago prev next
Incidentally, what monitoring software do you use to guarantee model performance and track concept drifts in production?
devopsenthus 6 months ago next
If you want a good monitoring solution, consider tools like Prometheus and Grafana. They can help you track your models' performance and ensure they adapt to changing environments without missing a beat.
anotheruser 6 months ago next
I'm curious, how do you efficiently incorporate domain knowledge into ML models? In my experience, an understanding of the problem context is key for high-performing and accurate models.
ml_expert 6 months ago next
There are multiple ways, and it depends on the model. Generally, domain knowledge can be incorporated as features or through constraints during training. Have you tried using feature engineering or regularization?