120 points by ml_genius 1 year ago flag hide 16 comments
mlfrauddetect 1 year ago next
Excited to see this discussion on Machine Learning Techniques for Real-time Fraud Detection! I recently worked on a project with similar goals, and it's amazing to see how powerful the correct algorithms can be.
curiousmind 1 year ago next
What kind of real-time fraud detection are you working on? Could you elaborate on the challenges of using ML techniques in this context?
mlfrauddetect 1 year ago next
@CuriousMind Sure, one challenge I've faced is ensuring low-latency and model simplicity, so that we don't introduce unneeded overhead in our production systems. Plus, explaining real-time model results to stakeholders is crucial.
handsonml 1 year ago next
Couldn't agree more – it's our responsibility as data scientists to demystify the outcomes of our models. It's about presenting a clear picture without misleading stakeholders into putting too much faith in Black Box outputs.
honestresponse 1 year ago next
Well said! As data scientists, we need to connect model outcomes to stakeholders' mental models. While I think SHAP and LIME are powerful, local interpretability isn't enough. We need to present results in context.
datasciencewiz 1 year ago prev next
Real-time fraud is critical to financial organizations. Common methods involve using decision trees, random forests, or neural networks for anomaly detection and flagging suspicious transactions.
interestedparty 1 year ago next
I agree, latency is key for real-time decisions. Have you tried using lightweight classifiers like logistic regression? I've heard they do well in fraud detection models while ensuring faster speeds.
mlfrauddetect 1 year ago next
@InterestedParty We have evaluated logistic regression models, and they work reasonably well. Our main challenge is adapting to changing fraud patterns, so we use ensembles with the occasional online-learning adjustments.
artificialintrigue 1 year ago next
Ensuring model adaptability to changing fraud patterns is crucial. How do you measure the effectiveness of your online-learning adjustments in the fraud detection model's ensemble?
frustratedfraud 1 year ago next
We monitor the false positive and false negative rates produced by each model in the ensemble. Eventually, we retire old models based on performance degradation or concept drift.
curiousaura 1 year ago next
What is your threshold when determining concept drifts, and how do you continuously ensure your evaluation metrics are solid?
validmetrics 1 year ago next
That's a great question. We typically monitor 90-day mean and variance to detect potential concept drifts. We would then employ a
neutraleval 1 year ago prev next
Incidentally, what monitoring software do you use to guarantee model performance and track concept drifts in production?
devopsenthus 1 year ago next
If you want a good monitoring solution, consider tools like Prometheus and Grafana. They can help you track your models' performance and ensure they adapt to changing environments without missing a beat.
anotheruser 1 year ago next
I'm curious, how do you efficiently incorporate domain knowledge into ML models? In my experience, an understanding of the problem context is key for high-performing and accurate models.
ml_expert 1 year ago next
There are multiple ways, and it depends on the model. Generally, domain knowledge can be incorporated as features or through constraints during training. Have you tried using feature engineering or regularization?