1 point by ml_newbie 11 months ago flag hide 25 comments
user1 11 months ago next
Some great suggestions here! I'd love to hear more about managing version control of models and data for reproducibility.
expert_ml 11 months ago prev next
When deploying ML models, it's crucial to use a reliable container technology like Docker to ensure consistent execution. We usually use Kubernetes to manage our ML microservices in production.
devops_user 11 months ago next
We use similar technology for our ML apps deployments. Kubernetes has been quite stable and easy to operate. How about scaling and maintaining high availability?
expert_ml 11 months ago next
K8s has built-in features for autoscaling, so if you carefully plan and setup your resource requests/limits it should be quite simple. For high availability, we suggest (and use) running at least three replicas.
ai_engineer 11 months ago prev next
Don't forget about MLOps best practices, such as implementing monitoring and alerting systems that report not just ML model performance but model drift as well.
data_scientist 11 months ago prev next
Thanks! Any recommendations for creating a pre-production environment that allows A/B testing and Canary deployments?
ml_infra 11 months ago next
Setting up a staging environment to simulate production is essential. This allows you to preview models, test any model update, and apply Canary releases gradually.
cs_major 11 months ago next
What tools do you suggest for staging environments? AWS, GCP, Azure, or on-prem?
cloud_expert 11 months ago next
I think it's a subjective question. Each cloud platform has its own solutions like Amazon SageMaker, GCP AI Platform or Azure Machine Learning. Factors such as cost, customizability, and personal preference may impact which you choose.
open_source_user 11 months ago prev next
Would love to utilize open source tools as much as possible. Any recommendations in this regard for ML deployments?
oss_enthusiast 11 months ago next
You can look into MLflow, Kubeflow, or Seldon Core, as they all provide extensive ML pipelines and deployment functionalities in an open-source fashion.
newbie_ml 11 months ago prev next
How do you address data security concerns while using cloud-based ML deployments?
sec_engineer 11 months ago next
Encryption, network security, identity access management, and logging audit trails are all crucial to maintain data security in cloud-based ML deployments.
newbie_ml 11 months ago next
Thanks! How do I choose a specific VPC or Vnet while deploying an ML application in the cloud?
network_admin 11 months ago next
When deploying, you can select an appropriate VPC or VNet via the cloud provider's console or API. Then, allocate or connect it to your computing instances or VMs.
performance_nut 11 months ago prev next
What about auto-scaling inference services like AWS SageMaker, GCP AI Platform, or Azure ML? How do I optimize latency and throughput?
infra_engineer 11 months ago next
It usually depends on your use case and priority (latency or throughput). You can try techniques like batching, load balancing, model fine-tuning, and GPU instance types.
research_scientist 11 months ago prev next
What tools or software libraries are available to help with the model interpretation, explaining the predictions to the end-users?
interpretable_ml 11 months ago next
There are several libraries for model interpretability. SHAP, LIME, Alibi, and TreeExplainer (for tree-based models) are some popular ones.
big_data_arch 11 months ago prev next
What is the current state and future trends of streaming ML in production?
streaming_ml 11 months ago next
Streaming ML is an active area of research and development. Real-time predictive analytics, online model training, and automated decision-making are popular use-cases. Tools like Akka, Spark Streaming, and Flink are gaining popularity.
cost_engineer 11 months ago prev next
What about cost optimization for running ML inference at scale?
saver_ml 11 months ago next
Consider GPU/TPU instance types, autoscaling, and spot instances for cloud-based ML. Cost-aware model selection, quantization, and pruning also help in reducing ML deployment expenses.
ml_team_lead 11 months ago prev next
In a real-world scenario, what is the average time taken for an ML model iteration (from data preparation to deploying in production)?
seasoned_ml 11 months ago next
It depends on the team size, ML subfield, and model complexity. For a well-staffed team, you can aim for 1-4 weeks, although complex models might require more time. Data versioning and feature pipelines may impact the duration as well.