14 points by code_junkie 1 year ago flag hide 24 comments
johnsmith 1 year ago next
Great work! Real-time ML with personal analytics is a powerful combo. Looking forward to more features in future updates.
mikejones 1 year ago next
Indeed, real-time ML is a HN favorite topic. Thanks for sharing! I'm curious about the tech stack used?
johnsmith 1 year ago next
We're using Python with TensorFlow for ML, and Flask for the backend. It was a fun project to build.
clarissalau 1 year ago prev next
Interesting, I used PyTorch and FastAPI for my project. Running ML models in real time can be challenging.
clarissalau 1 year ago prev next
I agree with johnsmith! I built something similar a while back, but real-time ML is a game changer. Nice work.
mikejones 1 year ago next
FastAPI is definitely worth checking out, I've heard many good things about it. Running ML models is indeed difficult, good job resolving those challenges.
davidkim 1 year ago prev next
Awesome work on the dashboard! I'm looking into integrating real-time ML into my own projects, any advice?
johnsmith 1 year ago next
It's important to consider the amount of resources needed for real-time ML and manage them efficiently. We're using Kubernetes to help with that.
clarissalau 1 year ago next
Kubernetes is a great choice for resource management! It's an important piece of the puzzle that you shouldn't skip.
mikejones 1 year ago prev next
Definitely, managing resources is key. Our team uses AWS's Elastic Kubernetes Service (EKS) to handle the infrastructure.
davidkim 1 year ago next
EKS sounds interesting, how does resource scaling work with that?
mikejones 1 year ago next
EKS allows you to automatically scale resources based on workloads. It's a huge benefit to real-time ML, where processing time can vary a lot.
kimlee 1 year ago prev next
Great to see real-time ML in action! How do you handle real-time data streams?
johnsmith 1 year ago next
We're using webhooks to receive real-time data streams. We then trigger a ML processing pipeline that takes care of the data processing.
clarissalau 1 year ago next
Webhooks are a great option for real-time data streams. With PostgreSQL, you can efficiently handle large data volumes without losing data integrity.
mikejones 1 year ago prev next
We are using something similar, webhooks are convenient for integrating with external services. For data storage, we're using PostgreSQL as the primary database to ensure data consistency.
donaldwu 1 year ago prev next
Would like to know more about the ML pipeline, are there any repos or articles available for reference?
johnsmith 1 year ago next
Definitely! We wrote a blog post explaining the pipeline, along with our Github repo. Here's the link: [github-repo]
mikejones 1 year ago prev next
Our ML pipeline is explained in this article: [medium-article]. We use TensorFlow and Kubernetes to orchestrate the pipeline.
jackieye 1 year ago prev next
Beautifully made dashboard and ML pipeline! I'm curious about the data visualization, what libraries did you use?
johnsmith 1 year ago next
We're using Plotly and Dash by Plotly for data visualization. The combination allows for interactivity and flexible customizations.
mikejones 1 year ago prev next
Looks great! For scatter plots, we've found that Plotly is best in class and integrates well with Dash. Bokeh is highly functional as well for charting, which we've used in some of our projects.
angela_li 1 year ago prev next
Building real-time ML pipelines seems like quite a challenge. Thanks for sharing your experiences, it's a valuable resource for the community!
johnsmith 1 year ago next
Thanks angela_li! It's a challenge but rewarding, and we're glad to help the community in their ML journeys. Best of luck to your projects!