14 points by code_junkie 4 months ago flag hide 24 comments
johnsmith 4 months ago next
Great work! Real-time ML with personal analytics is a powerful combo. Looking forward to more features in future updates.
mikejones 4 months ago next
Indeed, real-time ML is a HN favorite topic. Thanks for sharing! I'm curious about the tech stack used?
johnsmith 4 months ago next
We're using Python with TensorFlow for ML, and Flask for the backend. It was a fun project to build.
clarissalau 4 months ago prev next
Interesting, I used PyTorch and FastAPI for my project. Running ML models in real time can be challenging.
clarissalau 4 months ago prev next
I agree with johnsmith! I built something similar a while back, but real-time ML is a game changer. Nice work.
mikejones 4 months ago next
FastAPI is definitely worth checking out, I've heard many good things about it. Running ML models is indeed difficult, good job resolving those challenges.
davidkim 4 months ago prev next
Awesome work on the dashboard! I'm looking into integrating real-time ML into my own projects, any advice?
johnsmith 4 months ago next
It's important to consider the amount of resources needed for real-time ML and manage them efficiently. We're using Kubernetes to help with that.
clarissalau 4 months ago next
Kubernetes is a great choice for resource management! It's an important piece of the puzzle that you shouldn't skip.
mikejones 4 months ago prev next
Definitely, managing resources is key. Our team uses AWS's Elastic Kubernetes Service (EKS) to handle the infrastructure.
davidkim 4 months ago next
EKS sounds interesting, how does resource scaling work with that?
mikejones 4 months ago next
EKS allows you to automatically scale resources based on workloads. It's a huge benefit to real-time ML, where processing time can vary a lot.
kimlee 4 months ago prev next
Great to see real-time ML in action! How do you handle real-time data streams?
johnsmith 4 months ago next
We're using webhooks to receive real-time data streams. We then trigger a ML processing pipeline that takes care of the data processing.
clarissalau 4 months ago next
Webhooks are a great option for real-time data streams. With PostgreSQL, you can efficiently handle large data volumes without losing data integrity.
mikejones 4 months ago prev next
We are using something similar, webhooks are convenient for integrating with external services. For data storage, we're using PostgreSQL as the primary database to ensure data consistency.
donaldwu 4 months ago prev next
Would like to know more about the ML pipeline, are there any repos or articles available for reference?
johnsmith 4 months ago next
Definitely! We wrote a blog post explaining the pipeline, along with our Github repo. Here's the link: [github-repo]
mikejones 4 months ago prev next
Our ML pipeline is explained in this article: [medium-article]. We use TensorFlow and Kubernetes to orchestrate the pipeline.
jackieye 4 months ago prev next
Beautifully made dashboard and ML pipeline! I'm curious about the data visualization, what libraries did you use?
johnsmith 4 months ago next
We're using Plotly and Dash by Plotly for data visualization. The combination allows for interactivity and flexible customizations.
mikejones 4 months ago prev next
Looks great! For scatter plots, we've found that Plotly is best in class and integrates well with Dash. Bokeh is highly functional as well for charting, which we've used in some of our projects.
angela_li 4 months ago prev next
Building real-time ML pipelines seems like quite a challenge. Thanks for sharing your experiences, it's a valuable resource for the community!
johnsmith 4 months ago next
Thanks angela_li! It's a challenge but rewarding, and we're glad to help the community in their ML journeys. Best of luck to your projects!