65 points by mystartup-engineer 6 months ago flag hide 11 comments
john_doe 6 months ago next
Great write-up! I've been trying to achieve the same for my startup and this gave me a lot of ideas. We're using AWS RDS and it's already struggling with a fraction of your request volume. I think using AWS DynamoDB might be a good solution for us due to its scalability features. Any thoughts on this?
code_master 6 months ago next
We also use DynamoDB at my company, and it's fantastic for scaling and handling massive traffic. You'll just need to balance performance and cost, and consider auto-scaling settings. We actually wrote a post about our experience with DynamoDB, I can share the link here in a bit.
code_master 6 months ago next
@john_doe ElastiCache is a great choice for caching. I assume you use Redis since it pairs well with ElastiCache. Btw, we also use ALB and I'm curious about your approach to connection draining and how that's configured. Mind sharing more about it?
john_doe 6 months ago next
Sure, we use Redis with ElastiCache. Regarding connection draining, we prefer to avoid abrupt disconnections by enabling the deregistration delay on our targets, setting them to 60 seconds by default. We experienced significant improvements with this setting.
james_teamlead 6 months ago next
We considered the same approach, but we ended up using AWS Lambda for handling targets' connections, working seamlessly together with the load balancer and ElastiCache. This allowed for more efficient scaling during traffic bursts and prevented resources from being left idle during periods of low traffic.
database_guru 6 months ago prev next
Awesome to see such high traffic numbers! Care to share a bit more about your infrastructure stack? I am assuming you use load balancers for handling requests and your database is probably a distributed db solution. Could you provide more context on the database itself?
john_doe 6 months ago next
@database_guru We use a custom distributed caching solution based on AWS ElastiCache and PostgreSQL for persistent storage. We rely on AWS Application Load Balancer for handling incoming traffic.
database_guru 6 months ago next
This approach seems comparable to the Google Spanner Engine but customized for your exact needs. I'm impressed! How did your team arrive at this solution?
database_guru 6 months ago next
Our team also evaluated a few different approaches. After a couple of radical design iterations, we came to a similar solution. I'm glad to hear it worked well for you. Best wishes on continuing to drive your impressive traffic numbers.
big_data_scientist 6 months ago prev next
In case anyone is facing data analytics challenges with such scale, we found success leveraging Apache Spark and Apache Kafka for real-time insights. They pair well with our DynamoDB solution. Feel free to reach out if you have questions on this.
grade_a_geek 6 months ago next
I am curious if you considered using Hadoop instead of Apache Spark and Kafka. Although I haven't tried Hadoop with DynamoDB, I'm eager to hear your thoughts on this since the cost and community are more mature than Spark.