212 points by backendchief 1 year ago flag hide 11 comments
user1 1 year ago next
Nice work! I'd love to hear more about your caching strategy. How did you manage to reduce the load on your servers?
techlead 1 year ago next
We implemented a multi-layer caching infrastructure. We have a CDN in front, followed by Redis. We also use Memcached for some specific cases. This has significantly reduced the load on our servers and decreased the overall latency.
user2 1 year ago prev next
100M requests per day is impressive. We're struggling to handle just a fraction of that. What did you do to load balance your requests? Any tips for us?
techlead 1 year ago next
We went with Kubernetes for container orchestration and horizontal scaling. For load balancing, we use HAProxy. It works like a charm! It's vital to have a robust load balancing solution in place when handling a significant amount of requests.
infrastructureguru 1 year ago prev next
At that scale, you should also consider using auto-scaling. Your container orchestration system should be able to launch new instances whenever needed and stop them when they're not required. For example, Kubernetes has a great support for this.
user3 1 year ago prev next
Very interesting! Did you use any specific metrics to monitor your infrastructure, like server loads, error rates, or latency?
techlead 1 year ago next
Yes! We have a comprehensive monitoring solution in place, using tools like Prometheus, Grafana, and ELK stack for metrics visualization and logs analysis. Our DevOps team receives alerts through PagerDuty whenever something goes wrong.
user4 1 year ago prev next
How do you handle backpressure? Any tips on how to prevent your system from overloading?
techlead 1 year ago next
We implemented multiple rate limiting strategies and circuit breakers to prevent our system from overloading. Moreover, our API gateway has a queue that can be fine-tuned to control the incoming traffic. It's essential to have these safeguards in place while processing such a high volume of requests.
user5 1 year ago prev next
What database did you use and how did it withstand the load? Which querying language did you use?
techlead 1 year ago next
We used Google Cloud Spanner as our main database. It's a horizontally scalable relational database with a SQL-like query language. We also used BigQuery for our analytical workloads. Both databases served us well at this scale.