42 points by db_scaling_startup 11 months ago flag hide 21 comments
startupdev 11 months ago next
We're currently struggling with our database as our user base expands. Any tips for scaling our infrastructure?
dbadmin1 11 months ago next
Have you considered sharding? It can distribute your data across multiple machines.
startupdev 11 months ago next
That's interesting. How difficult is implementing sharding?
dbadmin1 11 months ago next
Sharding at a large scale can be complex, but tools and libraries are available to simplify it.
cloudadvisor 11 months ago prev next
I recommend moving your DB to the cloud with managed services like AWS RDS or Google Cloud SQL.
founderjoe 11 months ago next
Thanks. We'll look into those services. Any observations on performance or cost?
cloudadvisor 11 months ago next
Performance is great and costs are generally reasonable. Both companies offer generous free tiers.
nate0 11 months ago next
In addition to the managed solutions, take a look at serverless options—FaunaDB, AWS Lambda, and Google Cloud Functions.
founderjoe 11 months ago next
I read that serverless architecture simplifies scaling horizontally, correct?
nate0 11 months ago next
Yes, that's right. It also helps reduce infrastructure management efforts and the cold start problem.
optimizexpert 11 months ago prev next
Don't forget database optimization. Caching frequently accessed data does wonders.
startupdev 11 months ago next
Could you please brief us on available caching methods?
optimizexpert 11 months ago next
Memcached and Redis are two popular options. They offer quick data retrieval, decreasing database load.
startupdev 11 months ago next
What are the limitations of caching though? Can we trust it completely?
optimizexpert 11 months ago next
Caching isn't a 100% reliable solution as it can lead to inconsistencies. Therefore, consider using other methods in parallel.
smartengr 11 months ago prev next
What about columnar storage? It could bring query performance improvements for analytics.
startupdev 11 months ago next
Isn't columnar storage best suited for column-based queries? Could it work with transactional use cases?
smartengr 11 months ago next
Yes, it generally works better for analytics but can work for transactional in some cases. Check out Apache Cassandra, MongoDB, or HBase.
rollingupdates 11 months ago prev next
Database partitioning and distributed computing with Apache Spark or Flink might be helpful for data processing.
startupdev 11 months ago next
We'd love to know more about Apache Spark and similar technologies. Can they scale with us?
rollingupdates 11 months ago next
Absolutely. These systems can handle massive datasets and are seen as a conventional solution for big data problems.