34 points by distributed_guy 1 year ago flag hide 101 comments
user7 1 year ago next
RaftConsensus is a great open-source project for understanding the consensus protocol in distributed systems.
user8 1 year ago next
I second that. Raft is easier to understand than Paxos, and the Raft paper is well-written.
user9 1 year ago prev next
Another great resource is the SysArch videos on YouTube by Robbert Van Renesse. He covers various distributed systems topics.
user11 1 year ago next
Good suggestion. I learned a lot from the SysArch videos.
user10 1 year ago prev next
Distributed Systems by George Coulouris, Tim Kindberg, Graham Morgan, and Gordon Blair is another good book to consider.
user12 1 year ago next
I've heard good things about that book. I plan to check it out soon.
user13 1 year ago prev next
Lastly, don't forget about Apache Kafka. It's a popular and real-world implementation of distributed streaming.
user14 1 year ago next
Kafka is a great choice. I've used it in the past for distributed event processing.
user15 1 year ago prev next
I'd also recommend checking out Google's Spanner, a scalable and horizontally-partitioned database.
user16 1 year ago next
Spanner is amazing. I've heard great things about it. I've also heard that it's widely used within Google.
user17 1 year ago prev next
Having a solid foundation in computer networks and concurrency concepts is helpful when learning about distributed systems.
user18 1 year ago next
True. Having a solid foundation in computer networks is a must. I'd recommend the book 'Computer Networking: A Top-Down Approach' by Kurose & Ross.
user19 1 year ago prev next
I second user18. That book is a great resource for understanding computer networks concepts.
user20 1 year ago prev next
Distributed Systems for Fun and Profit is a free eBook by Mikito Takada that I found to be very entertaining and informative.
user21 1 year ago next
I've heard great things about that book. I plan to check it out soon.
user22 1 year ago prev next
A hands-on approach I'd recommend is building a simple distributed file system in a project.
user23 1 year ago next
I agree. Building a simple distributed file system to learn the concepts is a great idea.
user24 1 year ago prev next
If you're into online courses, I recommend 'Introduction to Distributed Systems' by James M. Wilcox on edX.
user25 1 year ago next
I've heard good things about that course. I plan to check it out soon.
user26 1 year ago prev next
Joining online communities, such as this HN community, is a great way to ask questions and learn from others' experiences.
user27 1 year ago next
True. This HN community has been a great resource for me as well.
user28 1 year ago prev next
Lastly, I'd recommend checking out the paper 'MapReduce: Simplified Data Processing on Large Clusters' by Jeffrey Dean and Sanjay Ghemawat.
user29 1 year ago next
MapReduce is a great paper and concept. I've used it in the past for distributed batch processing.
user30 1 year ago prev next
Distributed hash tables (DHT) are an important concept in distributed systems. I recommend looking into Chord and Kademlia.
user31 1 year ago next
Chord and Kademlia are interesting algorithms. I've used them in a few projects in the past.
user32 1 year ago prev next
I'd also recommend checking out the Dynamo paper by Giuseppe DeCandia et al. Dynamo is widely used within AWS.
user33 1 year ago next
I've heard good things about Dynamo. It's interesting that it's widely used within AWS.
user34 1 year ago prev next
Consider looking into the Riak database as an example of a real-world implementation of the Dynamo paper.
user35 1 year ago next
I've heard of Riak before. I'll check it out as an example of a real-world implementation of the Dynamo paper.
user36 1 year ago prev next
In my experience, a combination of reading books, taking online courses, and implementing distributed systems concepts is the best way to learn.
user37 1 year ago next
I couldn't agree more. A combination of multiple resources is the best way to learn distributed systems.
user38 1 year ago prev next
Don't forget to consider the trade-offs between consistency and availability in distributed systems design.
user39 1 year ago next
Thank you for the reminder. Consistency and availability trade-offs are crucial in distributed systems design.
user40 1 year ago prev next
I recommend looking into the CAP theorem and consistency models, such as linearizability, sequential consistency, and eventual consistency.
user41 1 year ago next
The CAP theorem is a great starting point for understanding consistency models. I recommend reading the original paper by Eric Brewer.
user42 1 year ago prev next
In my experience, understanding distributed transactions is key to mastering distributed systems design.
user43 1 year ago next
Understanding distributed transactions is crucial. I'd recommend looking into the 2-phase commit and saga patterns.
user44 1 year ago prev next
Another important concept in distributed systems is message queues, such as RabbitMQ and Kafka.
user45 1 year ago next
Message queues are important. I've used RabbitMQ and Kafka in the past for distributed event processing.
user46 1 year ago prev next
I suggest looking into the AWS SDKs for easy-to-use APIs for working with distributed systems on AWS.
user47 1 year ago next
I've used AWS SDKs before. They're great for making working with AWS easier.
user48 1 year ago prev next
In my experience, it's important to understand the failure modes and recovery techniques in distributed systems.
user49 1 year ago next
Failure modes and recovery techniques are crucial for building reliable distributed systems. Thanks for the reminder.
user50 1 year ago prev next
Lastly, don't forget about security considerations when building distributed systems.
user1 1 year ago prev next
Check out the book 'Designing Data-Intensive Applications' by Martin Kleppmann. It has a great section on distributed systems.
user2 1 year ago next
I agree with user1. The book is a must-read. Also, consider taking an online course to supplement the learning.
user4 1 year ago next
I recommend the online course 'Distributed Systems' by Chris Colohan on Coursera.
user5 1 year ago prev next
I suggest the Udacity course 'Designing and Building Distributed Systems' by George S. Avrunin.
user3 1 year ago prev next
I'd suggest implementing distributed systems concepts in a small project as a hands-on learning experience.
user6 1 year ago next
I implemented a simple key-value store in a project and it was invaluable for understanding distributed systems concepts.