456 points by distributed_guru 6 months ago flag hide 13 comments
user1 6 months ago next
Fascinating! I've heard about Gossip Protocols, but never in the context of distributed systems. Scalability with large systems is critical these days, so this is definitely note-worthy.
curious_developer 6 months ago next
Could someone ELI5 gossip protocols and what makes them so scalable?
user4 6 months ago next
Check the <https://www.example.com/gossip_protocols> link I posted above, it covers. it in more detail!
distributed_expert 6 months ago prev next
Glad you liked the post! Gossip Protocols, also known as Epidemic Protocols, spread information to all nodes in distributed systems exponentially. This leads to huge efficiency and significant scalability improvements compared to traditional tree or mesh structures. I'll post a link to a great article below.
hn_user 6 months ago prev next
I've been working with Raft and Paxos recently. How do gossip protocols compare to these techniques?
comparative_gn 6 months ago next
Both Raft and Paxos have their advantages in consensus algorithms, however, gossip protocols focus more on data dissemination and scalability. They can be used synergistically for better performance in complex, large-scale systems.
example_user 6 months ago prev next
Great insights. Thanks, everyone! Will definitely look more into Gossip Protocols and their integration with current systems I use.
another_hn_user 6 months ago prev next
What kind of real-world applications use gossip protocols?
knowledgable_hn 6 months ago next
They can be found in modern networks, IoT systems, databases, and more! Some implementations include Apache Cassandra, Amazon DynamoDB, and Riak. They dramatically improve system performance and fault tolerance in large clusters.
another_user01 6 months ago prev next
Do these gossip protocols implement any 'gossip DSL' to write their own protocol more easily? Or is it all manual coding? What would you recommend to start?
library_freak 6 months ago next
I wouldn't strictly label it a 'DSL'. More like a variety of libraries are available, such as HashiCorp's Serf and Google original 'gossip' library. They cover messaging, group formation, and state dissemination. These libraries are a great starting point!
brave_one 6 months ago prev next
Has anyone integrated gossip protocols with PKI infrastructure? Or would that add complexity with no real added benefits?
security_gn 6 months ago next
GossipKey <https://github.com/ApsaraDB/gossip-key> could be an interesting project that integrates PKI with gossip protocols. It might be a bit challenging to implement all pieces, but it's definitely worth investigating for scenarios requiring added security.