234 points by decen_social 1 year ago flag hide 11 comments
john_carmack 1 year ago next
Great work! Building a decentralized social media platform is no small feat. I'm impressed that you chose Rust for this project. I'd love to hear more about the challenges you faced while implementing the distributed data store and the consensus algorithm.
notoscope 1 year ago next
Thanks, john_carmack! I used HashiMap as a key-value store for storing individual user data and HyperLogLog for collecting the feed of the users. For the consensus algorithm, I tried out Raft, and it went quite smoothly.
cryptoknight 1 year ago prev next
Rust and decentralized networks are a powerful combination. I would be interested to read a detailed blog post on your implementation. How do you handle user authentication and identify verification?
swizec 1 year ago prev next
What made you choose peer-to-peer networking over client-server architecture?
notoscope 1 year ago next
swizec, a key factor was enabling features like offline publishing, user autonomy, robustness, and redundancy in downtime situations. I would say the limitations were working with the NAT traversal to establish a secure communication channel between peers and implementing a trust-building mechanism.
hertzfitz 1 year ago prev next
What was the overall complexity in terms of time and resources required to pull this project off? And do you plan to keep the code open-source? I'm sure the community would benefit from it.
notoscope 1 year ago next
Hertzfitz, I'm planning to open-source the project and write follow-up blog posts, giving an in-depth analysis of each module and architecture choices. The project took 6 intense months of research and development, working almost daily. I believe everyone should be able to run their networks, and the resources involved are a remote server or even a PC.
async_shane 1 year ago prev next
How did you address data replication and consistency concerns in the network, given multiple nodes? And how about incentives for maintaining network uptime, data storage, and improving the system?
notoscope 1 year ago next
Async_Shane, a DHT (Distributed Hash Table) was employed to replicate data and maintain consistency in the presence of multiple nodes. For incentives, a reputation-based system governs user interaction while proof-of-work type of mechanisms are encouraged to secure the network.
bad_idea 1 year ago prev next
Did you manage to get reasonable cold-start user experience and metrics on the scale and performance?
notoscope 1 year ago next
bad_idea, for the cold-start users, a WebTorrent based feed is used. Performance is debatable depending on the number of connections established, but scalability is always a challenge with these P2P systems, so legacy issues prevail. However, user feedback and metrics play a crucial role in further optimizations.