237 points by data_whiz 1 year ago flag hide 17 comments
johnsmith 1 year ago next
Great post! I've been working on a similar project and I can definitely relate to the challenges of low-latency streaming. Keep up the good work!
janesmith 1 year ago next
I'm curious, what stack/technologies did you use for building the app? I'm looking for a new tech for my current project, so your input would be valuable.
johnsmith 1 year ago next
We used Python with some async libraries, like AnyIO and aiohttp. It offers a great balance between high-level programming and low-level control, which is key in a real-time streaming project.
otherperson 1 year ago next
I've heard good things about AnyIO! Have you considered using WebSockets for the application layer protocol? Strong recommendation by the Web community.
experienced_developer 1 year ago next
I've been using AnyIO as well and the experience has been great. How about containerization for the app deployment, did you explore any options like Docker or Kubernetes?
someoneelse 1 year ago prev next
What kind of data are you streaming? And what do you think is the biggest challenge in creating a low-latency app like this? I'd love to hear your opinions.
janesmith 1 year ago next
We're streaming stock market data, it's a real-time Decision Support System. As for challenges, some of the biggest ones are dealing with network packet loss and making sure the data processing is balanced with latency requirements.
firstuser 1 year ago prev next
What about latency vs processing power? How do you balance this?
seconduser 1 year ago next
You could consider using multi-threading, or a reactive programming language can really help. Have a look at RxPY.
johnsmith 1 year ago next
Great point! Reactive programming is worth looking into for sure, gonna give it a try.
newgrammar 1 year ago prev next
Did you face any challenges building the end-users' React UI? How complicated was it, designing the view to show real-time stock prices?
realcoder 1 year ago next
Implementing the UI wasn't that difficult with the help of currently available libraries. There are a lot of them that update the view automatically as updatesCB fires. Check out MobX for example.
web_dev 1 year ago prev next
When working with network constraints, is there a pattern/practice you follow to mitigate the impact of packet loss? Also, the choice of a programming language seems relevant as it can help in reducing latency-related issues. Any inputs on this?
server_coder 1 year ago next
Sure! For packet loss, error-correcting codes and retransmission protocols like TCP can help. As for the language, performance is key in low-latency applications, and in our experience, compiled languages like C# and C++ exhibited the least latency as compared to the JVM family.
junior_developer 1 year ago prev next
I'm working on a real-time app for the first time. Could you give an overview of real-time data processing and the tools available for building a real-time app?
junior_developer 1 year ago next
Thank you! I'm overwhelmed with options and this certainly clarified a few misconceptions for me.
developer 1 year ago prev next
You'll likely use real-time messaging protocols such as MQTT, WebSocket, or SSE. You can also use a real-time platform like PubNub or Pusher. If you're looking for a language to build with, consider Go, Rust, or a Lisp for low-latency applications.