456 points by rust_scraper 5 months ago flag hide 12 comments
john_doe 5 months ago next
Great work! I've been looking for a real-time web scraping framework in Rust. I would love to try this out.
author 5 months ago next
Thanks for the kind words, john_doe! Let me know if you have any questions.
another_user 5 months ago prev next
That looks really promising, I'm going to give it a try and see how it handles some websites I've had issues scraping.
helpful_hn_user 5 months ago next
I had a similar use-case and this framework works great. Highly recommend it!
user4 5 months ago prev next
What dependencies does this have? I'd like to test it, but I'd prefer not to add too many new dependencies to my project.
author 5 months ago next
The only dependency is `scraper` for handling HTML scraping. The real-time update functionality is built on top of `tokio`
gg 5 months ago prev next
Just a small question, what's the performance like? I've had issues with the stability of some other frameworks in the past.
author 5 months ago next
Before releasing the framework, I've done some testing and found it to be quite stable. I'll be keeping an eye on it and addressing any issues that come up.
someone 5 months ago prev next
Whats the status of this? Is it still under development? When did you release it?
author 5 months ago next
It was released a few weeks ago, and I've been actively working on adding new features and fixing bugs. Check out the Github repo for the latest updates
test_person 5 months ago prev next
I'm curious, what's the scalability of the framework?
author 5 months ago next
It can handle a large number of simultaneous requests, however, the exact performance depends on the complexity of the websites being scraped, and available system resources.