121 points by tokio-search 6 months ago flag hide 26 comments
johnrodo 6 months ago next
This is really exciting! I've been waiting for a Rust-native search library with advanced features.
c0dep1rate 6 months ago next
I agree, Rust is a great choice for this type of library. Have you taken a look at the API yet?
johnrodo 6 months ago next
Yes, I have and it looks really comprehensive! I especially like the support for different tokenizers.
h4x0rch1c 6 months ago prev next
What kind of performance can we expect from this library? Are there any benchmarks available?
johnrodo 6 months ago next
I haven't seen any official benchmarks yet, but given that it's Rust, I would expect it to be pretty fast. I'm planning on doing some testing myself and will post the results on the project repo.
n3wbies1337 6 months ago prev next
Thank you for the reply. Look forward to the benchmark results. I hope they are promising.
an0nym0u5 6 months ago prev next
How does it compare to other search libraries like Elasticsearch or Solr?
johnrodo 6 months ago next
I haven't had a chance to do a side-by-side comparison yet, but the advanced features like fuzzy matching and field weighting are similar.
n3wbies1337 6 months ago prev next
Has anyone had a chance to integrate this library with a web framework like Rocket or Actix?
c0dep1rate 6 months ago next
I haven't tried Rocket yet, but I was able to integration with Actix pretty easily. I wrote a blog post about it on my website.
h4x0rch1c 6 months ago prev next
I'm curious about the indexing logic. Is it optimized for incremental indexing or should we rebuild the entire index every time?
johnrodo 6 months ago next
The library supports both approaches, but I personally think it's best suited for incremental indexing since Rust has really efficient memory management. But you can always manually rebuild the index if you prefer.
an0nym0u5 6 months ago prev next
Any plans to add support for natural language processing (NLP) techniques like stemming, lemmatization or part-of-speech tagging?
johnrodo 6 months ago next
That's a great suggestion! I haven't had a chance to implement NLP support yet, but it's definitely on my roadmap. Contributions are welcome!
an0nym0u5 6 months ago prev next
One concern I have with Rust is the long-term maintenance of the library. What's the ecosystem like for Rust libraries?
c0dep1rate 6 months ago next
The Rust community is really active and growing rapidly. There are a lot of libraries with great support and active communities, so I'm not too worried about long-term maintenance.
h4x0rch1c 6 months ago prev next
I'm curious to know more about the types of advanced features you mentioned. Can you give some examples?
johnrodo 6 months ago next
Sure! For example, the library supports phrase searches, fuzzy matching, field weighting, and different tokenizer options. You can also combine search terms with logical operators like AND, OR and NOT.
n3wbies1337 6 months ago prev next
Thanks for sharing the details. It's great that there's a new and powerful library for Rust.
an0nym0u5 6 months ago prev next
This is great news! It will be good to have another native full-text search option for Rust.
h4x0rch1c 6 months ago prev next
I'm interested in this library for its performance benefits over other languages, but not sure if the learning curve for Rust is worth it yet.
johnrodo 6 months ago next
That's a fair point. While Rust can be a bit difficult to pick up at first, I think the investment is worth it for the performance and community benefits. There are also many resources and tutorials available to help get started.
c0dep1rate 6 months ago prev next
I agree with John. Rust has a great community and learning resources available. Plus, the performance benefits of Rust are pretty significant.
an0nym0u5 6 months ago prev next
Thanks for creating this library, John! It's great to see Rust growing and expanding its ecosystem.
n3wbies1337 6 months ago prev next
Definitely looking forward to trying this library out and seeing what I can build with it!
h4x0rch1c 6 months ago prev next
Congratulations on the release, John! Let us know how we can support or contribute to your project!