121 points by tokio-search 11 months ago flag hide 26 comments
johnrodo 11 months ago next
This is really exciting! I've been waiting for a Rust-native search library with advanced features.
c0dep1rate 11 months ago next
I agree, Rust is a great choice for this type of library. Have you taken a look at the API yet?
johnrodo 11 months ago next
Yes, I have and it looks really comprehensive! I especially like the support for different tokenizers.
h4x0rch1c 11 months ago prev next
What kind of performance can we expect from this library? Are there any benchmarks available?
johnrodo 11 months ago next
I haven't seen any official benchmarks yet, but given that it's Rust, I would expect it to be pretty fast. I'm planning on doing some testing myself and will post the results on the project repo.
n3wbies1337 11 months ago prev next
Thank you for the reply. Look forward to the benchmark results. I hope they are promising.
an0nym0u5 11 months ago prev next
How does it compare to other search libraries like Elasticsearch or Solr?
johnrodo 11 months ago next
I haven't had a chance to do a side-by-side comparison yet, but the advanced features like fuzzy matching and field weighting are similar.
n3wbies1337 11 months ago prev next
Has anyone had a chance to integrate this library with a web framework like Rocket or Actix?
c0dep1rate 11 months ago next
I haven't tried Rocket yet, but I was able to integration with Actix pretty easily. I wrote a blog post about it on my website.
h4x0rch1c 11 months ago prev next
I'm curious about the indexing logic. Is it optimized for incremental indexing or should we rebuild the entire index every time?
johnrodo 11 months ago next
The library supports both approaches, but I personally think it's best suited for incremental indexing since Rust has really efficient memory management. But you can always manually rebuild the index if you prefer.
an0nym0u5 11 months ago prev next
Any plans to add support for natural language processing (NLP) techniques like stemming, lemmatization or part-of-speech tagging?
johnrodo 11 months ago next
That's a great suggestion! I haven't had a chance to implement NLP support yet, but it's definitely on my roadmap. Contributions are welcome!
an0nym0u5 11 months ago prev next
One concern I have with Rust is the long-term maintenance of the library. What's the ecosystem like for Rust libraries?
c0dep1rate 11 months ago next
The Rust community is really active and growing rapidly. There are a lot of libraries with great support and active communities, so I'm not too worried about long-term maintenance.
h4x0rch1c 11 months ago prev next
I'm curious to know more about the types of advanced features you mentioned. Can you give some examples?
johnrodo 11 months ago next
Sure! For example, the library supports phrase searches, fuzzy matching, field weighting, and different tokenizer options. You can also combine search terms with logical operators like AND, OR and NOT.
n3wbies1337 11 months ago prev next
Thanks for sharing the details. It's great that there's a new and powerful library for Rust.
an0nym0u5 11 months ago prev next
This is great news! It will be good to have another native full-text search option for Rust.
h4x0rch1c 11 months ago prev next
I'm interested in this library for its performance benefits over other languages, but not sure if the learning curve for Rust is worth it yet.
johnrodo 11 months ago next
That's a fair point. While Rust can be a bit difficult to pick up at first, I think the investment is worth it for the performance and community benefits. There are also many resources and tutorials available to help get started.
c0dep1rate 11 months ago prev next
I agree with John. Rust has a great community and learning resources available. Plus, the performance benefits of Rust are pretty significant.
an0nym0u5 11 months ago prev next
Thanks for creating this library, John! It's great to see Rust growing and expanding its ecosystem.
n3wbies1337 11 months ago prev next
Definitely looking forward to trying this library out and seeing what I can build with it!
h4x0rch1c 11 months ago prev next
Congratulations on the release, John! Let us know how we can support or contribute to your project!