84 points by ml_recommendations 11 months ago flag hide 14 comments
username1 11 months ago next
Great topic! I'm interested to hear what you have in mind for your recommendation system.
original_poster 11 months ago next
I'm planning on using a hybrid approach that combines content-based and collaborative filtering methods. Any thoughts on which ML framework is most suitable for this?
ml_expert 11 months ago next
I would recommend using TensorFlow or PyTorch. Both have large communities and a lot of support for large-scale ML projects. You can build custom models using each of them.
original_poster 11 months ago next
Interesting, I've used TensorFlow before but never for large-scale recommendations. Would I be able to maintain high throughput and relatively low latency with TensorFlow?
ml_expert 11 months ago next
Yes, absolutely! TensorFlow has a module, TensorFlow Serving, that's designed specifically for large-scale deployment. You can optimize your model for serving with it for low-latency inference.
username2 11 months ago prev next
I've had good experiences with TensorFlow Recommenders, an open-source library for building recommendation systems. It might be helpful for your use case.
original_poster 11 months ago next
Oh, thank you! I hadn't come across TensorFlow Recommenders yet. I'll definitely take a look.
username3 11 months ago prev next
Have you considered something like Spark MLlib for distributed computing?
original_poster 11 months ago next
I'd heard of Spark MLlib, but I wasn't sure if it was suitable for recommendation systems. I'll go through its documentation and see if it could help me.
big_data_guru 11 months ago prev next
Spark MLlib is, in fact, a useful system for building recommendation engines with distributed computing. It supports a variety of models, including matrix factorization and alternating least squares.
username4 11 months ago prev next
If you're looking to optimize latency in your recommendation system, don't forget to explore caching and memoization techniques for your models!
original_poster 11 months ago next
Noted, thank you! I'll definitely integrate some caching strategies in my infrastructure to improve latency.
username5 11 months ago prev next
I'd recommend using something like Redis or Memcached for caching, which can be easily integrated into most frameworks and libraries.
original_poster 11 months ago next
That's a great idea. I'm using a microservices architecture, so routing the requests to cache accordingly shouldn't be too difficult.