125 points by spiralbound 1 year ago flag hide 10 comments
thequux 1 year ago next
Fascinating read! I've been exploring various differentiable programming techniques for my projects and this article definitely provides a lot of food for thought.
johnsmith1776 1 year ago next
The examples in this post have really helped me get a better intuition about using reverse-mode AD in real-world programs. Thanks for sharing your insights!
theycallmelex 1 year ago next
One really important point is that with differentiable programming, you can express and optimize your entire workflow as a graph, rather thanjust small parts of it. It's a holistic approach that hugely pays off.
asdfsad 1 year ago next
I have to admit, the unification of differentiable programming and ML/AI workflows is insanely powerful! Handling both local and global optimizations within the same framework is such an elegant solution.
sarahg 1 year ago prev next
Definitely! I've noticed that incorporating differentiable methods in my tools has allowed me to train more complex and feature-rich models, as well as better tune existing ones with fantastic results.
nerdist42 1 year ago prev next
I've got to agree, the practical applications of these techniques are just mind-blowing! It's a really exciting time to be working on advanced ML/AI problems.
mathwiz314 1 year ago prev next
Still very early days though, I think. There's so much more potential to be unlocked as this field evolves and matures. Looking forward to the next big leaps in the coming years!
midnightmunchr 1 year ago next
Thank you for bringing this up, I've actually been experimenting with some libraries that enable differentiable programming for machine learning, and the abstractions are really well-suited for this kind of global optimization.
hiko 1 year ago prev next
Couldn't agree more! If anyone wants to dive deeper into this, I'd recommend checking out some of the papers listed in this article's 'Further Reading' section. It's really a fascinating world!