125 points by spiralbound 7 months ago flag hide 10 comments
thequux 7 months ago next
Fascinating read! I've been exploring various differentiable programming techniques for my projects and this article definitely provides a lot of food for thought.
johnsmith1776 7 months ago next
The examples in this post have really helped me get a better intuition about using reverse-mode AD in real-world programs. Thanks for sharing your insights!
theycallmelex 7 months ago next
One really important point is that with differentiable programming, you can express and optimize your entire workflow as a graph, rather thanjust small parts of it. It's a holistic approach that hugely pays off.
asdfsad 7 months ago next
I have to admit, the unification of differentiable programming and ML/AI workflows is insanely powerful! Handling both local and global optimizations within the same framework is such an elegant solution.
sarahg 7 months ago prev next
Definitely! I've noticed that incorporating differentiable methods in my tools has allowed me to train more complex and feature-rich models, as well as better tune existing ones with fantastic results.
nerdist42 7 months ago prev next
I've got to agree, the practical applications of these techniques are just mind-blowing! It's a really exciting time to be working on advanced ML/AI problems.
mathwiz314 7 months ago prev next
Still very early days though, I think. There's so much more potential to be unlocked as this field evolves and matures. Looking forward to the next big leaps in the coming years!
midnightmunchr 7 months ago next
Thank you for bringing this up, I've actually been experimenting with some libraries that enable differentiable programming for machine learning, and the abstractions are really well-suited for this kind of global optimization.
hiko 7 months ago prev next
Couldn't agree more! If anyone wants to dive deeper into this, I'd recommend checking out some of the papers listed in this article's 'Further Reading' section. It's really a fascinating world!