600 points by sundar_natarajan 6 months ago flag hide 11 comments
username1 6 months ago next
This is a really interesting approach to neural network compression. I like how they're using pruning techniques to reduce the network's size.
username3 6 months ago next
I agree, pruning is a powerful technique for compression. I'm curious if they tried using similar methods for fine-tuning the network after pruning.
username3 6 months ago next
I'll have to take a closer look at weight reinitialization then. That seems like a promising way to recover some performance after pruning.
username3 6 months ago next
I believe they briefly mentioned doing some initial testing with convolutional networks, and it seemed to work well.
username6 6 months ago prev next
Yeah, I'm interested to see if these techniques will work well with other types of neural networks beyond what they tested.
username2 6 months ago prev next
I've been working on a similar problem, and this looks like it could be a useful solution. Thanks for sharing!
username4 6 months ago next
Definitely! They discussed using a variation of weight reinitialization to improve post-pruning performance. Check it out in section 4.2 of the paper.
username5 6 months ago prev next
Interesting, I've never heard of that approach before. How well does the compressed network perform compared to the original network?
username2 6 months ago next
They report that the compressed network's performance is very close to the original network's performance. It's impressive!
username7 6 months ago prev next
Great research! I look forward to seeing how these techniques develop in the future.
username1 6 months ago prev next
Definitely! This could have a big impact on deep learning in resource-constrained environments.