120 points by neuromancer 6 months ago flag hide 15 comments
deeplearningfan 6 months ago next
This is a really interesting approach to neural networks compression! The potential for reducing computation cost and memory footprint could truly revolutionize the way we deploy deep learning in resource-constrained environments. Kudos to the authors!
neuralninja 6 months ago next
I totally agree! The exploration they did on quantization and pruning techniques has resulted in impressive results. Hoping to see production-ready implementations soon to try it out by myself.
aiwanderer 6 months ago next
Any early sketches of the implementation are already available at their repository on GitHub. Maybe some of us can start experimenting with it now and provide some early feedback:
hardwarehacker 6 months ago next
*raises hand* Got my eyes on you, paper! Will give it a try and report back if I find any gotchas.
mathguru 6 months ago prev next
The theory behind the paper is solid, but I would be interested to see how this holds up in practice, particularly in large-scale real-world use cases like NLP or complex CV tasks.
binarybard 6 months ago next
While model compression alone might not be sufficient for memory-starved applications, I think these results shall open up avenues to explore new approaches in conjunction with compression techniques - for example, incorporating novel memory allocation methods.
opinionatedophelia 6 months ago prev next
Before we start celebrating, remind us again of the number of other 'magical' compression techniques that were published and failed to materialize in similar use cases. How is the quality of the compressed networks during inference?
deeplearningfan 6 months ago next
@Opinionated Ophelia - the researchers have claimed an overall accuracy drop of <1% over their multiple use cases. They have documented it in tables of the paper you can find in Appendix A.
pseudocodecharlie 6 months ago prev next
As someone who has done some work on knowledge distillation, I can attest that model compression is an important line of research; particularly in mobile-dedicated applications. Looking forward to seeing how this progresses!
datadelta 6 months ago prev next
How about performance gains in deploying these on mobile devices, and the extra computation needed for coding and decoding? Has that been evaluated performance-wise too?
deeplearningfan 6 months ago next
@DataDelta The authors discussed and provided initial results in the last section of the paper, but that is surely an area where there is room for growth. Maybe they did not focus on measuring this, but I suspect there will be follow-up research.
quantizedqueen 6 months ago prev next
Regarding low-bit quantization, I wonder how they tackled the gradient approximation issue, considering most of the solutions published in the past had concerns about optimization. Looking forward to testing this.
optimizationoracle 6 months ago next
From the paper, the authors talk about the Hessian-aware quartile quantization for weight values, which could help with the optimization concerns in quantization. Check sections 3.3.4 and 4.2 for more details about it.
sporadicspike 6 months ago prev next
What kind of specific pruning techniques did the authors use? Were there any significant performance improvements over their previously published techniques?
deeplearningfan 6 months ago next
@SporadicSpike The authors employed what they have termed