125 points by codescholar 1 year ago flag hide 11 comments
johnsmith 1 year ago next
This is quite an interesting approach to neural network pruning! I'm excited to explore the implications of this new research.
anonymous 1 year ago next
I agree! I've been searching for a new way to optimize my neural networks and this might just be it.
randomguy 1 year ago next
Has anyone tried this technique with TensorFlow, PyTorch, or another major ML framework yet?
coderx 1 year ago next
According to the study, the authors used their method exclusively with TensorFlow so far. I wonder how it fairs against Keras.
algo_geek 1 year ago next
Implementing the pruning approach as a TensorFlow callback seems like a promising way to integrate these ideas seamlessly. This CNN tutorial may provide some inspiration for the implementation: (link)
doe_jane 1 year ago prev next
Any ideas how this could be applied to deep learning?
nnmaster 1 year ago next
It could be applied to deep learning by incorporating pruning methods earlier in the training process, making networks more efficient overall.
ml_lover 1 year ago prev next
One potential problem I see is that pruning reduces the computational power required but leaves the risk of losing some accuracy. What do people think about finding ways to maintain accuracy levels?
quant_computing 1 year ago next
Maintaining accuracy may be possible by using a fine-tuning process where the network is retrained after pruning. This should help refine predictions without relying on a large subset of the pruned nodes.
optimizer123 1 year ago prev next
I wonder if incorporating techniques like dropout during training processes could also assist in maintaining accuracy without the need for excessive overhead post-pruning.
researchsharing 1 year ago next
@ml_lover @quant_computing @optimizer123, the mentioned research suggests that maintaining accuracy is attainable with retraining the remaining nodes using a smaller number of epochs. Do refer to the study for more technical details.