250 points by alex_coding 1 year ago flag hide 14 comments
john__doe 1 year ago next
This is an amazing breakthrough! I wonder how much more efficient our models will become.
code_monk 1 year ago next
I agree, but we should also consider whether this can lead to overfitting.
ai_enthusiast 1 year ago prev next
This algorithm seems to be a major game changer. Can someone share the research paper?
researcher_xd 1 year ago next
@ai_enthusiast Here's the link to the research paper: [url to paper]
ml_engineer 1 year ago prev next
I've read the paper and implemented this algorithm. I can confirm it works like a charm and the neural network training speed has significantly increased.
data_samurai 1 year ago next
I'm curious, did you encounter any challenges during implementation?
ml_engineer 1 year ago next
@data_samurai I did face some challenges, especially
numpy_guru 1 year ago next
Having similar implementation troubles here. Could @ml_engineer share any tips on how they resolved the challenges?
ml_engineer 1 year ago next
@numpy_guru @data_samurai The major challenge was getting the gradients to work correctly. You'll need to ensure that the learning rate is appropriate. I recommend using a learning rate scheduler too.
optimizer_wizard 1 year ago prev next
It would be awesome if this algorithm could be integrated into common deep learning libraries like TensorFlow and PyTorch.
keras_user 1 year ago prev next
I heard there's a Keras implementation available here: [url to Keras implementation] just released.
pytorch_fan 1 year ago prev next
As for PyTorch users, there's an unofficial implementation available here: [url to PyTorch implementation].
stats_prof 1 year ago prev next
performance_junkie 1 year ago next
@stats_prof I'd love to see some extensive benchmark tests between the current state-of-the-art and this revolutionary