741 points by neural_linguist 6 months ago flag hide 33 comments
theoephraim 6 months ago next
Exciting to see more open source tools for deep learning language modeling!
altair 6 months ago next
Couldn't agree more, this looks like a promising framework. Has anyone tried using it for real-world applications yet?
josephmisiti 6 months ago next
I have been testing it out for the past few days and can report that it's quite easy to use for simple tasks. Hoping to push it to its limits soon.
sindresorhus 6 months ago prev next
I like that the team has made it compatible with different hardware platforms. Great work!
rem 6 months ago next
Do you think they might support GPU clusters in the future? I'm looking for an efficient solution to train large models.
sindresorhus 6 months ago next
They did actually mention something about it on their roadmap. Let's hope they manage to deliver.
pointfree 6 months ago prev next
Do you know if this framework supports LSTM layers? I'd really like to test out some ideas I have for them.
onury 6 months ago next
Yes, they have added extensive support for LSTM layers and even mention GRU layers in their documentation. Give it a try!
mjackson 6 months ago prev next
Great job to the team for making such a usable and powerful framework. I'm excited to check it out further.
leereilly 6 months ago prev next
I've always wanted a powerful and *flexible* framework for building language models from scratch. Big props on delivering this!
idriselle 6 months ago prev next
This looks amazing, but I'm worried about the documentation. Many frameworks lack proper documentation, which is really a shame. Have you tested it out yet?
othiym23 6 months ago next
Yes, I have actually tested the documentation and can confirm it is well-written, extensive, and clear. Great job, maintainers!
idelcoyo 6 months ago prev next
The project's active maintainance is especially impressive. I'm glad they quickly ironed out the bugs in the initial releases.
steveklabnik 6 months ago prev next
I like how they provide compatibility with different deep learning libraries. It's nice to see options for integration with TensorFlow and PyTorch for example.
pgjones 6 months ago next
Do you think they can maintain this compatibility as their framework evolves? I'd like to see how they tackle that challenge.
steveklabnik 6 months ago next
Absolutely. The development team mentioned that they're working closely with those libraries to maintain integration even as they make updates to their framework.
robinpowered 6 months ago prev next
I really appreciate the variety of examples, demos, and tutorials in this framework's repository. It makes the barrier to entry much lower for newcomers.
stormglass 6 months ago next
Indeed. It was very easy to get started with the basics and expand on that knowledge. Big props to the team!
spredfast 6 months ago prev next
Have any of you encountered the problem where the training and evaluation of the model becomes unstable? I experienced some nasty bugs and crashes in my testing.
dbashford 6 months ago next
Yes, actually. I noticed that the problem was due to exploding gradients and suggest implementing gradient clipping as a solution.
spredfast 6 months ago next
Thanks for the suggestion, I'll give that a try! Hopefully that solves the stability issues.
mxcl 6 months ago prev next
Does this framework handle transfer learning well? I'm specifically interested in adapting models to specific tasks or domains.
stevensacks 6 months ago next
Yes, it does! The team even provides built-in functions for transfer learning. I have been playing around with this feature and it works quite well.
hongbin 6 months ago next
Have you tried BERT or other transformers within their system? I keep hearing about the amazing results using them.
stevensacks 6 months ago next
Yes, they have extensive support for BERT and transformers. I'm currently playing around with those components and like what I'm seeing.
leereilly 6 months ago prev next
This framework has really ramped up my productivity and delivery times for deep learning projects. I'm recommending it to all my colleagues!
jakearchibald 6 months ago prev next
Is it easy to implement your own custom loss functions? I tried it and found that it took some effort, but is now straightforward.
johndoe 6 months ago next
It seems doing so requires diving into the source code a bit, but the team is working on making a simpler way to implement custom loss functions.
cyan 6 months ago prev next
Anyone know what's in the future for this framework? I'd like to know if they're considering adding more advanced features.
alaisi 6 months ago next
According to their repo, they want to include automatic optimization of hyperparameters, integration with more hardware platforms, and more demos/tutorials.
seblucas 6 months ago prev next
Just got started with this. I'm impressed by the ease with which I can get started, even without having too much prior knowledge in language modeling.
darkskyapp 6 months ago prev next
The various pre-trained models bundled with the framework are very well-curated. It's wonderful to have a jumpstart like this for projects.
dhh 6 months ago next
It's certainly one of the better curated collection of pre-trained models that I've seen lately. I like the attention to detail.