234 points by datasciencefan 5 months ago flag hide 23 comments
deeplearningdude 5 months ago next
Great work! I would love to know more about the dataset and model architecture you used.
mlfan 5 months ago next
I am curious if you considered using any other ML model, apart from what you mentioned?
deeplearningdude 5 months ago next
Yes, I tried a few different models, but the LSTM with attention was the most accurate. I'll include a comparison table in my blog post.
coolalgoengr 5 months ago next
Is anybody aware of any industry/academia research on ALGO trading using deep learning?
fintechwhiz 5 months ago next
There is quite a lot of research in ALGO trading and deep learning in finance. Check out papers from NIPS and ICML conferences from the past 3 years.
aiartist 5 months ago prev next
Just out of curiosity, how accurate have your predictions been so far?
financeperson 5 months ago next
Accuracy can depend on the benchmark. Have you compared your results against a naive baseline, like moving average?
deeplearningdude 5 months ago next
I did compare my results against a simple moving average, and the ALGO significantly outperformed it.
datajock 5 months ago prev next
I've tried building a similar algorithm in the past, but I always ran into issues with overfitting. What steps did you take to prevent that?
deeplearningdude 5 months ago next
I used a combination of cross-validation, regularization, and early stopping to prevent overfitting. I'll write up a blog post soon on all the details.
aihunter 5 months ago next
Looking forward to your write-up. Did you also experiment with different feature engineering techniques for the stock data?
deeplearningdude 5 months ago next
Yes, I tried several feature engineering techniques, ranging from simple stats like mean, median and std dev to advanced techniques like wavelet transforms.
newbiewithml 5 months ago prev next
I've been trying to learn more about ML and stock prices recently, this is great inspiration, thanks!
realtimelearner 5 months ago next
I'm planning to study this area. Any tips for me as a getting started?
deeplearningdude 5 months ago next
Start with learning the fundamentals of time series and RNNs. I suggest reading through https://machinelearningmastery.com for beginners.
knowerofstats 5 months ago prev next
The real question is: how well does your algorithm handle the impact of unexpected, global news events on stock prices?
deeplearningdude 5 months ago next
That's an excellent point, and it's one of the areas I'm working on improving. I think incorporating real-time news feeds would be a huge step in the right direction.
nlpwiz 5 months ago next
I'm interested in how you handle long-term dependencies in your LSTM network. Care to elaborate?
deeplearningdude 5 months ago next
I used a multi-layered LSTM with attention. It helps the network to pay attention to crucial input features and avoid losing information from long-term dependencies.
blocksushi 5 months ago prev next
This is awesome! I know how hard ML and finance can be. Did you consider events that could happen in the long run, like Fed rate changes?
deeplearningdude 5 months ago next
Yes, I used a modified version of the FED rate change in my data preprocessing. It impacted the training data significantly.
machinewhisperer 5 months ago next
How can you ensure that your ALGO doesn't have any bias towards specific stocks or sectors?
deeplearningdude 5 months ago next
I divided the data into random batches from every stock, avoiding ordering, and created a validation set out of different stocks.