67 points by mlenthusiast99 6 months ago flag hide 16 comments
finance_geek 6 months ago next
Fascinating work! I've been playing around with LSTMs and stock market prediction lately. I'm curious, which dataset did you use for training your model?
modelmaster 6 months ago next
Hi finance_geek! I appreciate your interest. I used the well-known S&P 500 dataset. I integrated it with historical prices and additional features such as volume and RSI.
data_scientist 6 months ago prev next
Could you share the architecture of your LSTM model? I'm currently trying to optimize the number of LSTM layers and units. Your insights would be valuable.
modelmaster 6 months ago next
Sure! My model consists of three LSTM layers with 100, 50, and 25 units respectively, followed by a dense layer with a single unit for prediction. I used dropout and recurrent dropout to prevent overfitting.
quant_algo 6 months ago prev next
What preprocessing did you apply to your dataset before feeding it to the LSTM model for training? Did you normalize or use any differential techniques?
modelmaster 6 months ago next
For preprocessing, I normalized the stock price features and added a few key technical indicators (moving average, RSI). I applied the Min-Max scaling technique for normalization and a window size of 10 timesteps. Concerning differential techniques, I considered the price change data, but decided not to use it since it added more noise than value in my case.
stat_analyst 6 months ago prev next
Hey all, I'm replicating this in Python, and I wanted to ask about the loss function and evaluation metrics you used. Did you try any other evaluation metrics besides the RMSE?
modelmaster 6 months ago next
Hi stat_analyst. I initially employed a mean-squared-error loss function, but I later found that using mean-absolute-error resulted in slightly better performance. I also utilized the R-squared, adjusted R-squared, and explained variance score as additional metrics. I've found the adjusted R-squared very helpful as it adjusts the calculation of R-squared to take the degrees of freedom into account, thus making it more robust.
algo_creator 6 months ago prev next
I've been struggling with vanishing gradients in my LSTM model. Any tips on overcoming this?
modelmaster 6 months ago next
Vanishing gradients can be a challenge with LSTMs. Consider using gradient clipping, which you can implement to maintain the norm of the gradients to a set clipping value. Another approach is to gradually increase the learning rate every few epochs (e.g., exponential learning rate policy) when training your model.
gpuminer 6 months ago prev next
Did you find GPU acceleration to be beneficial while training your LSTM model? I'm mulling over a GTX 1080 or RTX 3060 to train my deep learning models. Any ideas on which one to choose?
modelmaster 6 months ago next
Absolutely: training machine learning models, especially deep learning models, can significantly benefit from GPU acceleration. Both the GTX 1080 and the RTX 3060 are suitable for training LSTMs. The RTX 3060 will likely perform better due to its more advanced architecture and faster speed. Regarding power efficiency, the newer cards, such as the RTX 3060, have the upper hand. Ultimately, the choice comes down to your budget and your specific use case requirements.
tensorjedi 6 months ago prev next
Interested to know if you experimented with other architectures alongside LSTMs, such as GRUs or other types of RNNs.
modelmaster 6 months ago next
I did experiment with other recurrent architectures, such as GRUs and Vanilla RNNs, but found LSTMs to outperform them with this particular problem. However, depending on the dataset and problem context, those alternatives might prove more effective. It's beneficial to compare various architectures and find what suits your specific use case.
treehugger 6 months ago prev next
Very informative post! I'm concerned about the environment and carbon footprint. Any thoughts on how to speed up training time while decreasing the energy consumption? I'd love to hear any ideas around building sustainable ML and AI.
modelmaster 6 months ago next
Thanks for raising this concern, treehugger! There are several strategies to reduce the environmental impact of deep learning training. You can utilize power-efficient hardware, cloud-based computing services with greener infrastructure, and model parallelism to reduce the overall graph execution time. In addition, consider evaluating techniques such as network pruning, quantization, and low-precision training, all of which help to reduce energy consumption and speed up inference times. Remember: even the smallest adjustments in our processes can yield significant improvements in the collective effort to create more sustainable ML and AI systems.