76 points by mlwhiz 1 year ago flag hide 6 comments
dmurph 1 year ago next
This is a great article on end-to-end deep learning workflow automation with TensorFlow and Keras. It's really impressive to see the level of automation achieved here! However, I would love to hear more about the evaluation process. How did you evaluate and compare different models?
tianhe 1 year ago next
Hi dmurph, great question! We evaluated each model based on a set of predefined performance metrics, and also used techniques like cross-validation and test-train split to ensure robustness. We also compared the results with traditional machine learning models. Let me know if you'd like to see more details!
bigqueryfan 1 year ago prev next
Nice work with TensorFlow and Keras! I'm guessing it wasn't trivial to orchestrate everything to work together smoothly. Any tips for managing the workflow between different tools/frameworks?
ayazhan 1 year ago next
BigQueryFan, you're right. Orchestration was a challenge, but we used a combination of tools and best practices. For instance, we used Make for task automation and Docker for packaging and distribution. We also set up alerts and monitoring for any errors or warnings along the way
noahb 1 year ago prev next
Wow, this is incredibly useful. One thing I'm curious about is how you handled data preprocessing and feature engineering. I find that to be the most time-consuming part of deep learning projects. Any tips?
mlgrimes 1 year ago next
Hey noahb, we also faced similar challenges and used tools like TensorFlow DataSets, which provides a nice interface for data loading and preprocessing, along with built-in functions for feature engineering. One tip is to keep experimenting with different data preprocessing techniques so that the model can learn better