As more and more industries move to Machine Learning and Deep Learning, a lot has been written about the shortage of data scientists. This issue has been actively addressed in the last several years with online Data Science courses, specialty programs in universities for Data Science, and tools that simplify model creation (the democratization of data science). The latest approach to mitigating this problem – AutoML – promises to automate the process of model creation and selection, making it even easier to improve the productivity of a single data scientist or business analyst. This week Google announced AdaNet, a TensorFlow based Auto-ML framework for exploring and finding optimal neural network architectures (see link to blog (1) and link to Github repo (2)).
Progress on Auto-ML, such as AdaNet, further increases the need to manage “Production Debt,” the bottleneck that occurs when newly developed models cannot be quickly and efficiently brought to production. To manage Production Debt and scale ML-based value, Auto-ML tools need to be effectively integrated into a complete ML operational lifecycle (MLOps).
For example, AdaNet can already be seamlessly used with ParallelM’s MCenter to create a full production deployment lifecycle for Deep Learning Models:
1. Models created by AdaNet can now be directly imported into ParallelM’s MCenter, deployed into production deployment and managed through their lifecycle.
2. AdaNet jobs can also be launched by MCenter for production management of new model retraining and reselection.
By combining these two capabilities, data scientists or business analysts can quickly create a powerful deep learning model with AdaNet, connect it to MCenter, and rapidly solve real business problems in production.