Machine Learning Operations on AWS

MLOps is a set of practices that combines Machine Learning, DevOps, and continuous delivery to automate the end-to-end machine learning lifecycle.

Scale. MLOps is critical to scaling machine learning workflows. Similar to how DevOps can streamline and operationalize your development and deployment processes, MLOps helps you deploy models faster, achieve reproducibility between experiments, scale your solutions, and maintain consistency in your model quality. By helping your teams collaborate and automating the processes for these elements, your teams move faster.

They are, but the distinction is the support of your infrastructure and its operations overall (DevOps) versus the support of your data science and engineering teams in their specific development of ML algorithms and solutions (MLOps). There are challenges specific to how ML models are developed, deployed, and operated at scale which are distinct from AWS best practices for your overall environment.

This is a common concern, and something best supported by model monitoring infrastructure. Setting appropriate alerts for performance metrics can identify drift, and you may want to institute periodic retraining and validation against new data.

Having a robust cost analysis tool, like CloudHealth, which is included in both of our managed services, to help you track costs and attribute expenses can be critical to this work. You’ll also need to adopt best practices, like ensuring you’ve cleaned up unused resources, using spot instances for experimentation, and taking advantage of the elasticity of native services wherever possible.

Basic MLOps should always be in place for tracking the results of experiments and model tuning.  However, the scale of your MLOps processes and tool sets really depends on the aim of your work, the size of your team, and the maturity of your solution. For small projects with a few ML models that rarely update and have limited impact, a full-fledged MLOps approach might be overkill. But if you find yourself having to update models frequently, coordinate a large number of them, or scale their application, it may be time to begin adopting MLOps. Challenges with collaboration, scaling, experimentation and reproducibility, and compliance and security can all indicate a need to adopt MLOps practices.