All The AWS re:Invent 2020 Machine Learning Releases And Why They Matter | by Sam Palani | Dec, 2020
Last week the annual AWS technology conference re:Invent 2020 kicked off virtually. Typically a week-long physical conference held in Las Vegas this year, with the ongoing COVID19 pandemic, the conference moved to a virtual 3-week event. If you have paid attention to re:Invent, the past couple of years, machine learning has always taken center stage during re:Invent keynotes. So much that it has overshadowed other releases, this year, AWS decided, and rightly so, that Machine Learning deserved its own dedicated keynote.
Why should you care? While most machine learning experiments might start locally, you eventually end up in the cloud once you start doing machine learning at a production scale. Besides, when you look at these production workloads, they are overwhelmingly composed of algorithms in TensorFlow, PyTorch, and MXNet. And finally, over 90% of cloud machine learning based on TensorFlow and PyTorch runs on AWS.
Let start with the hardware.
AWS Trainium A machine learning chip custom-designed by AWS, specifically for training machine learning models in the cloud. This is the second silicon from AWS after AWS Infrentia and shares the same AWS Neuron SDK for developers. Why it matters? Cost-effective and high performance for your deep learning training on the cloud.
EC2 instances powered by Habana Gaudi EC2 instances powered by Habana Gaudi accelerators. Also available via Amazon SageMaker, AWS ECS, and AWS EKS.
Why it matters? Up to 40% better price-performance compared to the current GPU-based EC2 instances. This, in turn, enables cost-effective scaling for your deep learning training.
Amazon SageMaker is a fully managed machine learning service from AWS that enables you to build, train, and deploy machine learning. It is also the most robust machine learning services on the cloud today. Here are some of the SageMaker releases this re:Invent.
SageMaker Data Wrangler Enables you to process, transform, and visualize the data needed for machine learning with a few clicks. Why it matters? Data preparation undifferentiated task that consumes most of the time in machine learning. Data Wrangler makes it easy to clean and preprocess your data with over 300 built-in transformations while giving you visibility on what happens behind the scenes. You can also author reusable custom transformations in PySpark, SQL, and Pandas.
SageMaker Feature Store A purpose-built repository enables you to store, update, retrieve, and share features. Why it matters? Provides an easy way to reuse and share enriched features and avoid duplication of work within teams. Your team can check-in and check out features similar to how they work with code repositories. Offline and online options make it easy to consistently achieve consistency during training and inference when you need low latency.
SageMaker Pipelines A CI/CD service built for machine learning workloads.
Why it matters? You could stitch together a pipeline using a combination of AWS Lambda, Step Functions, and CodeCommit in the past. With pipelines, you can create workflow straight from SageMaker studio. Pipelines have built-in templates that enable you to get started quickly, and you also have the ability to create your own templates using cloud formation. You can define custom stages and build both automatic and manual approvals. Tip: Before trying out pipelines, you need, or your administrator needs to enable project templates in SageMaker studio.
SageMaker Profiler For Debugger Enables you to avoid bottlenecks and maximize resource utilization during training with a single parameter. Profiler is an additional capability of Sagemaker debugger, and you can enable it with a couple of additional parameters without any significant code changes. Why it matters? In the past, you had to write custom scripts or combine dashboard from CloudWatch to monitor resource usage while training was in progress. Profiler makes it easy to detect under or over-utilized resources and take remediation steps. The profiling results can be visualized from within SageMaker studio and also available via the SageMaker debugger API.
SageMaker Clarify A new feature that enables you to detect bias and explain model behavior. The reports are available directly from SageMaker studio, or you can get them from the S3 bucket where they are stored.
Why it matters? Bias in your dataset and models will lead to inferior predictions. With Clarify, you can detect both pre-training biases, that is, the bias already part of your dataset and the bias that is part of your trained model. In addition to bias, clarify also explains why the model the decisions it did. Clarify uses SHAP to explain the contribution each input feature makes to the final decision.
SageMaker Distributed Training While distributed training always existed in SageMaker; the new releases enable data parallelism and model parallelism with a few lines of code. Why it matters? New Distributed Training on Amazon SageMaker makes it possible to train large, complex deep learning models up to two times faster than current approaches. Distributed training with Amazon SageMaker’s Model Parallelism engine can efficiently split large, complex models with billions of parameters across multiple GPUs by automatically profiling and identifying the best way to partition models.
SageMaker Edge Manager allows developers to optimize, secure, monitor, and maintain machine learning models deployed on fleets of edge devices. Why it matters? Enables you to manage models on a fleet of edge devices and continuously monitors the model fleet to detect deterioration. It applies specific optimizations that make performances up to 30% compared to hand-tuning the models.
SageMaker Jumpstart provides a set of solutions for the most common use cases, such as fraud detection, predictive maintenance, and demand forecasting, that can be deployed readily with just a few clicks.
Why it matters? Developers that are new to machine learning find it hard to get started. Even experienced practitioners sometimes find it confusing to scale to meet production demands. With JumpStart, you can quickly find relevant information specific to your machine learning use cases.
Machine Learning for Databases? Apart from new releases within the AWS machine learning stack, re:Invent 2020 also had ML expand across other AWS services like Redshift ML and Neptune ML. This enables database developers and analysts with limited machine learning skills and seasoned practitioners to create, train, and run machine learning models directly using SQL code. Why it matters? Developers can directly plug in inference results from their data in RedShift into their BI reports. With Neptune ML, this enables common use cases like building knowledge graphs and recommendation systems.
Amazon Q for QuickSight A feature powered by machine learning uses natural language processing to answer your business questions. Why it matters? Build natural language query capabilities within your QuickSight dashboards with a couple of clicks. Simply put, ask questions in plain language and get answers instantly.
Amazon Lookout for Metrics An AI service that uses machine learning to automatically detect and diagnose anomalies in business and operational time series data. You can connect to multiple sources like S3, RDS, and third-party SAAS providers like Salesforce. Why it matters? Enables you to detect anomalies in virtually any time series data. In addition to detection, it also helps identify the root cause of the anomaly. You can use this via the console in a few clicks or programmatically integrate it with other applications via the API.
Machine Learning for industrial services received a special focus during this year’s re:Invent releases.
Amazon Monitron A machine learning solution to detect abnormal behavior and enable preventive maintenance in industrial machinery. Monitron has two main components: the Monitron sensors that attach to the machines to measure vibrations and temperature. The Monitron gateway receives input from the sensors and sends it to the AWS cloud to process and apply machine learning. The final component is a mobile app that can receive alerts when abnormal behavior is detected. Why it matters? An end to end solution for anomaly detection for industrial machinery, including all the hardware, software, and infrastructure. This makes it easy to deploy a turnkey solution in a matter of days.
Lookout for Equipment Works similar to Monitron, but you can use your own sensors and hardware.
Lookout for Vision Expands the anomaly detection capabilities to detect anomalies within a set of images with built-in sophistication to handle camera angle lighting variations.
Panorama Appliance If you have existing smart cameras, you can use the Panorama appliance to process feeds directly and send them to the AWS cloud for further analysis. The appliance comes with pre-built models in the appliance, which are optimized for different industry domains.
How these matter? Leverage your existing investments in industrial hardware and extend them with machine learning capabilities quickly.
DevOps Guru Another AI service that enables you to detect behaviors that deviate from normal operating patterns so you can identify operational issues long before they impact your customers. Why it matters? Deploy a machine learning-based solution that automatically anticipates DevOps issues before they appear with no manual setup or machine learning expertise required.
Amazon HealthLake A service that enables you to transform data using specialized machine learning models to identify trends and make predictions. Why it matters? A HIPPA eligible service that organizes data chronologically can be used to build machine learning models in SageMaker. The services also enable sharing data with other providers using standard file formats.
That was a lot of machine learning and artificial intelligence releases. AWS re:Invent is still underway, and you can register for free and watch any of these or any of the other technical deep-dive sessions. AWS also offers a generous free tier for a year. If you do not already have an AWS account, I highly recommend signing up for the free tier and trying out these services.
Read More …