Designing ML Orchestration Systems for Startups | by Matt Zhou | Oct, 2020
A case examine in constructing a light-weight production-grade ML orchestration system
I not too long ago had the possibility to construct out a machine learning platform at a healthcare startup.
This article covers the journey of structure design, technical tradeoffs, implementation particulars, and classes realized as a case examine on designing a machine learning orchestration platform for startups.
As the machine learning tooling ecosystem continues to mature and increase, there isn’t a scarcity of choices for constructing out a machine learning orchestration layer for manufacturing data science pipelines. The extra related problem is with the ability to decide which instruments match the necessity circumstances of your group.
The valuable lesson was structuring the supply management course of utilizing instruments already acquainted to data scientists.
The last toolset we ended up with was:
- Version managed SQL scripts underpinning supply dataset extraction
- Preprocessing code abstracted by a Python runner script executed inside a Docker container, accessing checkpointed information at relaxation throughout the information lake.
- Model coaching code abstracted inside a Python mannequin class that self-contained capabilities for loading information, artifact serialization/deserialization, coaching code, and prediction logic.
- Boilerplate Flask API endpoint wrappers for performing well being checks and returning inference requests.