The Pursuit of Lift. A tool to spot why many ML projects… | by Ian Xiao | Dec, 2020
The S.P.O.T. Framework
From working with companies and teams of various sizes and industries to operationalize ML, issues converging to four (4) key areas. The S.P.O.T. Framework helps teams spot (pun intended) the gaps in each of the four areas.
1 — Strategy (S). Do we have a clear strategy that Data Science (and other) teams can understand? “Strategy” is a big and fluffy word. In essence, a good strategy succinctly captures the “how” of our company needs to stay alive (e.g., keep making a profit in a competitive market); it is something everyone from top to mid-management can articulate and describe how their team can contribute to it with simple words.
With a good strategy, data scientists can define the right “target” (e.g., a churning customer means people who cancel their subscriptions, instead of decreased usage by a certain percentage) for their ML solutions. Most importantly, Data Science teams can understand what the minimum lift of their solution needs to be.
2 — Product (P). Do we have enough economically viable product offering? Once the ML solution identifies an opportunity, we need to do something to intervene — and ultimately, change the outcome. Depending on the actual use case, “product offering” has different meanings and forms.
In a Marketing context, an offer can range from simple email reminders, discount coupons on a new product or service to human intervention via service calls. In an Operation and Risk Management context, an offer (also known as “treatment”) can be an action, such as a staff opening an intake to investigate a fraudulent transaction or the system applies automatic correction rules.
However, one of the most prominent challenges is to have enough economically viable offers to select from. Every offer has a cost. We need to decide if it is cheaper to do nothing, do something, or let the customer leave.
3 — Operation(O). Do we have the right capability to execute offers? This is the Last Mile Problem of ML: once we identify the opportunity and find the right product offering, teams of specialists or frontlines have to prepare and deliver the offer. They must do it well to achieve the intended outcome — this is when the theoretical lift turns into actual monetary results.
For example, in a cross- or up-selling setting, a marketing specialist designs and adds offers and their specifications (e.g., how much discount, who is eligible, what product it applies to) to a system; then, a sales staff needs to present the offer to a customer who’s likely to purchase verbally in person, via the phone, or other digital means. How the staff nurtures conversation and presents the offer greatly impacts the actual lift the Data Science teams theorize.
4 — Technology (T). Do we have the right technology to execute the offers? “Technology” is another big word since it is so ubiquitous and can manifest in many forms. Generally, technology plays two key roles: a) enable and b) automate.
For example, to develop and run an ML model, we need a database and ML toolkit; to create and manage marketing offers, we need a content platform; to deliver offers, we need integration between the content platform and channel technologies (e.g., the tool sales staff use in-store, call centers, web-based chat windows).
In terms of scaling (minimize incremental cost as the volume of customers or transactions increases), we need technology to automate many repetitive tasks instead of having someone do it manually — wage is often more expensive than software licenses and electricity.
Read More …