In most businesses, the Client Experience (CX) organization is viewed as a cost center, with its contribution to revenue taken for granted and system interventions evaluated primarily from the perspective of operational efficiency.

At Stitch Fix, we are using algorithms to help turn this notion on its head. For us, CX is an essential part of how we build personal, valuable relationships with our clients. Every human interaction is an opportunity to strengthen a bond and improve our service for that individual client. The goal of our algorithmic development for CX is to maximize overall value minus cost. Sometimes this means decreasing costs by making CX more efficient. However, there is even more upside in increasing client long term value (LTV).

Our CX agents are highly-trained, very knowledgeable about our business, and seek to do what is right for our clients and to strengthen our relationship with them. As is often the case at Stitch Fix, we try to use algorithms to bring out the best in our human workforce, letting humans focus their efforts on what they do best by having machines do what they do best.

Through algorithms, we attempt to shift agents’ time to more value-driving activities by (a) removing tasks better handled by machines and (b) surfacing new opportunities for service improvement and relationship building, beyond what clients are contacting us about. This includes both adding value to existing client contacts and, crucially, it means proactively reaching out to clients who have not contacted us, fixing issues they may not even know they had, and improving service in ways they did not think possible. This proactive outreach is not only a pillar of our value-adding approach, it also allows an avenue for estimating the value of CX interactions through experimentation, which in turn helps with the value-maximization conceptual flip.

Such big changes don’t happen overnight, and require a long-term learn-as-you-go collaborative effort between algorithm developers, engineers, UX designers, CX managers and CX agents. We are still learning, and expect to be doing so indefinitely. The goal of this blog post is not to dive into details of the technical challenges we faced, but rather to describe our working vision and to give a high level overview of some of the components that we’ve built out so far. We hope that this will inspire other people in the community who are solving similar problems.

Ticket categorization

Stitch Fix CX agents label each ticket they solve according to an intricate hierarchical classification scheme for the purposes of post-hoc reporting. As a first algorithmic solution, we built a system that understands and classifies these tickets upon creation. Automating this menial task saves a bit of agent time. It also allows the CX org to use the categorization for ticket routing, getting clients quickly to the right agent, instead of just for reporting. More importantly, its infrastructure has served as a platform for the further interventions described below.

We implemented a hierarchical supervised learning algorithm. We encode the text data coming from both the subject and body text of the ticket along with client, order, and other contextual data. We then make all that flow through a hierarchical classifier to predict the issue type.

The right Data Platform infrastructure tooling is critical in both the training and serving sides. We use the Data Highway to communicate with an engineering service that integrates with Zendesk. Our main algo-cx service receives serialized ticket objects, fetches data from online stores (all of which we built using Platform tools, in the spirit of Engineers Shouldn’t Write ETL), then requests a classification from the classification service. We again use the Data Highway to communicate with the engineering service that updates the Zendesk ticket on our behalf. We add to all that extensive monitoring, logging, and alerting components. This would be a daunting path without the right tools, but with our Data Platform services we could focus more of our time and energy on algorithmic excellence.

Automation

Only very rarely do clients contact CX just to chat about the weather; most of the time they have something that needs to be addressed. We log the actions that agents take in our various internal systems as they are solving tickets. Treating these actions as labels, we trained classifiers (with a similar architecture to the ticket categorization model described above) to infer what action(s) are likely to be taken on an incoming ticket. For cases where (a) we are very sure that the ticket is about a particular simple action request (from the ticket categorization algorithm above), and (b) the action is of a type where the client would likely prefer speed of response over human touch, and (c) we are very sure that the agent would take a particular action on that ticket—with certainty thresholds defined in partnership with leaders in the CX organization—then we automate that action. This gets clients quick responses when they want them, and saves valuable agent time, removing some of the tedium from their work and allowing them to focus on what they do best: interacting with the client as a human.

Guided solving sidebar app

Our CX org is a power user of Zendesk; as a cross-functional effort with Engineering, Product, Design and CX, we have built a sidebar app using the Zendesk App Framework. This allows us to surface contextually-relevant client information within the ticket-solving UI, to provide shortcuts to expected actions, and to make algorithmic recommendations (discussed below), all to help the agent better and more efficiently serve the client.

Actions recommendations

Stitch Fix CX agents have access to a wide variety of tools and actions to satisfy the client; it is not always obvious which action or set of actions would be best in a given context. Our app can help with guidance. Learning from the actions and outcomes of other tickets, we recommend actions that are most likely to be client positive.

On any given ticket, there is only partial feedback, meaning that we observe the outcome only for one of the many possible actions, which makes the problem by nature a counterfactual estimation one. Trying random actions to collect data is prohibitive due to the combinatorial large number of cases and the possible negative effect on client experience. To address that, we use a multi-armed contextual bandits framework. The contextual part is a combination of a heterogeneous treatment effect (HTE) model and a more traditional prediction model. Once the distributions of HTE for each action are estimated, we use Thompson Sampling to surface a recommended action to the agent through the app.

Reply template recommendations

Agents spend a significant amount of their ticket solving time writing back to the client. We wanted to automate the part of that interaction that can be handled with algorithms and save time for the agent to focus more on client satisfaction.

For some types of tickets, agents use reply text templates as starting points for their response. This fits nicely within a learning to rank framework. We encode the ticket text, subject text, reply template text, client and order info and make them flow through a ranking component that ranks the available templates and surfaces the top ones to the agent. The agents are then free to either ignore the template suggestions or adopt one of them and build on the top of it in order to sympathize, empathize and improvise - the things that only humans can do.

Proactive CX (and the problem of attribution)

One of the reasons that CX tends to be viewed as a cost center is because it is much easier to compute the costs of CX than it is to attribute client outcomes to CX actions. The causal chain between a CX action and the next Fix outcome is long and littered with confounders. In addition, there is selection bias; the clients who reach out to CX are not a random sample of clients.

But our CX organization is committed to the well-being of all of our clients, not just those that vote for it. Our CX team anticipates the needs of clients - even those who have not initiated contact - and routinely reaches out to them.

We are actively developing expansions to the scope of this proactive outreach program by using an algorithmic approach. Through modeling, we can identify latent issues and opportunities to improve the client’s experience, ones that may not be apparent to a CX agent or stylist reviewing a client’s profile and history with us. We can then create tickets for some (but not all) of these cases, learning over time which ones are indeed beneficial interactions and which ones are not. This should not only let us improve our clients’ experience directly, it also gives us more insight into the impacts of the CX organization than would otherwise be possible … and this understanding, in turn, will let us continue to improve it over time.

This expansion of proactive outreach is a major and tricky business process change. Initial tests have highlighted many practical challenges; switching the business model of an org is a long and arduous process. But we take heart from the impacts we have seen thus far, the infrastructure we’ve built, and the cross-functional alignment in the vision, and we continue down the road. Expect another blog post on this in a year or two.

More human humans*

From a consumer perspective, Stitch Fix is transforming the way people find what they love through the combination of machine learning and human stylists. This pattern is repeated within our CX team. We combine algorithms and human judgment to elevate our CX service. Through ticket classification, automation, algo-guided solving, and proactive outreach, algorithms are helping Stitch Fix to transform how we all think of CX: not just as a cost center for the business, but rather as a value center.