The Fault In Our Models

One of the longest-tenured Algorithms tools at Stitch Fix is the Demand Model, owned and operated by a team named The FED (Forecasting and Estimation of Demand). As one can imagine, an accurate demand forecast supports running an efficient business and is invaluable to a number of applications from how to staff warehouses, to guiding merchandise purchases.

The demand model has served Stitch Fix well over the years. By taking advantage of the remarkable assortment of data infrastructure and modeling tools, The FED has been able to forecast the demands for Stitch Fix’s products as our offerings grow in scope and complexity. At a high level, the demand model is a sprawling state machine that models clients in aggregate across different states of engagement with the business. Woven together, this state machine with action probability models produces our forecast of demand, as well as client counts, and other useful client activity metrics. At a lower level, the demand model is an implementation of an in-house framework built specifically to encode relationships. This allows us to define client state metrics as functions of input signals and other computed values. The framework is agnostic to what those functions are, allowing us Data Scientists the freedom to define relationships as simple as the sums, or as complicated as a convolutional kernel.

Most commonly, the functions that predict the transition mappings between states are generalized linear models (or GLMs), that we fit using a variety of independent signals, including exogenous events. These events can be one time occurrences or cyclic phenomena. Examples of such factors include: holidays, credit incentives, website outages, or even international pandemics. Explicit inclusion of these variables help our models avoid absorbing this variance into other factors.

For instance, when Stitch Fix introduced our new Direct Order offering (the ability to buy clothes on demand rather than in a Fix) we offered a discount for existing clients to try out this new feature. As expected, this created a spike in demand beyond what is typical client behavior. We engineered a feature to represent the dates of the credit offers to include as a dummy variable in the GLM predicting Direct Order demand to avoid a falsely elevated future prediction.

Each month, the FED re-fits our models to examine the past, adjust for recent trends, and identify any of these unaccounted for exogenous shocks. Recently, we noticed a cyclical pattern of strong manual fix¹ orders for about 4-5 weeks at a time. They didn’t fit any of our seasonality curves, nor did they align with holidays, website changes, or credit offers.

In a brainstorming session, a teammate mentioned that irregular but cyclical patterns reminded her of lunar and star orbits, which don’t abide by our heliocentric calendar. With nothing to lose, we dug into it - with star charts and astrological calendars. Lo and behold, these periods of strong demand coincided perfectly with times when Mercury was in retrograde!

Apparent retrograde motion is a description of a perceived relative motion; it is when the motion of a planet appears to be in the direction opposite to that of other bodies within its system, as observed from a particular vantage point. We humans standing amidst this galactic flotsam have become accustomed to watching celestial bodies drift across the sky predictably: the planets crawl west to east across a blanket of stars, and the sun and moon rise in the East and set in the West (well, the Moon actually orbits Earth from west to east, like the majority of satellites, but supersynchronous orbit is a discussion for another day). In the case of Mercury, it completes its orbit in a shorter period of time than we do, so it periodically overtakes us. During this time, Mercury will appear to move in the opposite direction, in retrograde, until Earth passes back, and the apparent motion returns to normal, or prograde.

With the creation of horoscope apps and social media accounts, astrology’s popularity is on the rise like our ascending sun in the morning. Often we hear people ascribe a variety of blame to Mercury’s retrograde motion. Mercury is the planet that governs change (c.f. mercurial), as well as travel, communication, logic, and technology. As such, its reverse motion can wreak havoc on any number of those elements; a primary astrological principle is “as above, so too below”. From misunderstandings between friends to lost mail and packages during an epoch of backwards celestial movement, it is clear that trickster Mercury has a strong grip on our lives².

But what does this have to do with our client behavior and why might we be seeing it in our data?

Retrograde is thought to be a great celestial pause - a time to reclaim, redefine, and reconnect³ with ourselves. It can be a great time to think critically about our values, our profession, or even our sense of style. Mercury is unique in that the planet is thought of as neither masculine nor feminine, neither lucky nor inauspicious, neither diurnal nor is the planet nocturnal. A true chameleon and blank slate, we can use this period to revise our aura and outward projection. What better way to do this than to try a new trend we admired on the catwalk but never thought we could pull off? Or perhaps this time of reflection leads us down a new career path, one where we don’t have to abide by a strict dress code. Retrograde might be the ideal time for our customers to explore their wardrobe’s full potential.

Looking at the harmonic function swept out by the relative distance from the Earth-mother Gaia to Mercury, one can get a sense of how this magnitude contributes to the estimated demand curves. We can anticipate an impact from the periodic appearance and magnitude of Mercury’s retrograde motion on the demand sequence we observe at Stitch Fix. As previously mentioned, if we wish, we can include exogenous factors in our GLMs. With the incorporation of the Retrograde Index, we are able to accurately account for periodic increases in historical client behavior, and forecast when we should expect similar surges in the future.

Celestial bodies in retrograde have had an indelible effect on Science, our models of the solar system, and forecasting as a study. Johannes Kepler’s first work under the legendary Tycho Brahe was to assess and account for the mysterious orbit of Mars – in particular, to show why the apparent retrograde motion was compatible with the circular orbit predicted by the theory, but not evidenced by Brahe’s precise observations. After a five year study of this orbital perplexity, and a complicated model resolving the issue, Kepler instead threw out his new model–over eight minutes of arc and two observations–to formulate and argue for heliocentrism⁴.

So too is a modern forecaster presented with residual error, left grasping in the dark for exogenous factors that may serve as helpful predictors. One does not expect that the heavenly bodies are moved by market forces, and as such we are forced to accept that the fault lies not in our stars but in our models.

Notes: