Title: Tie-breaker designs
Talk Abstract:
Suppose we have a program to reward top customers. If we later want to measure its causal impact we can compare customers who just barely qualified to those who just barely missed. The statistical method for that is called a regression discontinuity design. In a tie-breaker design, we have three groups: those sure to receive the reward, those sure not to, and a middle group where the decision is based on a random coin toss. This talk shows how statistical efficiency is monotone increasing in the size of the randomized group. A fully random allocation is an A/B test that is 4 times as efficient as an RDD in our model. The cost to randomization is a less efficient allocation to the existing customers that produces an exploration/exploitation tradeoff. Interestingly, there is no benefit to having a sliding scale of reward probabilities beyond just 0%, 50% and 100%. The most recent finding is that the efficiency advantages of tie-breaker designs also hold when local linear kernel regression methods are used.
Much of this work was done for Google and was not part of Art Owen’s Stanford responsibilities.
Take Home Message:
When possible and ethical, there is a lot to gain by injecting a little randomness into a decision rule. The motivating example is customer loyalty programs.
Date and Time:
The talk will be on Tuesday, Jan 26th at 2:00PM PST. (9:00 PM UTC)
Zoom info:
Zoom Link Passcode: 800754
Speaker Info:
Art Owen is professor and chair in the department of Statistics at Stanford University. His teaching focusses on applied statistics including experimental design and Monte Carlo methods. He is best known for inventing empirical likelihood and for work on randomized quasi-Monte Carlo methods. The empirical likelihood is a data driven likelihood that does not require a parametric family of distributions, yet has power comparable to parametric likelihood methods. Randomized QMC is a Monte Carlo method that can approach \(\mathcal{O}(n^{-3})\) error variance. He has consulted on advertising effectiveness and click fraud. He has advised 23 PhD students. He is a fellow of the ASA and the IMS and received the 2021 Senior Noether Prize in nonparametric statistics from the ASA.
