Business Policy Experiments using Fractional Factorial Designs: Consumer Retention on DoorDash

Title: Business Policy Experiments using Fractional Factorial Designs: Consumer Retention on DoorDash

Authors: Yixin Tang (DoorDash, [email protected]; Yicong Lin (DoorDash, [email protected]; Navdeep S. Sahni (Stanford University, [email protected])

Abstract: This paper investigates an approach to both speed up business decision-making and lower the cost of learning through experimentation by factorizing business policies and employing fractional factorial experimental designs for their evaluation. We illustrate how this method integrates with advances in the estimation of heterogeneous treatment effects, elaborating on its advantages and foundational assumptions. We empirically demonstrate the implementation and benefits of our approach and assess its validity in evaluating consumer promotion policies at DoorDash, which is one of the largest delivery platforms in the US. Our approach discovers a policy with 5% incremental profit at 67% lower implementation cost.

Presented at the 2023-2024 Gary L. Lilien ISMS Practice Prize Competition