MIT CODE 2024 Plenary Session 3: Jessica Hullman and Avinash (Avi) Collie

Avinash (Avi) Collis – Assistant Professor, Carnegie Mellon University
The Consumer Welfare Effects of Online Ads: Evidence from a 9-Year Experiment
Research on the causal effects of online advertising on consumer welfare effects is limited due to challenges in running large-scale field experiments and tracking the effects over extended periods. In this study, we analyze a long-running field experiment of online advertising, launched in 2013, in which a random 0.5% of all users are assigned to a group that does not see ads (the “no-ads hold-out group”). We recruit a representative sample of Facebook users in the ads and no-ads groups and estimated their welfare gains from using Facebook using a series of incentive-compatible choice experiments. We find no significant differences in welfare gains from Facebook or time spent on Facebook between the two groups. Our estimates are relatively precisely estimated reflecting our large sample size (53,166 participants). Specifically, the minimum detectable difference in median valuations at standard thresholds is $3.18/month compared to a baseline valuation of $31.95/month for giving up access to Facebook. That is, we can reject the hypothesis that the median disutility from advertising exceeds 10% of the median baseline valuation. Our findings suggest that either the disutility of ads for consumers is relatively small, or that there are offsetting benefits, such as helping consumers find products and services of interest.

Jessica Hullman – Ginni Rometty Professor of Computer Science, Northwestern University
The Rational Agent Benchmark for Data Visualization
Understanding how helpful a visualization is for decision-making is difficult because the observed performance in an evaluation is confounded with aspects of the study design, such as how useful the information that is visualized is for the task. I will discuss how decision-theoretic frameworks that conceive of the performance of a Bayesian rational agent can transform how we design and evaluate visualizations and other decision-support interfaces, such as explanations of model predictions.