Adventures in Why
A Machine Learning Blog
Bob Wilson (he/him/his) is a Marketing Data Scientist at Meta Reality Labs, where he helps introduce the world to the future of technology. Prior roles include data science at Netflix, Director of Data Science (Marketing) at Ticketmaster, and Director of Analytics at Tinder. His interests include causal inference and convex optimization. When not tweaking his Emacs init file, Bob enjoys gardening, listening/singing along to Broadway musical soundtracks, and surfeiting on tacos.
Interests
- Causal Inference
- Convex Optimization
- Theoretical Statistics
Education
Recent Musings
Design-Based Inference and Sensitivity Analysis for Survey Sampling
In this note, we consider sampling from a finite population, without replacement and with unequal probabilities. We seek an estimate of the population mean of some characteristic.
Principal Stratification and Mediation
This post explores principal stratification and mediation analysis as tools for understanding causal effects, decomposing them into direct and indirect components. It covers scenarios like non-compliance, missing outcomes, and surrogate indices, highlighting the importance of assumptions such as no direct effects and no Defiers. Practical methods, including multiple imputation, regression, and matching, are discussed for estimating effects even when key quantities are unobserved. Real-world examples, like marketing lift studies and product funnels, illustrate the relevance of these techniques for addressing complex causal questions.
Interpretable and Validatable Uplift Modeling
In this note, we introduce a method for interpreting and validating the results of uplift modeling. We propose two novel strategies for controlling the Familywise Error Rate in this setting.
Modes of Inference in Randomized Experiments
Randomization provides the “reasoned basis for inference” in an experiment. Yet some approaches to analyzing experiments ignore the special structure of randomization. Simple, familiar approaches like regression models sometimes give wrong answers when applied to experiments. Approaches exploiting randomization deliver more reliable inferences than methods neglecting it. Randomization inference should be the first method we reach for when analyzing experiments.
Sensitivity Analysis for Matched Sets with One Treated Unit
Adjusting for observed factors does not elevate an observational study to the reliability of an experiment. P-values are not appropriate measures of the strength of evidence in an observational study. Instead, sensitivity analysis allows us to identify the magnitude of hidden biases that would be necessary to invalidate study conclusions. This leads to a strength-of-evidence metric appropriate for an observational study.
Projects
A/B Testing
Calculators for planning and analyzing A/B tests
gamdist
Generalized Additive Models in Python
orbpy
Orbit Propagator in Python
Homebrew-Calc
Homebrewed Beer Calculator
Unit Parser
Unit Parser and Conversions in Python
Selected Papers
Interpretable and Validatable Uplift Modeling
arXiv preprint arXiv:2610.37966
Uplift models estimate treatment effects for individuals, but we do not observe individual treatment effects, so the predictions are hard to interpret and validate. A decision-maker often wants trustworthy claims about groups where the treatment is beneficial or harmful, and to know when a treatment benefits some people more than others. Our procedure provides such a set of claims, certified simultaneously at a specified familywise error rate (FWER). We summarize an uplift model by a shallow tree fit to its predictions. Each node of the tree carries two hypotheses, one about the average treatment effect in the node and one about the difference in effects across the node's split; the two kinds are interwoven into a single hierarchy. Splitting the data enables honest estimation in the evaluation set. It lets us choose the direction of each one-sided test from the training half, allowing us to certify the sign of treatment effects in subpopulations, and to compare effects between groups. We demonstrate our procedure offers strong control of the FWER.
Randomization Inference for Matched Pairs with Binary Outcomes
arXiv preprint arXiv:2609.03227
We give an exact randomization-based confidence set for the average treatment effect (ATE) in matched-pair studies with a binary outcome, requiring neither monotonicity nor any distributional assumption beyond the within-pair coin flip. At its core is an analytic solution to the worst-case allocation of attributable effects: two binomial-symmetry lemmas identify the pattern hardest to reject as a single boundary corner, so testing null hypotheses needs no integer program and no numerical search. Inverting the test via binary search yields a prediction set for the attributable effect in O(log S) Binomial tail calculations; the Bonferroni proposition of Rigdon and Hudgens (2015) produces the ATE confidence set at the same computational cost. The same corner extends without further machinery to a sensitivity analysis for matched observational studies under Rosenbaum's Γ-model. A simple formula for the design sensitivity illuminates when an observational study can hope to provide evidence for an effect.
Solving Minimax Problems with Bilinear Objectives with ADMM
arXiv preprint arXiv:2604.20832
We consider minimax (saddle-point) problems of the form max_{c ∈ C} min_{β∈ S} g(c; β), where C and S are compact convex sets, and g is concave-convex. Applying the Alternating Direction Method of Multipliers (ADMM) requires evaluating a proximal operator that is, in general, as hard as the original problem. We show that when the outcome function g is bilinear, i.e. g(c; β) = c^T A β, the proximal operator reduces to a generalized projection onto the confidence region S. This reduction is exact -- it involves no approximation or linearization. The resulting ADMM algorithm alternates between (i) a generalized projection onto S and (ii) a Euclidean projection onto C. We describe the derivation, state the algorithm, and discuss convergence.
gamdist
Self-published
Over the last five years, gamdist has formed the backbone of my research agenda. While it is very much a work in progress, this paper summarizes everything I have learned about regression. I think it is most useful as a collection of references! Still to come: details on regularization and the alternating direction method of multipliers.
Robust Orbit Determination via Convex Optimization
Project Report, EE364B, Stanford University
We present a method of orbit determination robust against non-normal measurement errors. We approach the non-convex optimization problem by repeatedly linearizing the dynamics about the current estimate of the orbital parameters, then minimizing a convex cost function involving a robust penalty on the measurement residuals and a trust region penalty.
ThankBeer: A Beer Recommendation Engine
Paper for CS229 at Stanford University, Fall Quarter 2012.
We discuss a beer recommendation engine that predicts whether a user has had a given beer as well as the rating the user will assign that beer based on the beers the user has had and the assigned ratings. k-means clustering is used to group similar users for both prediction problems. This framework may be valuable to bars or breweries trying to learn the preferences of their demographic, to consumers wondering what beer to order next, or to beer judges trying to objectively assess quality despite subjective preferences.
Other Papers
Star Identification via Computer Vision Techniques
Paper for EE368 at Stanford University, Spring Quarter 2012.
We discuss a method for identifying stars in a photograph. We first filter the image using region labeling and locally adaptive thresholding. We then compare the stars in the image to a star catalog. This project turned out to be overly-ambitious for a class project, and we never really got it working.
A Discussion of Relativistic Phenomena and Construction of Spacetime Diagrams
A paper I wrote at the request of a curious friend
We discuss how the Special Theory of Relativity proceeds from the absence of an absolute definition of stationarity, as well as the observation that light travels at the same speed in all reference frames. Some interesting phenomena follow: two observers in relative motion cannot always agree on the length of an object, the time between two events, or even in what order the events occurred.
Recent & Upcoming Talks
Beyond A/B Testing: Getting More from Experiments
Nike Experimentation and Causal Inference Lunch & Learn Series
In my journey to improve the design and analysis of A/B tests, I have turned to the literature on observational causal inference. Along the way, I have learned several techniques to level up experiments. These techniques include tests of equivalence and non-inferiority, closed testing procedures, methods for non-compliance, and heterogeneous treatment effects.
