Observational Studies on Adventures in Why

Randomization Inference for Matched Pairs with Binary Outcomes

bob@ConvexAnalytics.com (Bob Wilson) — Thu, 03 Sep 2026 00:00:00 +0000

Sensitivity Analysis for Matched Sets with One Treated Unit

bob@ConvexAnalytics.com (Bob Wilson) — Fri, 29 Dec 2023 00:00:00 +0000

Adjusting for observed factors does not elevate an observational study to the reliability of an experiment. P-values are not appropriate measures of the strength of evidence in an observational study. Instead, sensitivity analysis allows us to identify the magnitude of hidden biases that would be necessary to invalidate study conclusions. This leads to a strength-of-evidence metric appropriate for an observational study.

Sensitivity Analysis for Matched Pairs

bob@ConvexAnalytics.com (Bob Wilson) — Fri, 03 Nov 2023 00:00:00 +0000

Observational studies involve more uncertainty than randomized experiments. Sensitivity analysis offers an approach to quantifying this uncertainty.

The Alternative to Causal Inference is Worse Causal Inference

bob@ConvexAnalytics.com (Bob Wilson) — Sun, 10 Oct 2021 00:00:00 +0000

Some of the most important questions data scientists investigate are causal questions. They’re also some of the hardest to answer! A well-designed A/B test often provides the cleanest answer, but when a test is infeasible, there are plenty of other causal inference techniques that may be useful. While not perfect, these techniques are much better than the alternative: ad hoc methods with no logical foundation.

Naive Comparisons Under Endogeneity

bob@ConvexAnalytics.com (Bob Wilson) — Sun, 28 Feb 2021 00:00:00 +0000

Recently I have been reading Causal Inference: The Mixtape by Scott Cunningham. One thing I think Cunningham explains very well is the role of endogeneity in confounding even simple comparisons. I don’t have a background in economics, so I had never really grokked the concepts of endogenous and exogenous factors, especially as it related to causal inference. In this post, I’m going to discuss a few examples that highlight why it’s such an important distinction.