AB Testing
Articles on experimentation design, statistical analysis, and simulation-driven decision making.
AB Testing is an Optimization Problem
June 2026 · AB Testing · Bayesian · Optimization · Simulation · Experimentation
Frequentist stopping rules are not derived from any business objective. Reframes AB testing as a decision optimization problem and shows how Bayesian methods align stopping criteria with what actually matters.
The Winner’s Curse
Jan 2026 · Bias · Bayesian
Statistical significance is not an endorsement of the treatment effect. Quantifies how often significant results overestimate true lift.
Beta vs Binomial Distribution
Dec 2025 · Distributions · Bayesian
When to use each distribution in experimentation. Covers p-values, credible intervals, and the Normal approximation — all built with simulations.
The Value of Big Swings
Aug 2025 · Strategy · Simulation
Why bold experiments deliver more value with less traffic. Compares guaranteed small wins vs. high-variance transformational bets using 30,000 simulations.
Experiment Design & Analysis with NumPy
Feb 2025 · Simulation · Power
Understanding p-values and statistical power through simulation. Builds a full AB test framework from scratch using numpy.random.
The Power of Data Simulation
Nov 2024 · Simulation · Intuition
Using Python’s NumPy random generator to model randomness and build intuition for rate-based metrics. A foundation for all simulation-based work.