Russ Zaliznyak
Russ Zaliznyak
Principal Data Scientist
AB Testing Python Statistics Math
Writing on A/B testing, Bayesian methods, and Python. Articles use simulation and code to build intuition — not just theory.
AB Testing is an Optimization Problem
Frequentist stopping rules — run until p < 0.05 — are not derived from any business objective. Reframes AB testing as a decision optimization problem and shows how Bayesian methods align stopping criteria with what actually matters.
The Winner’s Curse
Significance tells you an effect exists — not how large it is. When only extreme results cross the significance threshold, reported effect sizes are systematically inflated. In an 80%-powered experiment, 62.5% of significant results overestimate the true lift. Two fixes: run a bigger test, or shrink estimates toward a prior.
Beta vs Binomial Distribution
Both distributions arise from the same relationship — one models outcomes given a probability, the other models the probability given outcomes. Analyzes the same experiment both ways: Binomial for p-values, Beta for credible intervals. Extends to two-proportion tests and shows both are well-approximated by the Normal under large samples.
Going Bayes: Shifting Our Testing Methods To Reflect Our Priorities
How aligning business priorities with Bayesian Risk-Based Testing reduced Intuit’s experiment duration by over 60%. Covers the tradeoffs of moving away from frequentist defaults and scaling adoption across data science, product, and leadership.
Why Switch to Polars from Pandas
Benchmarking Polars vs Pandas across common DataFrame operations: read, write, construct, pivot, join, and filter. Polars wins across the board — often by 10–30x.
The Value of Big Swings
Why bold experiments deliver more value with less traffic. Compares guaranteed small wins vs. high-variance transformational bets using 30,000 simulations.
Experiment Design & Analysis with NumPy
Understanding p-values and statistical power through simulation. Builds a full AB test framework from scratch using numpy.random.
The Power of Data Simulation
Using Python’s NumPy random generator to model randomness and build intuition for rate-based metrics. A foundation for all simulation-based work.
How Intuit Uses Dash Enterprise
Highlight reel from a Plotly webinar on building Intuit’s experimentation suite in Dash Enterprise — a sample size calculator, sequential AB testing app, and experiment simulation tool — adopted by 500+ users across Intuit’s A2D org. Reduced experiment runtime by 70%+.