Skip to content

fix(stat_summary): compute mean_se from the sample variance - #1119

Merged
has2k1 merged 1 commit into
has2k1:mainfrom
dylanpulver:fix-mean-se-sample-variance
Sep 7, 2026
Merged

fix(stat_summary): compute mean_se from the sample variance#1119
has2k1 merged 1 commit into
has2k1:mainfrom
dylanpulver:fix-mean-se-sample-variance

Conversation

@dylanpulver

@dylanpulver dylanpulver commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

mean_se builds the standard error from np.var(series), whose default ddof=0 is the population variance. The standard error of the mean uses the sample variance, so the interval is too narrow by sqrt((n-1)/n) — 29% at n=2, 5% at n=10.

ggplot2's reference implementation (R/stat-summary.R):

se <- mult * sqrt(stats::var(x) / length(x))

R's stats::var is the sample variance.

The two sibling summary functions in this module already use sample estimators — mean_sdl via Series.std() and mean_cl_normal via scipy.stats.sem, both ddof=1. mean_se was the only one on ddof=0.

Measured half-width for [2,4,4,4,5,5,7,9] (n=8):

half-width
before 0.7071067812
s/sqrt(n) = scipy.stats.sem = ggplot2 0.7559289460

ratio 0.9354143467 = sqrt(7/8) exactly.

The fix keeps ggplot2's sqrt(var/n) shape. series.std()/sqrt(n) also passes the suite, but is worse: ndarray.std() defaults to ddof=0, so it would keep the bug for array input and raise on a list.

mean_se is the default fun_data for stat_summary_bin, so mean_se.png is regenerated (via tools/update_baseline_images.py, then oxipng -o 4 --strip safe).

Tests: 850 passed, 5 skipped, 1 xpassed, 0 failed. An earlier version of this description reported 6 test_geom_smooth failures; those were missing dependencies in my checkout. The added test fails on the unpatched function.

Found and drafted with Claude Opus 5 (claude-opus-5); I reviewed and verified the change.

@codecov

codecov Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.20%. Comparing base (8aa2879) to head (de31647).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1119   +/-   ##
=======================================
  Coverage   88.20%   88.20%           
=======================================
  Files         222      222           
  Lines       16253    16253           
  Branches     2080     2080           
=======================================
  Hits        14336    14336           
  Misses       1315     1315           
  Partials      602      602           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@has2k1 has2k1 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching that. For now I prefer no AI attribution in the commit message.

Comment thread tests/test_stat_summary.py Outdated
half_width = res2["ymax"].iloc[0] - res2["y"].iloc[0]
assert half_width == pytest.approx(2 * expected)


Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this test. The comment in the function and the visual test are sufficient.

@dylanpulver
dylanpulver force-pushed the fix-mean-se-sample-variance branch from acf798e to a57b090 Compare September 4, 2026 13:35
@has2k1
has2k1 force-pushed the fix-mean-se-sample-variance branch from de31647 to a57b090 Compare September 7, 2026 16:25
`mean_se` used `np.var(series)`, whose default `ddof=0` is the population
variance. The standard error of the mean is built from the sample
variance, so the interval was too narrow by a factor of sqrt((n-1)/n)
(29% at n=2, 5% at n=10).

ggplot2's reference implementation is
`se <- mult * sqrt(stats::var(x) / length(x))`, and R's `stats::var` is
the sample variance. The two sibling summary functions in this module
already use sample estimators: `mean_sdl` uses `Series.std()` and
`mean_cl_normal` uses `scipy.stats.sem`, both `ddof=1`.

`mean_se` is also the default `fun_data` for `stat_summary_bin`, so the
baseline image for the affected test is regenerated.
@has2k1
has2k1 force-pushed the fix-mean-se-sample-variance branch from a57b090 to e3696f0 Compare September 7, 2026 16:27
@has2k1
has2k1 merged commit e3696f0 into has2k1:main Sep 7, 2026
@has2k1

has2k1 commented Sep 7, 2026

Copy link
Copy Markdown
Owner

@dylanpulver, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants