Gauge

Equivalence

Are the two sides close enough?

🔒 All your data stays on your device. Nothing is sent.

Two one-sided tests (TOST), for two groups or for pairs.

EquivalenceEquivalent at the 5% levelJump to result

Two groups, or pairs

Two groups means two separate bunches, like two classes. Paired means the same thing measured twice, like a shoe before and after it is stretched.More detail. Two-sample TOST compares mean A − mean B. Paired TOST compares the mean of the within-pair differences with the margins.
Paste the readings, or type the averages and the spreads if that is all you have.More detail. Summary mode uses the means, standard deviations, and sample sizes you type. It does not redraw a point-by-point check.
Turn this on when the two groups are spread out by different amounts. Leave it off for the ordinary test, which assumes a similar spread.More detail. Welch–Satterthwaite standard error and degrees of freedom. Off uses a pooled variance and df = n1 + n2 − 2.
The difference is A minus B.
The readings in the second group. Group A minus this group is the difference the test uses.More detail. Parsed the same way as group A.
5% uses a 90% confidence interval.
The largest difference that wouldn’t matter in practice.

Off lets you set different lower and upper limits, e.g. −0.3 to +0.5.

Tick this when “too low” and “too high” use the same distance.More detail. Sets the bounds to −|δ| and +|δ|.

Verdict

The data nerd, smirking

Close enough. The difference (-0.167) sits inside -2 to 2, the gap you said would not matter in practice.

Equivalent at the 5% level.Equivalent means the difference is small enough to count as close enough. Not equivalent means you cannot say that yet. It does not mean the groups are far apart.More detail. Pass only when both one-sided p-values are below alpha, which is the same statement as the (1 − 2α) interval lying inside both margins.
Both one-sided p-values are below 0.05, and the 90% confidence interval sits inside the margins.
The statistics

What this means: how far apart the two sides are, the range that difference would usually fall in, and how sure the close-enough call is. The exact tests are just below.

How far apart
-0.1667
Average of group A minus average of group B · 12 in A, 12 in B. Zero would mean they match.
Likely range
-0.499 to 0.166
About 90% of repeats of this study would land in this range.
How sure (p)
< 0.0001
Smaller means the close-enough call is firmer. This is the TOST p-value.

One check for each side of the gap. Both have to come out small. The figures below are the real test statistics.

Lower one-sided p
< 0.0001
t = 9.466
Upper one-sided p
< 0.0001
t = -11.187
Degrees of freedom
22.00
Pooled two-sample t: n1 + n2 − 2. Tick Welch if the spreads differ.
Standard error
0.1937

Lower test: t = (difference − lower practical difference) / SE = (-0.1667 − (-2.0000)) / 0.1937 = 9.466.

Upper test: t = (difference − upper practical difference) / SE = (-0.1667 − 2.0000) / 0.1937 = -11.187.

Practical difference -2.000 to 2.000. Alpha 0.05. Method: pooled.

Group A: Shapiro–Wilk p = 0.9976.

Theoretical normal quantile
QQ plot with Blom plotting positions. The line is the sample mean and overall standard deviation.

Group B: Shapiro–Wilk p = 0.9998.

Theoretical normal quantile
QQ plot with Blom plotting positions. The line is the sample mean and overall standard deviation.
LowerUpperEstimate
The dot is the estimated difference. The thick bar is the confidence interval. Dashed lines are the margins.

Both one-sided tests are compared with the significance level you set. Close enough is only the limit you typed.

Assumptions and references

Assumptions

  • The difference is group A minus group B, or column A minus column B for pairs.
  • The default two-group test is the pooled two-sample t, which assumes a common variance. Tick Welch if the spreads look different. Welch uses the Satterthwaite degrees of freedom.
  • Paired data use the standard deviation of the differences and n − 1 degrees of freedom.
  • Alpha of 0.05 gives a (1 − 2α) = 90% confidence interval. Both one-sided p-values have to be below alpha.
  • The margins are a product or process decision. The calculator does not choose them for you.

How this is calculated

  • Lower test: t = (difference − lower margin) / SE, p = upper tail of t.
  • Upper test: t = (difference − upper margin) / SE, p = lower tail of t.
  • TOST p-value = the larger of those two p-values.
  • Interval = difference ± t_(1−α, df) × SE. Note the critical value is t at α, not α/2.
  • Pooled SE = √[s_p² (1/n1 + 1/n2)], with s_p² weighted by n − 1 in each group.
Methods and references
  • The procedure is Schuirmann’s two one-sided tests: Schuirmann, D. J. (1987), Journal of Pharmacokinetics and Biopharmaceutics, 15, 657–680. Equivalence at level α holds only when both one-sided p-values are below α.
  • That rule is the same as the (1 − 2α) confidence interval for the mean difference lying inside the margins. At α = 0.05 the interval is a 90% interval, and the critical value is t at α, not α/2.
  • Two groups default to the pooled two-sample t. Welch uses the Satterthwaite degrees of freedom. Pairs use the standard deviation of the within-pair differences and n − 1 degrees of freedom. These are the ordinary t statistics; the equivalence step is only the comparison with the two margins.