molecalculator

Moles & Molar Mass

Equivalence Testing in R: TOST & More

Calculator No. 01-12392 · Updated · Formula and assumptions shown below

The calculator on this page is switched off while it is checked. The formula is below, and the worked example shows how to do the sum by hand.

For study and lab planning. The result follows from the numbers you enter and the formula shown. Check it against your method before you weigh, mix or dilute anything, follow the safety data sheet and your supervisor, and do not use it for clinical, dosing or medication work.

Equivalence testing in R is a statistical method used to determine whether two treatments or measurements are practically equivalent within a pre-specified margin, rather than merely testing if they are different. You enter your two samples and an equivalence bound, and R returns a test statistic and p-value that tell you whether the observed difference is small enough to conclude equivalence. The most common approach is the Two One-Sided Tests (TOST) procedure, implemented in packages like TOSTER and equivalence.

Formula

The TOST procedure tests two one-sided hypotheses. The null hypothesis for the first test is that the mean difference is less than or equal to the lower bound (−Δ), and for the second test that it is greater than or equal to the upper bound (+Δ). The test statistics are:

t1 = ( (m1 – m2) + Δ ) / SE

t2 = ( (m1 – m2) – Δ ) / SE

where SE = √(s1²/n1 + s2²/n2), m1, m2 are sample means, s1, s2 are sample standard deviations, n1, n2 are sample sizes, and Δ is the equivalence bound (the largest difference you consider practically unimportant). The p-value for each one-sided test is obtained from the t-distribution with Welch degrees of freedom (approximate). The TOST p-value is the larger of the two one-sided p-values. If this p-value is less than your significance level (e.g., 0.05), you conclude equivalence.

Worked Example

Suppose you have two groups: group 1 has mean 100, SD 10, n=30; group 2 has mean 102, SD 10, n=30. You set an equivalence bound of ±5 units. Using the default values in the calculator:

SE = √(10²/30 + 10²/30) = √(100/30 + 100/30) = √(6.6667) ≈ 2.5820

t1 = (100 – 102 + 5) / 2.5820 = 3 / 2.5820 ≈ 1.1619

t2 = (100 – 102 – 5) / 2.5820 = –7 / 2.5820 ≈ –2.7111

Welch df = ( (10²/30 + 10²/30)² ) / ( (10²/30)²/(30–1) + (10²/30)²/(30–1) ) = (6.6667²) / (2 × (3.3333²)/29) ≈ 44.444 / (2 × 11.111/29) = 44.444 / 0.7663 ≈ 58.0

p1 = P(T > 1.1619) with df=58 ≈ 0.1250

p2 = P(T < –2.7111) with df=58 ≈ 0.0044

TOST p = max(0.1250, 0.0044) = 0.1250

Since 0.1250 > 0.05, you cannot conclude equivalence. The observed difference of 2 is within ±5, but the variability and sample size do not provide enough evidence to rule out a difference larger than 5.

Assumptions

  • Independent samples (for two-sample tests).
  • Approximate normality of the sampling distribution of the mean (Central Limit Theorem helps with moderate sample sizes).
  • The equivalence bound Δ is chosen before looking at the data, based on practical or clinical relevance.
  • Welch’s t-test does not assume equal variances; if you assume equal variances, you can use a pooled SE and df = n1+n2–2.

Background and Why Not Just Use a t-test?

Traditional null hypothesis significance testing (NHST) asks “is there a difference?” and a non-significant p-value does not prove that there is no difference – it only says the data are not strong enough to detect one. Equivalence testing flips the question: you define a range of differences that you consider practically irrelevant, and then test whether the observed difference is unlikely to fall outside that range. This is essential in generic drug bioequivalence studies, method validation, quality control, and any field where you need to show that a new method or product performs the same as an existing one within acceptable tolerances.

How to Choose the Equivalence Bound

The bound Δ is the most important and subjective part of equivalence testing. It must be set before collecting data, based on domain knowledge. For example, in bioequivalence, regulatory agencies often use ±20% of the reference mean (or a log-scale margin of 0.8 to 1.25). In analytical chemistry, you might set Δ as 2% of the measured value. In psychology, you might use a fraction of a standard deviation (e.g., 0.2 SD as a small effect). The bound should be small enough that any difference within it is scientifically meaningless, but not so small that it becomes practically impossible to demonstrate equivalence without a huge sample size.

Common Problems and Misconceptions

  • Confusing “not significant” with “equivalent”: A non-significant t-test does not imply equivalence; you must run a TOST or similar procedure.
  • Choosing the bound after seeing the data: This inflates the type I error rate. Always pre-specify Δ.
  • Using a one-sided test incorrectly: TOST is not a single one-sided test; it requires two tests and taking the maximum p-value.
  • Ignoring the confidence interval approach: Equivalence can also be concluded if the (1–2α)% confidence interval for the difference lies entirely within (–Δ, +Δ). This is mathematically equivalent to TOST.
  • Forgetting to check assumptions: If sample sizes are very small or data are highly skewed, consider nonparametric equivalence tests (e.g., Wilcoxon-based TOST) or bootstrap methods.

Comparisons with Other Approaches

Besides TOST, there are other ways to assess equivalence: the confidence interval (CI) method, Bayesian equivalence testing, and the “two one-sided confidence intervals” approach. The CI method is often preferred because it gives a visual sense of precision. Bayesian methods allow incorporating prior information and provide a posterior probability of equivalence. In R, the TOSTER package offers TOST, equivalence bounds for t-tests, ANOVA, correlations, and regression coefficients. The equivalence package provides basic TOST for two-sample and paired designs. For nonparametric data, you can use the boot package to construct bootstrap confidence intervals and check if they fall within the bounds.

Costs and Sample Size Considerations

Equivalence tests often require larger sample sizes than traditional difference tests, because you need enough power to detect a difference that is smaller than the bound. The required sample size depends on the true difference (assumed to be 0 for planning), the standard deviation, the bound, and the desired power. In R, you can use the power_t_TOST function from TOSTER to calculate power or sample size. For example, with Δ=5, SD=10, α=0.05, power=0.80, you might need around 63 per group to detect a true difference of 0. This is a cost in time and resources, but it is necessary to make a credible claim of equivalence.

Safety and Scope

Equivalence testing is a statistical tool for research and quality control. It is not a substitute for clinical judgment or regulatory approval. When used in a laboratory setting, always follow your lab method, the safety data sheet (SDS) and your supervisor. The interpretation of equivalence should always be tied to the practical context and the pre-specified bound.

Frequently Asked Questions

How do I run TOST in R?

Install the TOSTER package and use tsum_TOST(m1=100, sd1=10, n1=30, m2=102, sd2=10, n2=30, eqb=5) for summary statistics, or t_TOST(x, y, eqb=5) for raw data. The output gives the TOST p-value and confidence interval.

What if my data are not normally distributed?

For moderate sample sizes, the t-test is robust to mild non-normality. For severe skewness or small samples, consider a nonparametric equivalence test, such as the Wilcoxon rank-based TOST in the TOSTER package, or use bootstrap confidence intervals.

Can I use equivalence testing for paired data?

Yes. Use a paired TOST, which is equivalent to a one-sample TOST on the differences. In TOSTER, use paired_TOST or provide the difference scores to tsum_TOST with one sample.

What is the difference between equivalence and non-inferiority?

Non-inferiority tests whether a new treatment is not worse than the reference by more than a margin, using a one-sided test. Equivalence tests whether the difference is within a two-sided margin. Non-inferiority is a one-sided version of equivalence.

How do I choose the equivalence bound for my field?

Look for published guidelines in your discipline (e.g., FDA bioequivalence limits, ISO standards for measurement methods). If none exist, justify the bound based on the smallest effect that would be practically important, and document it in your analysis plan.

Can I conclude equivalence if the confidence interval is inside the bounds?

Yes. If a (1–2α)% confidence interval (e.g., 90% CI for α=0.05) lies entirely within (–Δ, +Δ), you can reject the null hypothesis of non-equivalence at the α level. This is exactly equivalent to the TOST procedure.

Related

Formula used
in n1 | Sample size (group 1) | - | 30 | 2
in n2 | Sample size (group 2) | - | 30 | 2
in m1 | Mean (group 1) | - | 100 | 0
in m2 | Mean (group 2) | - | 102 | 0
in sd1 | SD (group 1) | - | 10 | 0.001
in sd2 | SD (group 2) | - | 10 | 0.001
in bound | Equivalence bound (absolute) | - | 5 | 0.001
out se | Standard error of difference | - | sqrt(sd1^2/n1 + sd2^2/n2) | 4
out t1 | Lower t-statistic | - | (m1 - m2 + bound) / se | 3
out t2 | Upper t-statistic | - | (m1 - m2 - bound) / se | 3
out df | Welch degrees of freedom | - | ( (sd1^2/n1 + sd2^2/n2)^2 ) / ( (sd1^2/n1)^2/(n1-1) + (sd2^2/n2)^2/(n2-1) ) | 2
out p1 | p-value (lower) | - | pt(t1, df, lower.tail = FALSE) | 4
out p2 | p-value (upper) | - | pt(t2, df, lower.tail = TRUE) | 4
out tost_p | TOST p-value (max of p1, p2) | - | max(p1, p2) | 4

Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work

Method: the standard formula for this calculation with the constants and conversion factors written on the page, rounded as each result states. See method and constants, the disclaimer or report an error.

More in this section

The whole Moles & Molar Mass section →