The t test is moderately robust to violations of normality, especially with larger sample sizes (typically n ≥ 30 per group) and when the violation is not extreme. However, for small samples, heavy tails, or strong skewness, the t test can produce misleading p-values, and alternatives like the Mann-Whitney U test or Welch’s t test are safer choices.
Formula
The Welch t statistic (used when variances are unequal) is:
t = (x̄₁ – x̄₂) / √(s₁²/n₁ + s₂²/n₂)
where:
- x̄₁, x̄₂ = sample means of group 1 and group 2
- s₁², s₂² = sample variances (standard deviation squared) of each group
- n₁, n₂ = sample sizes of each group
The degrees of freedom for Welch’s t test is approximated by the Welch–Satterthwaite equation:
df = (s₁²/n₁ + s₂²/n₂)² / [ (s₁²/n₁)²/(n₁-1) + (s₂²/n₂)²/(n₂-1) ]
This calculator computes the t statistic, the Welch degrees of freedom, and the standard error of the difference. To get a p-value, you would compare the t statistic to the t-distribution with the calculated degrees of freedom (using a table or software).
Worked Example
Using the default inputs: n₁ = 30, n₂ = 30, s₁ = 10, s₂ = 12, and mean difference = 5.
Step 1: Compute the standard error:
SE = √(10²/30 + 12²/30) = √(100/30 + 144/30) = √(3.3333 + 4.8) = √8.1333 = 2.8519
Step 2: Compute the t statistic:
t = 5 / 2.8519 = 1.753
Step 3: Compute the Welch degrees of freedom:
df = (8.1333)² / [ (100/30)²/(29) + (144/30)²/(29) ] = 66.15 / [ (3.3333²/29) + (4.8²/29) ] = 66.15 / [ 11.111/29 + 23.04/29 ] = 66.15 / [0.38314 + 0.79448] = 66.15 / 1.1776 = 56.17
So the t statistic is 1.753, the degrees of freedom are approximately 56.2, and the standard error is 2.852. With df = 56, a t of 1.753 corresponds to a two-tailed p-value of about 0.085, which is not significant at α = 0.05.
Assumptions
- The two samples are independent (not paired).
- The data are continuous (or at least interval-level).
- The populations have equal variances (for the standard Student’s t test; Welch’s t test does not require this).
- Each sample is drawn randomly from its population.
- The sampling distribution of the mean is approximately normal (which holds if the population is normal or if the sample size is large enough per the Central Limit Theorem).
When Does Non-Normality Actually Matter?
The t test is based on the assumption that the sample means follow a normal distribution. Thanks to the Central Limit Theorem, as sample sizes grow, the distribution of the sample mean becomes more normal even if the underlying population is not. This is why the t test is often called “robust” to non-normality: for moderate to large samples, the test statistic still approximately follows a t distribution.
However, robustness is not guaranteed. The t test is sensitive to:
- Small sample sizes (n < 30): The Central Limit Theorem hasn’t kicked in fully, so the mean’s distribution may still be skewed.
- Heavy tails (outliers): Extreme values inflate the standard deviation, reducing the t statistic and power, and can also inflate the Type I error rate.
- Strong skewness: One-tailed tests are particularly affected; the actual alpha can be much higher than 0.05.
- Unequal variances combined with unequal sample sizes: This can distort both Type I and Type II errors, but Welch’s t test handles this well.
In practice, a common rule of thumb is that if each group has at least 30 observations, the t test is usually fine even with non-normal data, provided there are no extreme outliers. For smaller samples, you should check normality (e.g., Shapiro-Wilk test, Q-Q plot) and consider alternatives.
How to Check Normality and What to Do If It Fails
Before deciding whether to trust a t test, you can assess normality visually and statistically:
- Histogram or Q-Q plot: Look for symmetry and whether points fall roughly on a straight line in a Q-Q plot.
- Shapiro-Wilk test: A significant p-value (< 0.05) suggests the data deviate from normality, but this test is sensitive to large samples, so combine with visual checks.
- Skewness and kurtosis: Values between -2 and +2 are often considered acceptable, but this is a rough guideline.
If normality is clearly violated, you have several options:
- Use a nonparametric test: The Mann-Whitney U test (for independent groups) or Wilcoxon signed-rank test (for paired data) do not assume normality. They test for differences in location (median) rather than mean.
- Transform the data: Log, square root, or Box-Cox transformations can sometimes make skewed data more normal. Then run the t test on the transformed data.
- Use a robust method: The Welch t test is more robust to variance inequality, but not to heavy tails. Trimmed means or bootstrap methods can be used, but they are more complex.
- Increase sample size: If possible, collecting more data often resolves normality issues due to the Central Limit Theorem.
Common Misconceptions and Practical Tips
Misconception 1: “The t test requires both populations to be normal.” Actually, it only requires that the sampling distribution of the mean be normal. For large samples, this holds even if the population is not normal.
Misconception 2: “If the data are not normal, the t test is completely invalid.” Not true. The t test is robust in many cases, especially with balanced designs and equal variances. The problem arises with small samples and severe violations.
Misconception 3: “The Mann-Whitney U test is always better when data are non-normal.” The Mann-Whitney test has slightly less power than the t test when data are normal, and it tests a different parameter (stochastic dominance) than the mean. It’s a good alternative, but not automatically superior.
Practical tips:
- Always report the sample size and show a plot of the data (histogram or boxplot) so readers can judge normality themselves.
- If you run a t test despite non-normality, consider also running a nonparametric test as a sensitivity analysis. If both give the same conclusion, your result is more robust.
- For very small samples (n < 10), the t test is quite sensitive to non-normality; use a nonparametric test or a permutation test.
- Outliers are more influential than skewness. If you have outliers, consider winsorizing or using a robust estimator.
Alternatives to the t Test for Non-Normal Data
If you decide the t test is not appropriate, here are the most common alternatives:
- Mann-Whitney U test (independent samples): Nonparametric, compares whether one group tends to have larger values than the other. Works with ordinal or continuous data.
- Wilcoxon signed-rank test (paired samples): Nonparametric counterpart to the paired t test.
- Permutation test: Resamples the data to build a null distribution. Very flexible and makes no normality assumption, but requires computational power.
- Bootstrap confidence interval: Resamples with replacement to estimate the sampling distribution of the difference in means. Can be used to construct a confidence interval without normality.
- Welch’s t test: Even though it’s still a t test, it is more robust to unequal variances, which often accompany non-normality.
For paired designs, the Wilcoxon signed-rank test is the most common alternative. For independent designs, the Mann-Whitney U test is the go-to. If you have more than two groups, use the Kruskal-Wallis test instead of one-way ANOVA.
Frequently Asked Questions
Is the t test robust to non-normality if the sample size is large?
Yes, generally. With n ≥ 30 per group, the Central Limit Theorem ensures the sampling distribution of the mean is approximately normal, so the t test performs well even if the population is not normal, provided there are no extreme outliers or severe skewness.
What sample size is needed for the t test to be robust to non-normality?
There is no universal cutoff, but n = 30 per group is a common rule of thumb. For n < 15, the t test is quite sensitive to non-normality. For n between 15 and 30, it depends on the degree of skewness and outliers.
Should I use the Mann-Whitney U test instead of the t test for non-normal data?
It’s a reasonable choice. The Mann-Whitney U test does not assume normality and is almost as efficient as the t test when data are normal. However, it tests whether one group tends to have larger values, not specifically the mean. If you are interested in the mean difference, a bootstrap or permutation test might be more aligned.
Does the t test assume equal variances?
The standard Student’s t test assumes equal variances, but Welch’s t test does not. Welch’s test is generally recommended because it performs well even when variances are equal, and it protects against unequal variances. Many statistical software packages default to Welch’s test.
Can I transform my data to make it normal, then use a t test?
Yes, transformations like log, square root, or Box-Cox can often reduce skewness. After transformation, the t test is valid, but the interpretation changes: you are comparing means of the transformed values, not the original units. Be sure to mention the transformation in your report.
What if my data has outliers? Is the t test still robust?
Outliers can seriously affect the t test because they inflate the standard deviation and pull the mean. The t test is not robust to outliers. Consider removing genuine errors, using a robust estimator like trimmed mean, or using a nonparametric test.