molecalculator

Moles & Molar Mass

How to Interpret Normality Test Results

Guide 01-8136 · Updated · Step by step

This guide explains how to read the output of a statistical normality test (such as Shapiro–Wilk, Kolmogorov–Smirnov, or Anderson–Darling). You will learn what the p-value means, how to decide whether your data are normally distributed, and what to do if they are not. No prior statistical knowledge is required, but you should have the test output (test statistic and p-value) from your software.

What You’ll Need

  • Output from a normality test (e.g., Shapiro–Wilk W statistic and p-value, or Kolmogorov–Smirnov D statistic and p-value).
  • Your chosen significance level (commonly 0.05).
  • Basic understanding of hypothesis testing (null and alternative hypotheses).

Step-by-Step

  1. Identify the null and alternative hypotheses. For all normality tests, the null hypothesis (H₀) states that the data come from a normally distributed population. The alternative hypothesis (H₁) states that the data do not come from a normal distribution.
  2. Locate the p-value in the output. Most software reports the p-value directly next to the test statistic. If not, you can compare the test statistic to critical values from a table, but the p-value is more straightforward.
  3. Set your significance level (α). This is the threshold for deciding whether to reject the null hypothesis. A common choice is 0.05, meaning you accept a 5% chance of incorrectly rejecting normality when the data actually are normal.
  4. Compare the p-value to α.
    • If p-value ≥ α (e.g., p ≥ 0.05): Fail to reject H₀. You do not have enough evidence to say the data are non‑normal. Practically, you can treat the data as approximately normal.
    • If p-value < α (e.g., p < 0.05): Reject H₀. The data are significantly different from a normal distribution, so you should not assume normality.
  5. Consider the sample size. With very small samples (n < 20), normality tests have low power and may fail to detect non‑normality even when it exists. With very large samples (n > 1000), even trivial deviations from normality can produce a significant p‑value. Always interpret the p‑value in the context of your sample size.
  6. Look at a visual check as well. A histogram, Q‑Q plot, or boxplot can help you see whether the deviation is severe enough to matter. The test tells you if there is a statistically significant departure, but the plot tells you if it is practically important.
  7. Decide how to proceed. If the data are normal, you can use parametric tests (t‑tests, ANOVA, Pearson correlation). If not, consider transformations (log, square root) or use non‑parametric tests (Mann–Whitney U, Kruskal–Wallis, Spearman).

Common Mistakes

  • Misinterpreting “fail to reject” as “prove normal.” A non‑significant p‑value does not prove normality; it simply means the data are not inconsistent with normality. The test cannot confirm the null hypothesis.
  • Using α = 0.05 blindly. In some fields, a stricter α (0.01) or more lenient (0.10) may be appropriate. Choose α before running the test to avoid bias.
  • Ignoring sample size. Small samples often pass normality tests even when the population is non‑normal. Large samples often fail even when the deviation is minor. Use visual plots to judge practical significance.
  • Checking normality on the raw data when the test is for residuals. For regression or ANOVA, the normality assumption applies to the residuals, not the raw dependent variable. Always test the residuals.
  • Using the wrong test for the sample size. Shapiro–Wilk is generally recommended for small to medium samples (n < 5000). Kolmogorov–Smirnov is less powerful and often outdated; use Anderson–Darling or Shapiro–Wilk if available.

Tips

  • Always combine the test with a Q‑Q plot. If the points roughly follow the diagonal line, normality is reasonable even if the p‑value is slightly below 0.05.
  • If your sample size is very small (n < 10), the test may be unreliable. Consider using a Shapiro–Wilk test, which is designed for small samples, and interpret results cautiously.
  • For large datasets (n > 1000), rely more on the visual plot and the shape of the histogram than on the p‑value, because the test will almost always be significant.
  • When data are non‑normal, try a transformation (log, square root, Box‑Cox) and re‑test. Often the transformed data will meet the normality assumption.
  • If you are unsure, consult a statistician or use robust methods that do not require normality (e.g., bootstrapping).

Troubleshooting

  • Your p‑value is exactly 0.000. This means the p‑value is so small that the software rounds it to three decimal places. It is still less than 0.001, so you reject the null hypothesis. Report it as p < 0.001.
  • The test statistic is negative or weird. Some tests (like Shapiro–Wilk) produce a W statistic between 0 and 1; others (like Kolmogorov–Smirnov) produce a D statistic between 0 and 1. If you see a negative value, you may be looking at a different statistic (e.g., z‑score). Check the test name and software documentation.
  • You get different results from different tests. This is normal because tests have different sensitivities. Focus on the test you planned to use, and if they disagree, look at the visual plots to decide.
  • Your data have ties or are discrete. Normality tests assume continuous data. If you have many ties (e.g., Likert scale responses), the test may be inaccurate. Consider using a test that handles ties, or treat the data as ordinal and use non‑parametric methods.
  • You are testing residuals from a model. Make sure you extract the residuals correctly. In R, use resid(model); in Python, use model.resid. Testing the raw outcome variable instead of residuals is a common error.

Frequently Asked Questions

What does a p-value of 0.08 mean?

It means that if the data were truly normal, there is an 8% chance of observing a test statistic as extreme as yours. Since 0.08 is greater than 0.05, you fail to reject the null hypothesis and can treat the data as approximately normal.

Is it okay to use a t-test if my data are not perfectly normal?

Yes, if the sample size is large (n > 30) due to the Central Limit Theorem, or if the deviation is mild. For small samples, prefer non‑parametric tests if normality is seriously violated.

Which normality test should I use?

Shapiro–Wilk is the most powerful for most situations and is recommended for sample sizes up to about 5000. For larger samples, Anderson–Darling or visual inspection is often more practical.

Can I transform my data to make it normal?

Yes, common transformations include log, square root, reciprocal, or Box‑Cox. After transformation, re‑run the normality test on the transformed data. If the p‑value becomes non‑significant, you can proceed with parametric tests.

What is a Q-Q plot and how do I read it?

A Q‑Q (quantile‑quantile) plot compares your data quantiles to the quantiles of a normal distribution. If the points lie roughly along the diagonal line, the data are approximately normal. Systematic deviations (curves, S‑shapes) indicate skewness or heavy tails.

Do I need to test normality for every group separately?

Yes, if you are comparing groups (e.g., in ANOVA), you should check normality within each group, not on the combined data. The assumption applies to each group’s distribution.

Related

General steps for current mainstream versions; menus can differ by device and version. See the disclaimer or report an error.

More in this section

The whole Moles & Molar Mass section →