A post hoc power analysis calculator computes the statistical power of a study after the results are known, using the observed effect size, sample size, and significance level. You enter your sample size, the effect size you observed (or the means and standard deviations), and the alpha level; the calculator returns the power, which is the probability that your test would have detected an effect of that size if one truly existed. This is often used to interpret non-significant results, though its value is debated—read on to understand why.
Formula
For a two-sample t-test (equal group sizes), power is calculated using the non-central t-distribution. The formula is:
Power = 1 – Ft(tcrit | df, ncp)
where:
- Ft is the cumulative distribution function of the non-central t-distribution.
- tcrit is the critical t-value for the given alpha and degrees of freedom (df = n1 + n2 – 2). For a two-tailed test, tcrit = t1-α/2, df.
- ncp is the non-centrality parameter, calculated as d × √(n1n2 / (n1 + n2)), where d is Cohen’s d (effect size) and n1, n2 are group sizes. For equal groups, ncp = d × √(n/2).
- n is the sample size per group.
- d is the observed effect size (Cohen’s d).
- α (alpha) is the significance level (typically 0.05).
The calculator above uses the non-central t-distribution to compute power directly. The result is a probability between 0 and 1, where 0.8 or higher is conventionally considered adequate power.
Worked Example
Suppose you ran a study with 30 participants per group and observed a Cohen’s d of 0.5. You set alpha at 0.05 (two-tailed). What is the post hoc power?
- Degrees of freedom: df = 30 + 30 – 2 = 58.
- Critical t-value for α = 0.05, two-tailed, df = 58: tcrit ≈ 2.0017.
- Non-centrality parameter: ncp = 0.5 × √(30/2) = 0.5 × √15 ≈ 0.5 × 3.87298 = 1.93649.
- Using the non-central t-distribution with df = 58 and ncp = 1.93649, find the probability that a t-value exceeds 2.0017. This is the power.
- Power ≈ 0.478 (47.8%).
The calculator gives the same result: power ≈ 0.478. This means that if the true effect size were 0.5, your study would have only about a 48% chance of detecting it at α = 0.05—well below the usual 80% threshold.
Assumptions
- The test is a two-sample t-test with equal group sizes.
- The data are normally distributed within each group.
- The two groups have equal variances.
- The effect size is measured as Cohen’s d (difference in means divided by pooled standard deviation).
- The significance level (alpha) is fixed before the analysis.
What Is Post Hoc Power?
Post hoc power (also called observed power or retrospective power) is the statistical power of a test calculated after the data have been collected and analyzed. It uses the observed effect size (from your sample) rather than a hypothesized effect size. The idea is to answer: “Given the effect I saw, how likely was my study to detect it?”
This is different from a priori power analysis, which is done before a study to determine the sample size needed to detect a meaningful effect. A priori power uses an expected effect size based on theory or prior research. Post hoc power uses the effect size you actually found.
The calculator above is for a two-sample t-test, the most common scenario. For other tests (ANOVA, chi-square, regression), the calculation differs, but the principle is the same: power depends on effect size, sample size, alpha, and the test’s sensitivity.
Why Use a Post Hoc Power Calculator?
Researchers typically use post hoc power in two situations:
- Interpreting non-significant results: When a study fails to reject the null hypothesis, you might wonder if the study was underpowered. Post hoc power can tell you how likely it was to detect the observed effect. Low power suggests the study might have missed a real effect.
- Reviewing published studies: Some reviewers and journals ask for post hoc power to assess whether a non-significant result is convincing evidence of no effect, or simply a consequence of low power.
However, many statisticians advise against post hoc power analysis. The main criticism is that it is a one-to-one function of the p-value: if p is low, power is high; if p is high, power is low. So it adds no new information beyond the p-value. For example, a non-significant result with low post hoc power doesn’t tell you whether the effect is truly absent or just small—it’s the same information as the p-value.
Instead, it is often more useful to report confidence intervals around the effect size. A confidence interval shows the range of plausible effect sizes, which directly addresses whether the study was precise enough to detect a meaningful effect.
How to Read the Results
When you run the calculator, you get two numbers:
- Power (1 – beta): The probability of rejecting the null hypothesis if the true effect size equals the observed effect size. A value of 0.8 or higher is typically considered adequate.
- Type II error rate (beta): The probability of failing to reject the null hypothesis when there is a true effect. This is simply 1 – power.
For example, a power of 0.478 means that if the true effect size is 0.5, you would only detect it 47.8% of the time. This is often described as “underpowered.”
Keep in mind that post hoc power is a retrospective estimate. It is not a guarantee of what happened in your study; it’s a hypothetical probability based on the effect you observed.
Common Mistakes and Pitfalls
- Using observed power to claim a result is meaningful: High post hoc power does not strengthen a significant result. If p < 0.05, power is usually high, but that doesn't make the effect more real.
- Ignoring the effect size: Post hoc power is meaningless without the effect size. A tiny effect can be “detected” with a huge sample, but that doesn’t mean it’s practically important.
- Confusing post hoc with a priori: Post hoc power does not tell you what sample size you should have used. For that, you need a priori power analysis with a pre-specified effect size.
- Using the wrong test: The calculator above assumes a two-sample t-test. If your study used a different design (paired t-test, ANOVA, correlation), the power calculation differs.
- Rounding errors: Effect sizes and sample sizes should be entered accurately. Small changes can noticeably alter power.
Alternatives to Post Hoc Power
Instead of post hoc power, statisticians often recommend:
- Confidence intervals: Report a 95% confidence interval around the effect size. This shows the range of plausible effects and whether it includes zero or a meaningful threshold.
- A priori power analysis: Plan your sample size before the study based on a meaningful effect size. This is the gold standard for designing studies.
- Bayesian approaches: Calculate the posterior probability of the null hypothesis, which gives a more intuitive measure of evidence.
If you are writing a paper and a reviewer asks for post hoc power, you can provide it, but also include confidence intervals to give a fuller picture.
Frequently Asked Questions
Is post hoc power analysis ever appropriate?
Some researchers use it to explain a non-significant result, but many statisticians consider it redundant because it is directly determined by the p-value. A more useful approach is to report the effect size and its confidence interval. If you must report post hoc power, do so alongside these, not instead of them.
What is a good power level?
Conventionally, 0.80 (80%) is considered adequate power. This means you have an 80% chance of detecting a true effect of the specified size. However, the appropriate level depends on the field and the consequences of missing an effect. For high-stakes research, you might aim for 0.90 or higher.
Can I use this calculator for a one-sample t-test?
No, this calculator is for a two-sample t-test with equal group sizes. For a one-sample t-test, the non-centrality parameter is different (ncp = d × √n, where n is the total sample size). You would need a different calculator or formula.
What if my groups have unequal sizes?
The formula changes slightly. For unequal group sizes n1 and n2, the non-centrality parameter is ncp = d × √(n1 × n2 / (n1 + n2)). You can use a more general power analysis tool that allows unequal n.
Does post hoc power tell me if my study was underpowered?
Not directly. It tells you the probability of detecting the observed effect, but it doesn’t tell you if that effect is real or important. A study can be underpowered for a small effect but well-powered for a large one. The key is to consider the smallest effect you care about, not the one you happened to observe.
Why does my power come out as 0.9999?
That’s because the observed effect size and sample size are large enough that the test would almost always detect the effect. This often happens with significant results. It’s not a problem—it just means the study was very sensitive to that effect size.
Related
Formula used
in n | Sample size (per group) | - | 30 | 2 in d | Observed effect size (Cohen's d) | - | 0.5 | 0.001 in a | Significance level (alpha) | - | 0.05 | 0.001 | 1 out p | Power (1 - beta) | - | 1 - nctcdf(nctinv(1 - a, n - 1), n - 1, d * sqrt(n / 2)) | 4 out beta | Type II error rate (beta) | - | 1 - p | 4
Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work
Method: the standard formula for this calculation with the constants and conversion factors written on the page, rounded as each result states. See method and constants, the disclaimer or report an error.
More in this section
- QPP Calculator Canada: How It Works & How to Use It
- PPP Calculator World Bank: How It Works & How to Use It2,000.00
- PPP Calculator: India vs USA Cost Comparison1,000.00 USD
- PPP Calculator USD to INR: How It Works2,335 INR
- PPP Calculator India vs UK: Real Cost Comparison103.63 GBP
- PPP Calculator India vs Dubai: Real Cost Comparison8,954.58 AED