Common Statistics Mistakes in Academic Papers

Most statistics mistakes in academic papers aren't arithmetic errors. The software does the arithmetic. What goes wrong is the reasoning around it. A test chosen for the wrong data. A p-value read as something it isn't. A finding called important when the numbers only say it's detectable.

These errors cluster by stage. Some are locked in before data collection starts and can't be fixed afterward. Others happen during analysis, and the rest appear only in the write-up. This guide works through them in that order, because knowing when a mistake becomes permanent tells you how urgently to catch it.

The Mistakes at a Glance

Design-Stage Mistakes

These are the expensive ones. By the time you're analyzing data, they're already fixed in place, and the only honest response is to name them as limitations.

Collecting too small a sample

An underpowered study can run a flawless analysis and still miss a real effect. The result is a non-significant finding that proves nothing, because the study was never capable of detecting what it was looking for. Reviewers in most quantitative fields check this early.

The fix is a power analysis before recruitment, which is covered in the guide to calculating sample size. If the data is already collected, say plainly that the study was powered to detect effects of a certain size and no smaller.

Assuming an optimistic effect size

This is usually what causes the mistake above. Assuming a large effect keeps the required sample manageable, which is exactly why it's tempting. When the true effect turns out to be medium or small, the study can't detect it.

Base the estimate on prior published research or pilot data. Where neither exists, use the smallest effect that would be theoretically interesting rather than the largest you can defend. See the guide to effect sizes for where those estimates come from.

Ignoring nested or clustered data

When participants sit inside groups, such as students within classrooms or patients within clinics, their responses aren't independent. Treating them as independent inflates your effective sample size and your significance along with it. A multilevel model handles this, but only if the design accounted for it.

Analysis-Stage Mistakes

These are recoverable. Each one means rerunning something, not recollecting anything.

Choosing the wrong test for the data

The test has to match the measurement level and the design. Consider a rank-based test on nominal categories, or an ANOVA where the outcome is binary. Each produces a number that means nothing. The guides to t-tests, ANOVA, and non-parametric tests cover which applies when.

Never checking assumptions

Parametric tests assume things about your data: normality, equal variances, independence, linearity. Software runs the test whether or not those hold. Checking them is your job, and reporting that you checked is increasingly expected.

When an assumption fails, you have options. Transform the variable, use a robust alternative, or switch to a rank-based test. What you can't do is proceed silently and hope no one asks.

Running many comparisons without correction

Every test carries a false positive risk. Run twenty at an alpha of .05 and you'd expect one significant result from noise alone. Reporting that one as a finding is how spurious results enter a literature.

Apply a correction such as Bonferroni or Tukey for planned multiple comparisons, and name it in the write-up. If you explored the data broadly and then reported what surfaced, say so and label those analyses exploratory. Presenting an exploratory finding as a hypothesis you held all along is a serious reporting problem, not a stylistic one.

Running a power analysis after the fact

Post-hoc power calculated from your observed effect size tells you nothing new. The calculation is circular: a large observed effect computes as high power, a small one as low power, regardless of the truth. Reviewers who know this will ask why it's there.

If you want to convey precision after the fact, report the confidence interval around your effect. A wide interval shows the imprecision that post-hoc power was trying to describe, and it does so honestly.

Interpretation Mistakes

These are the errors that survive peer review most often, because the numbers are correct and only the sentences around them are wrong.

Misreading what a p-value says

A p-value is the probability of data at least this extreme if the null hypothesis were true. It is not the probability that the null hypothesis is true, and it is not the probability that your finding is a fluke. Those readings are different claims, and both are common in published work.

The practical consequence is overstatement. A p of .04 becomes "there's only a 4 percent chance we're wrong," which the number never supported. See the guide to hypothesis testing for what the test actually asks.

Treating significance as importance

Significance depends heavily on sample size. With enough participants, a difference too small to matter clears any threshold. A large study can report a two-point difference on a hundred-point scale at p < .001. The result is real in the narrow statistical sense and irrelevant in every other one.

Report the effect size alongside the test, then make the case for practical importance explicitly, with reference to what the outcome means in your field. The number sets up that argument. It doesn't settle it.

Reading non-significance as proof of no effect

Failing to reject the null isn't the same as demonstrating the null. A non-significant result means your study didn't detect an effect. That could be because no effect exists, or because you lacked the power to find it. Writing "there was no difference between groups" claims more than the test supports. "No significant difference was detected" is the accurate form. The guide to Type I and Type II errors covers both failure modes.

Sliding from correlation to causation

Everyone knows the rule and it still happens, usually through verb choice rather than an explicit claim. "Increased study time improved exam scores" asserts causation from correlational data. "Study time was positively associated with exam scores" doesn't.

Watch for causal verbs: improved, caused, led to, resulted in, drove. In a correlational design, replace them with associational language. The same caution applies to regression, where a predictor's coefficient describes association, not effect, unless your design supports more. See the guides to what the correlation coefficient measures and multiple regression.

Reporting only what supported the hypothesis

Leaving out null or unexpected results is a form of bias, and peer reviewers are trained to look for it. Unexpected findings are part of the record and often the most interesting thing in a paper. The broader patterns here are covered in the research bias guide.

Reporting Mistakes

These are the easiest to fix and the most visible to a reviewer, since they're apparent on a first read.

• Vague results statements. "Results were positive" or "the intervention worked" aren't findings. Give the exact values.
• Missing degrees of freedom. A test statistic without df can't be evaluated or replicated.
• Reporting p = .000. Software output, not a real value. A p-value is never exactly zero.
• No effect size. Most journals in psychology, education, and health now treat significance without magnitude as incomplete reporting.
• Inconsistent rounding. Two decimals in one sentence and three in the next, with no reason, reads as inattention.
• Interpreting inside the results section. Report the numbers there. What they mean belongs in the discussion.
• Mean and standard deviation for skewed data. Use the median and interquartile range when the distribution is skewed or the data is ordinal. See standard deviation and variance.
• Tables nothing refers to. Every table needs a sentence pointing to it, and shouldn't be duplicated in prose.

The notation conventions behind most of these are covered in the guide to reporting statistical results in APA format. The structure of the section itself is covered in the guide to writing a results section.

A Pre-Submission Checklist

Work through this before the manuscript leaves your desk.

1. Confirm the test matches the data. Check measurement level, design, and the number of groups against the test you ran.
2. Confirm assumptions were checked. State in the methods that you checked them and what you did when one failed.
3. Confirm every test has an effect size. Match the measure to the test rather than defaulting to Cohen's d.
4. Confirm every statistic has its degrees of freedom and an exact p-value. Replace any p = .000 with p < .001.
5. Search your results section for causal verbs. Replace them with associational language unless the design supports causation.
6. Search for vague claims. Any sentence describing a result without a number needs one.
7. Confirm null findings are reported. Include what didn't work and what surprised you.
8. Confirm interpretation stayed in the discussion. If a results sentence says "this suggests," move it.

Where to Go Next

Each mistake here has a fuller treatment elsewhere in the cluster. For the planning decisions, see calculating sample size and effect sizes. For the reasoning, see hypothesis testing and Type I and Type II errors. For the write-up, see reporting statistical results in APA format.

A second reader catches what a fourth self-edit can't. Editor World's academic editing, dissertation editing, and journal article editing services include review of statistical reporting. You choose your own editor by field, so the person reading your results knows your discipline's conventions.

Frequently Asked Questions

Page last reviewed: September 2026. Content reviewed by Editor World editorial staff. Editor World, founded in 2010 by Patti Fisher, PhD, graduate of The Ohio State University, provides professional editing and proofreading services for academic researchers, doctoral candidates, faculty, business professionals, and authors worldwide. 100% human editing, no AI at any stage. BBB A+ accredited since 2010 with 5.0/5 Google Reviews and 5.0/5 Facebook Reviews. 16 years in business with 140 million+ words edited for over 8,000 clients in 65+ countries. Multiple Gold and Bronze Stevie Award winner. Native English editors from the United States, the United Kingdom, and Canada. Less than 5% of applicants are accepted to the editor panel. Recommended by the Boston University Economics Department, University of San Diego, University of Michigan, UCLA, University of Missouri, and more.