Publication Bias in Meta-Analysis: How to Detect and Address It

Publication bias arises because studies with statistically significant or favorable results are more likely to get published, and published faster, than studies with null or unfavorable findings. If your meta-analysis pools only what made it into the published literature, your pooled estimate can be systematically inflated relative to the true underlying effect, sometimes substantially.

Why this matters more than it might seem

A meta-analysis is only as trustworthy as the body of literature feeding into it. If an entire tier of null-result studies exists but never got published, was never indexed, or sits in a file drawer, no amount of careful screening or statistical modeling within your review can recover that missing evidence. Publication bias is a threat to validity that operates upstream of your own methodology, which is exactly why reviewers expect to see it addressed explicitly rather than assumed away.

The funnel plot as a first check

A funnel plot puts each study's effect estimate on the horizontal axis and a measure of its precision, commonly standard error, on the vertical axis, with precision increasing as you move up the plot. In the absence of publication bias, the plot should look roughly like an inverted funnel: precise, large studies cluster near the top around the true effect, while smaller, less precise studies scatter more widely near the bottom, symmetrically on both sides of the pooled estimate.

Publication bias distorts this shape asymmetrically. If small studies with null or unfavorable results are missing from the literature, you see a gap in the bottom corner of the funnel on the unfavorable side -- small studies are present where they show a positive effect, but conspicuously absent where they would show none. This asymmetry is the visual signature publication bias leaves behind.

Why visual inspection alone is not enough

Funnel plot asymmetry can also arise from genuine heterogeneity, from small-study effects unrelated to publication bias (such as smaller studies genuinely having less rigorous methodology and larger measured effects for reasons other than selective publication), or simply from chance when the number of included studies is small. Visual inspection is a reasonable first pass, but it should not be your only evidence, and reviewers increasingly expect a formal statistical test alongside it.

Egger's test and its limitations

Egger's regression test formally quantifies funnel plot asymmetry by regressing standardized effect estimates against their precision, and testing whether the intercept differs significantly from zero. A significant result suggests asymmetry beyond what chance would produce. Egger's test has real limitations, however: it performs poorly with fewer than ten included studies, and it cannot distinguish publication bias from other causes of asymmetry, so a significant result should be interpreted as evidence of asymmetry requiring explanation, not proof of publication bias specifically.

Trim-and-fill as a sensitivity check

The trim-and-fill method estimates how many studies may be "missing" from one side of a funnel plot to restore symmetry, imputes those hypothetical missing studies, and recalculates the pooled estimate including them. This is not a way to recover real missing data -- it is a sensitivity analysis showing how much your pooled estimate might shift if the asymmetry were entirely due to publication bias. Reporting both the original and trim-and-fill-adjusted pooled estimates gives readers a transparent sense of how sensitive your conclusion is to this threat.

Other approaches worth knowing

Comparing published results against clinical trial registries can reveal registered but never-published or never-reported studies directly, which is a more direct check than any funnel-plot-based statistical method, though it requires the intervention area to have a reasonably complete trial registry to compare against. Selection models and p-curve analysis are more advanced approaches that explicitly model the publication process itself, and are increasingly used alongside funnel-plot methods in more statistically sophisticated reviews.

What to actually report

At minimum, a defensible meta-analysis with ten or more included studies should report a funnel plot, a formal asymmetry test such as Egger's test, and an honest discussion of what any detected asymmetry might mean for the reliability of the pooled estimate. With fewer than ten studies, acknowledge in your limitations that formal publication bias testing was underpowered, rather than omitting the discussion entirely or running a test you know is unreliable at that sample size and reporting it without caveat.

Publication bias assessment is one of the PRISMA 2020 checklist items most often handled superficially -- a funnel plot included without any accompanying statistical test, or a test run without acknowledging its limitations at low study counts.

Time-lag and language bias, two related threats

Publication bias is not the only selective-reporting threat worth naming. Time-lag bias occurs when studies with striking, favorable results get published faster than studies with null results, meaning an early meta-analysis on an emerging topic can be skewed simply because the null-result studies have not caught up in the publication pipeline yet. Language bias occurs when a search strategy restricted to English-language databases systematically misses non-English studies, which can skew a pooled estimate if the excluded studies differ systematically in their findings. A related and often overlooked issue is outcome reporting bias, where a study reports some measured outcomes but selectively omits others -- typically the outcomes that turned out null or unfavorable -- even though the study itself was published in full. This differs from publication bias in that the study exists and is indexed, but the specific outcome you need for your meta-analysis may simply be missing from its published results, sometimes despite having been pre-specified in the study's own trial registration. Comparing a study's registered outcomes against what it actually reports is a useful, if time-consuming, check worth applying to your included studies where trial registries are available. Both are worth acknowledging explicitly in your limitations section, since they operate through a similar mechanism to publication bias but require different mitigation -- broader database coverage and multilingual search terms, rather than a funnel plot.

Doing this section properly, and being honest about what your specific data can and cannot tell you, is a meaningful signal of methodological maturity to anyone reviewing your work.