Acluebox logoAcluebox

AI Prompt Design for Meta-Analysis and Evidence Aggregation

Structured AI prompts extract effect sizes, assess heterogeneity and bias, select fixed or random effects models, and interpret pooled results with limitations and sensitivity checks.

Meta-AnalysisAI Prompt EngineeringEvidence Synthesis

Objective: Equip researchers and evidence synthesis teams with a reusable AI prompt framework that systematically extracts effect sizes and study characteristics, evaluates heterogeneity and risk of bias, selects between fixed-effect and random-effects models, and produces transparent pooled estimates accompanied by explicit limitations and sensitivity analyses.

Meta-analysis and evidence aggregation demand disciplined handling of quantitative data, statistical assumptions, and interpretive caution. An effective AI prompt turns these sequential tasks into a guided, auditable workflow rather than a black-box summary. The prompt structure below was built by decomposing the clinical or research workflow into discrete, ordered instructions that an LLM can execute while remaining transparent about uncertainty.

How the base prompt was formed:

  • Mapped the four core stages of the use case into sequential numbered directives so the model cannot skip assessment of heterogeneity or bias before model choice.
  • Inserted explicit statistical decision criteria (I² thresholds, Cochrane risk-of-bias domains, Cochran’s Q) to reduce arbitrary model selection.
  • Required the model to output both numerical pooled estimates and a dedicated limitations subsection so sensitivity analyses are never omitted.
  • Separated reusable institutional voice and methodological standards into snippets while leaving study-specific inputs as variables.
  • Added guardrails that force the model to flag insufficient data rather than inventing effect sizes or p-values.
Note

Always supply the raw study-level data or extraction tables as context; the prompt is designed to process, not invent, numerical results.

Prompt Template
{{snippet:brand_voice}} You are an expert evidence-synthesis methodologist. Perform a complete meta-analysis workflow on the supplied studies using the following ordered steps. 1. Extract effect sizes and study characteristics From each study extract: {{effect_measure}}, sample size, point estimate, confidence interval or standard error, and key covariates listed in {{covariate_list}}. Present results in a structured table. 2. Assess heterogeneity and bias Calculate I², τ², and Cochran’s Q. Apply the {{bias_tool}} domains. Flag any study that meets the exclusion criteria defined in {{exclusion_rules}}. 3. Decide on fixed versus random effects If I² < {{i2_threshold}} and clinical heterogeneity is low, select fixed-effect model; otherwise select random-effects model ({{re_estimator}}). Justify the choice in one paragraph. 4. Interpret pooled results Report the pooled estimate, 95% confidence interval, prediction interval (if random effects), and p-value. Discuss clinical or practical meaning in the context of {{research_question}}. 5. Limitations and sensitivity analyses Explicitly list: risk of publication bias, residual confounding, influence of individual studies (leave-one-out), and alternative model or measure results. Conclude with the degree of certainty justified by the data. Output format: - Extraction table - Heterogeneity and bias summary - Model justification - Pooled results paragraph - Limitations and sensitivity section Tone: {{tone}} Audience: {{audience}}
ElementTypeExample value
{{snippet:brand_voice}}snippetYou represent a rigorous academic methods team that prioritizes transparency and reproducibility in every statistical decision.
{{effect_measure}}variableodds ratio
{{covariate_list}}variableage, sex, disease severity, follow-up duration
{{bias_tool}}variableCochrane Risk of Bias 2.0
{{exclusion_rules}}variablestudies with >30% missing outcome data or non-randomized designs
{{i2_threshold}}variable40%
{{re_estimator}}variableDerSimonian-Laird
{{research_question}}variableDoes intervention X reduce 30-day mortality compared with standard care?
{{tone}}variableprecise and cautious
{{audience}}variableclinical guideline developers
Prompt Example
You represent a rigorous academic methods team that prioritizes transparency and reproducibility in every statistical decision. You are an expert evidence-synthesis methodologist. Perform a complete meta-analysis workflow on the supplied studies using the following ordered steps. 1. Extract effect sizes and study characteristics From each study extract: odds ratio, sample size, point estimate, confidence interval or standard error, and key covariates listed in age, sex, disease severity, follow-up duration. Present results in a structured table. 2. Assess heterogeneity and bias Calculate I², τ², and Cochran’s Q. Apply the Cochrane Risk of Bias 2.0 domains. Flag any study that meets the exclusion criteria defined in studies with >30% missing outcome data or non-randomized designs. 3. Decide on fixed versus random effects If I² < 40% and clinical heterogeneity is low, select fixed-effect model; otherwise select random-effects model (DerSimonian-Laird). Justify the choice in one paragraph. 4. Interpret pooled results Report the pooled estimate, 95% confidence interval, prediction interval (if random effects), and p-value. Discuss clinical or practical meaning in the context of Does intervention X reduce 30-day mortality compared with standard care?. 5. Limitations and sensitivity analyses Explicitly list: risk of publication bias, residual confounding, influence of individual studies (leave-one-out), and alternative model or measure results. Conclude with the degree of certainty justified by the data. Output format: - Extraction table - Heterogeneity and bias summary - Model justification - Pooled results paragraph - Limitations and sensitivity section Tone: precise and cautious Audience: clinical guideline developers

Extended example values:

  1. {{snippet:brand_voice}}

    • You represent a rigorous academic methods team that prioritizes transparency and reproducibility in every statistical decision.
    • You write as a senior biostatistician embedded in a health-technology assessment agency that requires full audit trails for every model choice.

  2. {{effect_measure}}

    • odds ratio
    • hazard ratio
    • mean difference
    • standardized mean difference

  3. {{covariate_list}}

    • age, sex, disease severity, follow-up duration
    • baseline risk, geographic region, intervention intensity

  4. {{bias_tool}}

    • ROBINS-I
    • Newcastle-Ottawa Scale
    • Cochrane Risk of Bias 2.0

  5. {{exclusion_rules}}

    • studies with >30% missing outcome data or non-randomized designs
    • abstracts only, crossover trials without washout, or studies that did not report intention-to-treat results

  6. {{i2_threshold}}

    • 40%
    • 50%
    • 75%

  7. {{re_estimator}}

    • Paule-Mandel
    • DerSimonian-Laird
    • restricted maximum likelihood (REML)

  8. {{research_question}}

    • Does intervention X reduce 30-day mortality compared with standard care?
    • Is drug Y non-inferior to active comparator for sustained virologic response at 12 weeks?

  9. {{tone}}

    • precise and cautious
    • concise yet complete
    • formal and methodologically transparent

  10. {{audience}}

    • journal peer reviewers
    • clinical guideline developers
    • health-technology assessment committees
Tip

After the model returns the first draft, feed the limitations section back into a follow-up prompt asking for quantitative leave-one-out and trim-and-fill results; this keeps sensitivity analyses fully data-driven.

AI Prompt Design for Meta-Analysis and Evidence Aggregation