The difference between a rapid evidence assessment and a proper systematic review
If someone hands you a research brief and a six-week deadline in the same breath, do you actually know which kind of evidence review you are commissioning? Most project leads say yes, then describe a rapid evidence assessment when they mean a systematic review, or the other way around. The confusion is understandable: both involve searching literature, screening studies, and producing a synthesis document, and both get called "the evidence base" in board papers. But the differences between them are not cosmetic. They determine how far you can trust the answer you get back, and whether that answer will survive scrutiny from a policy audience, a funder, or a regulator. This piece maps those differences honestly, including the parts that most methodology guides quietly skip over.
What question are you actually trying to answer?
The single most useful thing you can do before choosing a review method is write your research question in one sentence and then ask whether it is asking what exists, what works, or how much you can trust the evidence that things work. Rapid evidence assessments are well suited to the first two. Systematic reviews with meta-analysis are the right tool for the third. That distinction sounds obvious, but a large share of review commissions conflate all three into a single brief, which is how organisations end up with a six-week rapid review that is then cited as if it carried the evidentiary weight of a Cochrane review.
A concrete example helps. Suppose a national health agency wants to know whether community-based diabetes prevention programmes improve HbA1c outcomes in adults over 55. That is a tightly scoped effectiveness question with a measurable outcome, a defined population, and an implied comparison. It deserves a systematic review. If the same agency instead asks what community-based diabetes programmes currently exist across the country and what their main design features are, that is a mapping question, and a rapid evidence assessment is entirely appropriate. The problem is that both briefs often land in the same inbox under the heading 'evidence review', and the method gets chosen by deadline rather than by question type.
What a rapid evidence assessment actually does — and does not do
A rapid evidence assessment, often shortened to REA, applies a systematic search strategy but then deliberately limits scope to make the work feasible in a compressed timeframe. That compression typically shows up in four places: the number of databases searched, the date range of the search, the language of included studies, and the depth of quality appraisal applied to each paper. A well-conducted REA will be transparent about all four constraints in its methodology section. A poorly conducted one will not mention them at all, which is where reputational risk for commissioners begins.
The honest case for an REA is that it can surface the broad contours of an evidence landscape in four to eight weeks, which is genuinely useful for scoping a policy question, informing a funding call, or preparing a project team before a more thorough review begins. The Cabinet Office in the UK formalised REAs as a standard tool for exactly this purpose, and equivalent guidance has been developed in Australia through the Australian Institute of Health and Welfare and in Canada through the National Collaborating Centre for Methods and Tools. The honest case against an REA is that its conclusions carry a confidence ceiling. Because quality appraisal is abbreviated, because grey literature is often excluded, and because non-English studies are routinely missed, an REA can give you a confident-sounding summary of a biased subset of the evidence without anyone in the review team having done anything wrong.
An REA tells you what the loudest part of the literature is saying — not necessarily what the full literature is saying.
What a proper systematic review requires
A systematic review conducted to contemporary standards — PRISMA 2020 is the most widely used reporting framework — involves a pre-registered protocol, a search strategy peer-reviewed by a second librarian or information specialist, dual screening of titles and abstracts, dual full-text review, structured quality appraisal using a validated tool such as the Cochrane Risk of Bias 2 tool or the GRADE framework, and a synthesis that explicitly accounts for heterogeneity across studies. Each of those steps takes time. A realistic timeline for a systematic review covering a moderately active research area is nine to eighteen months, and that assumes a team of at least three reviewers plus statistical support if a meta-analysis is planned.
The registration step is worth dwelling on, because it is what separates a systematic review from a well-organised literature search with a tidy report at the end. Pre-registering a protocol on PROSPERO (for health and social care reviews) or the Open Science Framework means that the research question, eligibility criteria, and planned analysis are publicly committed before a single paper is screened. That commitment makes it much harder to quietly adjust the inclusion criteria after you have seen the results, which is one of the main ways that a synthesis can produce findings that look robust but are actually shaped by post-hoc decisions. Commissioners who do not ask whether a review was pre-registered are leaving themselves open to that risk.
Where each method tends to break down
REAs break down most visibly when their outputs are used to make stronger claims than the method supports. This happens more often than methodology purists would like to admit, because the people commissioning reviews and the people writing policy papers are often not the same people, and the caveats in the methodology section rarely survive intact into the executive summary. A 2021 analysis of government-commissioned evidence reviews in Australia found that confidence language in executive summaries was consistently stronger than the confidence language in the underlying methodology sections — a finding that should make anyone who writes or reads executive summaries a little uncomfortable.
Systematic reviews break down differently. Their main failure mode is not overconfidence but obsolescence and inaccessibility. A systematic review published in month one of a fast-moving research area can be substantially out of date by month eighteen, and the technical density of a well-conducted review — forest plots, I-squared statistics, GRADE evidence tables — means that the audiences most likely to act on the findings are also the audiences least equipped to interpret them without translation. Living systematic reviews, which update continuously as new trials are published, are an emerging response to the obsolescence problem, but they require sustained funding and a committed author team, neither of which is easy to secure.
A practical checklist for choosing between the two
Most of the time, the choice comes down to four variables: the precision of the decision that depends on the evidence, the available timeline, the budget, and the existing volume of literature in the area. The following points are the ones worth working through before you brief a review team.
- Write the decision the evidence will inform in one sentence. If it involves setting a national standard, approving a clinical protocol, or allocating significant long-term funding, a systematic review is the appropriate tool.
- Check PROSPERO and the Cochrane Library before commissioning anything. An existing systematic review published in the last three years may answer the question already.
- If your timeline is under three months and the question is primarily a mapping or scoping question, an REA is defensible — provided its limitations are reported clearly and are carried into any documents that cite it.
- Ask whether the evidence base is primarily in English. If significant research exists in other languages relevant to your context (common in European policy work), an English-only rapid review will systematically underrepresent it.
- Budget for quality appraisal even in a rapid review. Cutting this step entirely is where REAs lose credibility fastest.
- If the question will be revisited in twelve to twenty-four months, consider commissioning an REA now with a planned update, rather than a single systematic review that may be outdated before it is used.
The version of this decision that most organisations actually face
In practice, the choice between an REA and a systematic review rarely presents itself cleanly. What usually happens is that a policy team needs something in six weeks, the brief says 'systematic review', and a review team either quietly conducts an REA and calls it something more impressive, or attempts a genuine systematic review on an REA timeline and produces something that satisfies neither standard. Both outcomes are common enough to have become a running tension in the evidence synthesis community, and both are largely preventable.
The most effective intervention is earlier involvement of whoever will conduct the review in the scoping conversation, before the brief is written and before the timeline is set. Review teams can often help commissioners articulate what kind of question they are actually asking, which then makes the method choice straightforward. When that conversation happens after the brief is finalised, the method is usually already implied by the deadline, and the work that follows is a negotiation rather than a design.
Across the evidence synthesis projects Aurora Pioneer has been involved in over the past several years, the pattern holds: the reviews that generated the most durable, credible outputs were the ones where the question type, the method, and the timeline were aligned from the start. The reviews that caused the most trouble were the ones where a systematic review was promised on a rapid review budget, or where an REA was presented without its limitations intact. Neither problem is technically complicated. Both are organisational.
The distinction between these two review types is not arcane methodology — it is a practical question about how much confidence a piece of evidence synthesis is actually licensed to carry. Getting that match right at the start of a project is considerably less costly than correcting it after a report has been cited in a cabinet submission or a funding announcement. Most active research areas now have enough published systematic reviews and REAs that a brief literature check before commissioning is worth the afternoon it takes.