Definitions

What is a systematic review?

Researchers, providers and practitioners receive unmanageable amounts of information on health topics. However, they don’t always have the resources or skills to critically appraise this information and incorporate it into their practices. Systematic and other types of reviews identify, evaluate and synthesize research and present evidence in a clear, accessible format for translation into policy and practice.

A systematic review uses clear and transparent procedures to search for studies, assess the studies for relevance and quality, and synthesize the results, based on a clearly defined research question. Reviews can be quantitative, qualitative, or mixed methods in nature depending on the research question. Reviews typically follow similar procedures, which are explicitly outlined in advance to promote transparency and minimize bias. Other types of reviews may modify aspects of these steps due to time and resource constraints.

The Five Step Process

Reviews published on Emerging Questions typically follow a five-step process.

Emerging Question

The critical first step of conducting a review is to define the research question. Emerging Questions come from applied researchers, practitioners and other professionals in adolescent health (to submit an idea for an Emerging Question, go to our survey). Questions may be very broad or narrowly specified, depending on the goal of the review or other factors, such as time and resources.

Review questions typically specify the population of interest (e.g., LGBTQ youth), an intervention type (e.g., peer mentoring) or other exposure (e.g. the influence of parent-child connectedness), and outcomes (e.g. young people’s attitudes, knowledge or behaviors). These components of the review question help form the basis for eligibility criteria for including studies in the review.

Inclusion Criteria

Inclusion criteria (also called eligibility or exclusion criteria) are developed to guide the search and screening of studies for the synthesis. In addition to clearly outlining the population, intervention or other exposure, and outcomes defined in the research question, the inclusion criteria also explicitly defines other study characteristics, such as: study design (e.g. randomised control trial, qualitative, etc.), timeframe, language, and setting (e.g. country, rural, school, etc.). Authors may register a protocol, outlining their inclusion criteria and other review methods, to establish their methods have been determined prior to conducting the review.

Search Strategy

Using the established inclusion criteria, a search strategy is designed to identify as many relevant studies as possible within the available time or resources. Search terms are developed according to the population, type of intervention or exposure, outcome and study design, using Boolean operators such as “and”, “or”, and “not” to expand or limit searches. Search strategies typically comprise of bibliographic and journal database searches and may also include general internet searches for studies not captured in peer-reviewed literature. After the search has been conducted, study abstracts and full text are screened to determine whether the study meets the criteria for inclusion in the synthesis.

Coding and Analysis

If a study meets the inclusion criteria, then relevant data, such as study characteristics and outcomes, are recorded into a standard format, such as a table or information management system. Depending on the time and resources available, the data extraction phase may be conducted by multiple reviewers independently to minimize errors and reduce bias. Information from data extraction is used as a part of quality appraisal, synthesis and in summary of findings tables.

Before study information is synthesized, studies are coded for quality (called quality assessment, critical appraisal or risk of bias). Standards used to assess quality are different within quantitative and qualitative reviews. Given the wide range of reporting standards across fields, qualitative reviews typically assess for quality but do not exclude studies based on quality. Quantitative studies, especially those engaging in meta-analysis, will assess study risk of bias and may choose to exclude some studies based on the level of bias determined. Results from the quality assessment phase will influence the analysis, interpretation, and conclusions of the review.

Synthesis and Translation

Multiple methods exist for the synthesis of qualitative, quantitative and mixed methods reviews. Qualitative reviews use synthesis methods based on the same principles as analysis in primary studies. Some examples of qualitative synthesis methods include meta-ethnography developed by Noblit and Hare (1988) and thematic synthesis (Thomas and Harden 2008). The most common form of quantitative synthesis is a meta-analysis, the use of statistical methods to combine the results of two or more independent studies. Meta-analysis is not a required step of a systematic review; however, when used appropriately, meta-analysis has many potential advantages, including increased power, improved precision, the ability to answer questions beyond those posed by individual studies, and the opportunity to settle controversies arising from conflicting claims.

A final and crucial step is the translation of evidence. Tables and figures help present included studies and their findings in a clear and accessible format. Considering the quality of the included studies, results of the synthesis are discussed in the context of existing research and practice within the field. These findings are then used to make recommendations in practice, policy and research. 

Resources on Systematic and Other Reviews

Cochrane Handbook for Systematic Reviews of Interventions provides detailed guidance the development and execution of reviews, with standards for conducting meta-analysis. Cochrane Library hosts six databases that contain different types of high-quality, independent evidence to inform healthcare decision-making.

Campbell Collaboration hosts a database of reviews, summaries and policy briefs for evidence-based decision making in health and includes resources and training in review topics.

PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

EPPI-Centre provides resources on methods for systematic reviews and research syntheses and studies the use/non-use of research evidence in personal, practice and political decision-making.

Gough D, Oliver S and Thomas J. (2017). An Introduction to Systematic Reviews. London: Sage.

Noblit GW and Hare RD (1988). Meta-Ethnography: Synthesizing Qualitative Studies. London: Sage.

Thomas J and Harden A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8(45).