Skip to Main Content

Health Research Literature: an Overview

A guide to types & hierarchy of medical research sources

Overview of the Systematic review

 

Unlike other types of reviews " a systematic review attempts to collate all the empirical evidence that fits prespecified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which to draw conclusion and make decisions."

Higgins, J. (ed.) (2019) Cochrane Handbook for systematic reviews of interventions. p.4

A systematic review is undertaken to
  • evaluate evidence based studies on the effectiveness of a clinical treatment  or
  •  evaluate the evidence on diagnostic test accuracy or
  • examine prognostic factors or prognostic models.
and is based on
  • a clearly defined clinical question modeled on  PICO : Patient or Population, Intervention Comparison Outcome  
  • adherence to a strict protocol 
  • a comprehensive peer reviewed & reproducible search strategy performed on at least two relevant databases
  •  a standardized method of data extraction
  • critical appraisal of all studies included
  • synthesis & summary of  all findings.

Systematic Review Steps

1. Determine the need for a systematic review on your topic based on your PICO question.

  • Run a quick search on PubMed for a reviews related to your topic to see
    • if it has been systematically reviewed & and if so, when.
    • If it was not done recently, try a forward citation search on a tool like Scopus or Google Scholar to see if recent research that cites the review may be relevant.
  • If you find no reviews on your topic, look more closely at the studies you do find to see if it is feasible to build a review from them. Do not be surprised if this exercise results in slight changes to your original topic question.
  • You can also search systematic review protocol registries  to see any reviews in progress

2. Find your team : A Systematic Review with or without meta- analysis requires team members with different types of experience:

  • A librarian or other person experienced in developing search strategies & knowledgeable about how to systematically search databases appropriate for your review
  • At least 3 "Expert in the field" reviewers for screening &  evaluation of all studies found
  • A statistician if doing a meta-analysis

3. Consider your time frame. With a well organized plan in place, a systematic review can take anywhere from a year to 18 months to complete.

4. Consider publishing requirements  Will you be able fulfill the requirements for Systematic Reviews?

 

 A Protocol is a detailed planning document that describes the methodology you will use in detail. There are several resources/examples related to protocols that may be useful:

For consistency & to avoid bias, after the research question is developed but before your search is completed, your protocol should describe your criteria for determining eligibility of studies to be included in the review. For example, decide on :

Date of publication Languages Study Design

Intervention
of interest

Outcome
of Interest

Setting Publication Type

A date range
specific to
timing of
intervention or
therapy

Only English?

What other languages
can your team handle?

Which type of
studies will
best address
your question?

Include studies describing
your intervention
of interest only?

Exclude all others?

Include studies
with your outcomes
of interest only?

GP surgery?
Ambulatory
patients only?
Classroom? etc.

 

Exclude:
Review articles?
Editorials?
Commentaries?
Letters?

Guides to Protocol Creation/Registration

A flow chart that describes the screening of your search results is mandatory in a systematic review.

Once you have completed your  literature searching & sorted all citations found, screening is the next step.  The purpose of screening is to eliminate studies that do not meet your inclusion criteria. Two independent reviewers should screen all studies, starting with a title and abstract screening, followed by a full-text screening. A third reviewer should resolve any conflicts.

Start by title/abstract screening each reference. Each reviewer should read the title & abstract to decide:

  • No: This article does not meet inclusion criteria and should not be included in the systematic review.
  • Maybe: There is not enough information in the title/abstract to make a decision. Add to full-text screening stage 
  • Yes: This article appears to meet inclusion criteria and should move to the full-text screening stage.

During the full-text screening, for each reference, read the full-text and make a decision:

  • Include: This article meets inclusion criteria and should be included in the systematic review.
  • Exclude: This article does not meet inclusion criteria and should not be included in the systematic review.

Voting  on what should be included or excluded should be blinded. Team team members should not be able  to see how others voted.

Tools to evaluate specific study types

Case-controlled Studies

CASP Checklist:11 questions to help you make sense of a Case Control Study

Tool to Assess Risk of Bias in Case Control Studies

Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomised studies in meta-analyses

Joanna Briggs Institute Checklist for case-controlled studies

Mixed Methods

McGill Mixed Methods Appraisal Tool (MMAT)

Cohort Studies

CASP Checklist:12 questions to help you make sense of a Cohort Study

Joanna Briggs Institute Checklist for Cohort Studies

The Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomised studies in meta-analyses

Qualitative studies

CASP Checklist:10 questions to help you make sense of Qualitative research study

 

Diagnostic Studies

CASP Checklist:12 questions to help you make sense of a Diagnostic Test study

QUADAS : quality assessment tool for diagnostic accuracy studies

Joanna Briggs Institute Checklist for Diagnostic Test Accuracy Studies

Randomized Controlled Trials

RoB 2 tool : revised Cochrane risk of bias tool for randomized trials

CASP Checklist:11 questions to help you make sense of a Randomised Controlled Trial

Centre for Evidence Based Medicine (CEBM) Appraisal Tool for RCTs

Joanna Briggs Institute Checklist for Randomized Controlled Trials

Economic studies

CASP Checklist:12 questions to help you make sense of an Economic Evaluation

CHEC list: Consensus Health Economic Criteria

 

 

Systematic reviews

AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews

AMSTAR : Assessing the methodological quality of systematic reviews

CASP Checklist:10 questions to help you make sense of a Systematic Review

 

 

  • Excel
  • Google Sheets

Once your literature search is complete,  the studies retrieved  must be organized by database in which they were found & deduplicated before the screening process begins. Most databases also enable you to download your search results to Excel or other spreadsheet software .This may be the best way to manage large search results files from several databases.Please contact the library if you need further assistance.

ProQuest Refworks is the bibliographic management software available free to all students and faculty suited to smaller research results management. It is also useful for creation of your final reference list for your systematic review. Please visit our guide here

 

It is a good idea to revisit the methodology & reporting guidance sources listed here as you write each section of your systematic review.

Systematic review support resources

Systematic review methodology & protocol guides
PRISMA guidance for transparent reporting of systematic reviews & meta analysis 

In addition to the methodology guides, use these recently update guides to correctly report what was done & what was found in the evidence gathered in your systematic review.

When planning a protocol for your systematic review, see also