Methodology is key!

From Michael Ireland's presentation at USQ

Like any research study a review must adhere to a reliable, robust and replicable protocol (a detailed plan) 

The protocol specifies its central objectives, concepts, and methods in advance

Steps and decisions are carefully documented so that readers can follow and evaluate reviewers’ methods 

The research question is formally stated

Writing this down can identify issues early on since many people don’t realise they are lacking a clear and specific research question until they have to write it out

The search strategies are carefully thought out

Think of it like an empirical research study where you are going out collecting data to answer a research question or test hypotheses

Careful thought needs to go into the nature of the sample you will collect

Criteria for inclusion and exclusion of studies is specified a priori

A priori specification of selection criteria limits reviewers’ freedom to select studies on the basis of their results or on some other basis

Thus protecting the review from unexamined selection bias

The tools for extraction and appraisal are decided and available

The timeline is given consideration