Read more (original 2016 version)

The ROBINS-I tool (Risk Of Bias In Non-randomized Studies - of Interventions) is concerned with evaluating the risk of bias (RoB) in the results of non-randomized studies of interventions (NRSI) that compare the health effects of two or more interventions. The types of NRSI that can be evaluated using this tool are quantitative studies estimating the effectiveness (harm or benefit) of an intervention, which did not use randomization to allocate units (individuals or clusters of individuals) to comparison groups. This includes studies where allocation occurs during the course of usual treatment decisions or peoples’ choices: such studies are often called “observational”. There are many types of such NRSI, including cohort studies, case-control studies, controlled before-and-after studies, interrupted-time-series studies and controlled trials in which intervention groups are allocated using a method that falls short of full randomization (sometimes called “quasi-randomized” studies).

Read our paper about the tool in The BMJ.

The ROBINS-I tool is based on the Cochrane RoB tool for randomized trials, which was launched in 2008 and modified in 2011 (see BMJ paper). As in the tool for randomized trials, risk of bias is assessed within specified bias domains, and review authors are asked to document the information on which judgements are based. ROBINS-I also builds on related tools such as the QUADAS 2 tool for assessment of diagnostic accuracy studies by providing signalling questions whose answers flag the potential for bias and should help review authors reach risk of bias judgements. Therefore ROBINS-I provides a systematic way to organize and present the available evidence relating to risk of bias in NRSI.

Both ROBINS-I and the Cochrane RoB tool for randomized trials focus on studies’ internal validity, For both types of study, we define bias as a tendency for study results to differ systematically from the results expected from a randomized trial, conducted on the same participant group, that had no flaws in its conduct (for example, a large trial that achieved concealment of randomized allocation; maintained blinding of patients, health care professionals and outcome assessors to intervention received throughout follow up; ascertained outcomes in all randomized participants; and reported intervention effects for all measured outcomes). Such bias is distinct from issues of generalizability (applicability) to types of individual who were excluded from the study. For example, restricting the study sample to individuals free of comorbidities may limit the utility of its findings because they cannot be generalized to clinical practice, where comorbidities are common.

ROBINS-I (2016) was developed by members of the Cochrane Bias Methods Group and the Cochrane Non-Randomised Studies Methods Group. See a list of the team who were involved. It was supported by a grant from the Cochrane Methods Innovation Fund (2011).

To cite the ROBINS-I tool, please use:

Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, Henry D, Altman DG, Ansari MT, Boutron I, Carpenter JR, Chan AW, Churchill R, Deeks JJ, Hróbjartsson A, Kirkham J, Jüni P, Loke YK, Pigott TD, Ramsay CR, Regidor D, Rothstein HR, Sandhu L, Santaguida PL, Schünemann HJ, Shea B, Shrier I, Tugwell P, Turner L, Valentine JC, Waddington H, Waters E, Wells GA, Whiting PF, Higgins JPT. ROBINS-I: a tool for assessing risk of bias in non-randomized studies of interventions. BMJ 2016; 355; i4919; doi: 10.1136/bmj.i4919.

Archived version

Note: The tool was previously called A Cochrane Risk Of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBAT-NRSI). See archived version.

To cite the earlier ACROBAT-NRSI tool, please use:

Sterne JAC, Higgins JPT, Reeves BC on behalf of the development group for ACROBAT-NRSI. A Cochrane Risk Of Bias Assessment Tool: for Non-Randomized Studies of Interventions (ACROBAT-NRSI), Version 1.0.0, 24 September 2014.  Available from http://www.riskofbias.info [accessed {date}].