Whether you side with the first group of researchers or the second group of researchers, the point is clear: assessing for risk of bias requires judgement and is subjective.
If you’ve taken our earlier courses, you’ll know that this step is different from the way we approach most other steps of a systematic review. There are relatively clear and straightforward methods for steps like searching for studies or extracting data.
Because of the subjectivity involved, it’s more imperative to be transparent. Whatever you decide about the risk of bias for the studies you find, be sure to document your decisions, describe your reasoning, and be consistent.
Let’s move on to some common approaches for making these decisions.