What approach to use?

Back to: Analyze & Synthesize Data

If averaging appraoches make so much of a difference, what appraoch should you use- #1 or #2

If you are conducting meta-analysis, you should always use approach #2, where you find the difference between groups in each study first, and then average those differences across each study to find the overall difference.

We do this because in each individual study, the participants are all exposed to a similar environment, which is rarely the same as the environments for participants in other studies.  When we compare differences within a study first, we’re able to pretty much hold those environmental influences constant. That makes it more likely we’re comparing the actual difference between the groups and not some other factor.

It’s sort of like the way we organize sporting competitions. We don’t choose the best team in the world as the one with the best overall record because all the teams are not playing against the same level of competition.

Instead, we have teams compete within their league (the same environment) and then we have competitions across environments by having the champions of each league compete in a final tournament.

But, what if we know that one league is more competitive than another league? Or we believe that one study is more reliable than another?

Meta-analysis can account for those differences using what we call “weighting.”