Comprehensive Meta-Analysis

How does CMA compare to Revman?

Data entry is much easier with CMA.

The mechanics of data entry are much simpler in CMA – you work with a spreadsheet interface, and can copy-and paste-data as easily as you could in Excel. By contrast, the data entry process in Revman requires the user to set up tables and comparisons before starting data entry.

CMA will compute the effect sizes automatically.

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference.

Revman will accept summary data in only two formats – events and sample size, or means and standard deviations. If any studies provide data in another format (such as odds ratio and confidence intervals) you would need to compute the effect sizes and variances manually for those studies. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. Equally important, Revman requires that data for all studies be entered using the same format. By contrast, with CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis. CMA also supports a much wider range of effect sizes than Revman.

CMA is able to create a customized, high-resolution forest plot.

The forest plot in Revman offers few options for customization. By contrast, CMA allows the user full control over all elements in the forest plot, will create scalable plots (that print at the highest resolution possible for the printer or journal), and allows the user to control the color for every element on the plot. CMA also allows one-click export to other programs such as PowerPoint™ and Word™.

Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

Advanced Functions

CMA allows you to assess the impact of moderator variables. Use analysis of variance to compare the treatment effect across groups (“Is the treatment more effective for acute patients than for chronic patients?”). Use meta-regression to assess the impact of continuous moderators (“Does the treatment effect increase with dosage?”). To assess the potential impact of publication bias CMA includes an array of functions including a funnel plot, where Revman includes only the funnel plot. CMA will run a cumulative meta-analysis to show how the evidence has shifted over time. It will also run a one-study removed analysis to show the impact of each study on the combined effect.

Does CMA offer the same formulas as Revman?

Yes. The development team for CMA includes some of the same people responsible for the development of Revman. CMA includes all of the same computational formulas, has been validated against Revman and provides exactly the same results (see documentation). CMA offers additional options as well, but includes a button to "Use the same options as Revman," which sets all options to match Revman.

NEXT: R

How does CMA compare to R?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

R is a general purpose statistical package. While R has no intrinsic support for meta-analysis, various experts have written procedures for basic analysis, for cumulative analysis, for meta-regression, for publication bias, and more. Since the functionality of R and CMA are comparable, the main difference is in ease of use and in the options for customizing the output.

R is a command-driven language, which means that you type commands, or use a dialog box to create commands, which are then submitted to the program. CMA is a menu-driven program, similar to Excel™.

Computing effect sizes

In R, you need to compute the effect size and variance for each study and then build a database of these effects prior to running the meta-analysis. If different studies provide data in different formats you would need to compute the effect sizes and variances using various functions. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. With CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis.

Analyses

R computes values and sends these to a DOS-like window for viewing. In CMA the analysis screen is interactive – you can use the screen interactively to explore the impact of different studies, the effect of alternate weighting schemes, and so on.

Forest plots

To create a forest plot in R you need to work with code that is not terribly intuitive. By contrast, the forest plot in CMA can be customized extensively.

Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

Does CMA offer the same formulas as R?

Yes. CMA includes all of the same computational formulas, was validated against R and provides exactly the same results (see documentation).

Can CMA import data from R?

Yes. You can open the data sheet in R, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column. The process takes only moments to complete.

PREVIOUS: Revman NEXT: Stata

How does CMA compare to Stata?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

Stata is a general purpose statistical package. The functionality of Stata and CMA are comparable. The main difference is in ease of use and in the options for customizing the output. Stata is a command-driven language, which means that you type commands, or use a dialog box to create commands, which are then submitted to the program. CMA is a menu-driven program, similar to Excel™.

Computing effect sizes

Stata will accept summary data in only three formats – events and sample size, means and standard deviations, or (in some cases) point estimate and confidence interval. If any studies provide data in another format you would need to compute the effect sizes and variances manually or by writing code. By contrast, CMA allows you to enter data in more than 100 formats, and will compute the effect size and variance for all of these formats. Equally important, Stata requires that data for all studies be entered using the same format. By contrast, with CMA you can enter data for each study in its own format, and use as many formats as needed in the same analysis.

Analyses

Stata computes values and sends these to a DOS-like window for viewing. In CMA the analysis screen is interactive – you can use the screen interactively to explore the impact of different studies, the effect of alternate weighting schemes, and so on.

Forest plots

Stata’s forest plot offers few options for customization. It includes a column for the study name and a symbol representing the point estimate and confidence interval. By contrast, the forest plot in CMA can be customized extensively.

Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

Does CMA offer the same formulas as Stata?

Yes. The development team for CMA includes some of the same people who developed the Stata macros. CMA includes all of the same computational formulas, was validated against Stata and provides exactly the same results (see documentation). CMA offers additional options as well, but includes a button to "Use the same options as Stata," which sets all options to match Stata.

Can CMA import data from Stata?

Yes. You can open the data sheet in Stata, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column. The process takes only moments to complete.

PREVIOUS: R NEXT: Spss

How does CMA compare to SPSS?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

SPSS is a general purpose statistical package with no intrinsic support for meta-analysis. However, David Wilson has written macros that can be incorporated into SPSS and will run a basic meta-analysis, an analysis of variance, and meta-regression.

Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Wilson’s macros require the user to compute an effect size and variance for each study, and then provide these values to the program. Therefore, the user must either compute these values separately and then enter them as data, or write code to compute these values within SPSS.

By contrast, with CMA you enter the data directly in almost any format(s), and the program computes the effect size and variance automatically.

Analyses and forest plots

Wilson’s macros will report all relevant statistics, but will not create graphics such as a forest plot. CMA is able to create a forest plot, which can play a key role in helping the researcher to interpret the data and to convey it to others.

Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

Does CMA offer the same formulas as SPSS?

SPSS itself does not include any support for meta-analysis. Wilson’s macros use the same formulas as CMA (CMA offers additional options as well), and so will yield identical results. This assumes, of course, that the user has used the same formulas to compute effect sizes and variance for each study.

Can CMA import data from SPSS?

Yes. You can open the data sheet in SPSS, copy the data onto the Windows clipboard, and then paste it into CMA. Then you tell CMA what kind of data is located in each column.

PREVIOUS: Stata NEXT: Sas

How does CMA compare to SAS?

CMA is a program developed specifically for meta-analysis. As such, it includes functions to automatically compute effect sizes, to perform basic and advanced meta-analyses, and to create publication quality graphics.

SAS is a general purpose statistical package with no intrinsic support for meta-analysis. David Wilson has written macros that can be incorporated into SAS and will run a basic meta-analysis, an analysis of variance, and meta-regression.

Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Wilson’s macros require the user to compute an effect size and variance for each study, and then provide these values to the program. Therefore, the user must either compute these values separately and then enter them as data, or write code to compute these values within SAS.

By contrast, with CMA you enter the data directly in almost any format(s), and the program computes the effect size and variance automatically.

Analyses and forest plots

Wilson’s macros will report all relevant statistics, but will not create graphics such as a forest plot. CMA is able to create a forest plot, which can play a key role in helping the researcher to interpret the data and to convey it to others.

Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

Does CMA offer the same formulas as SAS?

SAS itself does not include any support for meta-analysis. Wilson’s macros use the same formulas as CMA (CMA offers additional options as well), and so will yield identical results. This assumes, of course, that the user has used the same formulas to compute effect sizes and variance for each study.

Can CMA import data from SAS?

Yes. Since SAS does not support copy-and-paste, it is not possible to simply copy the data to CMA. However, SAS is able to export data to a file, which can then be imported to CMA.

PREVIOUS: SPSS NEXT: Excel

How does this program compare to Excel?

Excel is a spreadsheet program with no intrinsic support for meta-analysis. While it is possible to program all meta-analysis formulas in Excel, this requires knowledge of the formulas and a substantial investment of time for development and testing. Also, there is no mechanism in Excel that can be used to create a forest plot.

By contrast, all of this functionality is built into CMA.

PREVIOUS: SAS NEXT: Metawin

How does CMA compare to Metawin?

CMA and Metawin are both dedicated meta-analysis programs but CMA incorporates a much wider and more fully developed set of options.

Computing effect sizes

In every meta-analysis you start with the published summary data for each study and compute the treatment effect (or effect size). For example, if a study reports the number of events in each group you might compute the odds ratio. Or, if a study reports means and standard deviations you might compute the standardized mean difference. Additionally, you need to compute the variance for each effect size.

Both Metawin and CMA will allow you to enter summary data and will compute the effect size from that data. However, CMA will work with a much wider array of data formats. Metawin will accept data in a few formats where CMA can accept more than 100. Metawin requires that all studies provide data in the same format while CMA allows you to enter data for each study in its own format. Metawin can work with a few indices of treatment effect (or effect size) where CMA includes more than 15.

Analyses

Metawin will run the analysis and show the computed values. CMA displays all values as part of a scrollable grid which makes the analysis transparent – you can see which studies are included in the analysis, how the studies were weighted, and so on. CMA also includes a much more extensive set of computational options.

Forest plots

In CMA you can fully customize the plot, to ensure that each study stands out clearly, so that the plot is proportioned properly on the page, include all relevant columns, and so on. In Metawin the plot is very basic, allowing one column for study names but little control over formatting.

Reports

CMA will create a two-page text that reports all the statistics and can serve as a template for publication. It also offers the option to annotate the report with a discussion of what each statistic means, what assumptions are made in computing the statistic, and how to interpret that statistic properly. The program will insert references and you can export the report to Word with one click. Click here to see a sample report.

Prediction Intervals

All software will compute the confidence interval. This tells us how precisely we have estimated the mean effect size but says nothing about how widely the effect size varies across studies. This is the domain of the prediction interval, and most publication guidelines for meta-analysis now encourage researchers to report the prediction interval. CMA allows you to include the prediction interval on the forest plot. Additionally, it is the only program that will create a plot that displays the entire distribution of true effects. This plot can be exported to Word or PowerPoint.

Video Tutorials

CMA includes video tutorials. These tutorials use case studies that show how to perform an analysis from start to finish and how to report the results. Importantly, the videos explain the logic of each step in the context of the larger analysis. You can watch a video from start to finish, to learn about the entire process. Or, you can jump to the part of the video that is relevant to a specific part of the program.

Avoiding Common Mistakes

The vast majority of meta-analyses submitted for publication include mistakes in interpreting the statistics. CMA includes a link to PDFs that discuss these common (and sometimes serious) mistakes in detail. And, the PDFs explain how to avoid these mistakes in your analysis.

PREVIOUS: Excel