Chapter 61

Meta-Analysis in Exercise Physiology

Frank B. Wyatt, PhD
 
 

Meta-analysis has been defined as an analysis of analyses [2].  It is a compilation of research quantified to reach a conclusion about the level of affect a given variable has on a population [3].  In general, a meta-analytic procedure provides a synthesis of all (supposedly) published material pertaining to findings of a specific outcome that may or may not be in question.  Usually it is the former.  In Exercise Physiology it is a procedure in need of application for several reasons: 

  • Humans as subjects research tends to have low sample sizes thus reducing power.
  • Utilizing the mean, more weight is given to studies allowing for proper random selection from the population.
  • Provides a consensus of material in a quantitative analysis of some controversial subject and/or findings.
  • Because the analysis is quantitative in nature, there is reduced subjectivity when compared to traditional qualitative reviews.
Because of the aforementioned reasons for applying a meta-analytic review of research,  research in Exercise Physiology can benefit from such a procedure by forming a consensus about controversial findings, increase power by combining studies and their sample sizes, excluding studies with poor design and spurious findings and by establishing statistical differences through effect size calculations.  This chapter will review the procedures involved with meta-analysis, give examples of research in the field of Exercise Physiology and provide direction for possible future research.

BASIC CONCEPTS IN META-ANALYSIS
Definitions 
As stated before, a meta-analysis is a review of past literature that combines the findings quantitatively and hopefully provides a consensus finding.  Hyllegard et al. [3] describes a meta-analysis as a quantitative approach that analyzes conclusions from several empirical studies concerning a specific topic of research.  As with more traditional forms of research, the general purpose is to reduce the information to allow for general conclusions.  In utilizing this process of research, there is a definite methodology and procedural steps to eventually provide a standard metric that will allow for statistical analysis [10].   Two procedures identified in the meta-analytic procedure distinguish it from the more traditional review methodology:  (1) there is a definite methodology employed in the research analysis; and 
(2) the results of the included studies are quantified to a standard metric thus allowing for statistical techniques for further analysis. 

One of the primary aims when utilizing this research design is to reach a conclusion related to the magnitude of the effect on a specific sample inferred to the population.  So the findings are the mean effect of all the studies included with an overall confidence interval.  The mean is highly influenced by studies with better designs.  This includes those studies with greater sample sizes, proper random selection of the sample and random assignment of treatment, double blinding and low attrition rate of subjects.  Studies of poor design can be excluded from the meta-analysis procedure. 

Another equally important concept associated with empirical research is to develop theory [9].  In general, sound theory provides an adequate and strong explanation of processes that take place in a phenomenon.  An example of this is in relation to the fates of lactate.  Once thought to be a dead-end metabolite associated with delayed onset of muscle soreness, we now know the body utilizes lactate as a fuel during high intensity work.  In establishing a theory the relationship between variables provide the basis of the theory.  Meta-analysis combines these relationships to establish a stronger basis in the establishment of a theory.  A meta-analysis by Wyatt [13] strengthened the relationship between the lactate threshold and ventilatory threshold.   This particular meta-analysis provided a greater basis to the theory that the lactate and ventilatory thresholds are not different.  Lokey et al. [5] provided information for re-establishing guidelines for exercise during pregnancy. 

Schmidt [9] notes that there is a two-step approach in theory development when using the meta-analytic procedure:  (1) use the meta-analytic design to provide more precise estimates of relationships among variables.  The precision is enhanced because the procedure averages out the sampling error deviations from the correct values.  In addition, the procedure provides corrections to mean values with distortions due to measurement error and other possible artifacts; and (2) use the more precise estimates in path analysis to develop and test theories. 
Meta-analysis, when compared to the more traditional literature review, often reveals that qualitative conclusions based on traditional interpretation of statistical findings are often erroneous. 

The procedure clarifies the critical role of sampling error, measurement error and other sources of error in establishing conclusions from individual studies.  Lubsen [6] compared large scale placebo-controlled clinical studies to meta-analysis and concluded a development program can be facilitated with this procedure.  Thus the findings revealed about variable relations and data collection procedures utilizing the meta-analysis procedure may provide major paradigm shifts in empirical research design and knowledge outcomes. 

PROCEDURAL STEPS  IN A META-ANALYSIS
The objective of a meta-analysis is to allow for quantitative analysis of reviewed research literature.  As with traditional experimental designs, the objectives are the following:

  • Describe/Summarize degree to which constructs are related
  • Identify factors explaining variability in relationships of interests
  • Suggest directions for future research
  • Offer new theoretical perspective or resolve conflicts between opposing theories
  • Suggest possible applications of findings
If applied appropriately and interpreted correctly, this type of design can reduce the findings of many studies to applicable principles.  These principles can then become the basis for the development of new programs, provide insight into future research needs and challenge the validity of theoretical constructs [4].   As with all research, care should be taken in this design and it’s application so that further error is not compounded.  Thus, one should adhere to specific procedures when utilizing meta-analysis as the research design.  These steps can be seen below in Table 1.
Table 1 - Procedures in Conducting a Meta-Analysis
___________________________________________

1.  Identification of the problem
2.  Literature search examples

  • computer search
  • journals searched
  • theses and dissertations
3.  Reading each study and coding the characteristics examples
  • internal validity of the study
  • published or unpublished study
  • gender of subjects
  • dependent variable
 4.  Quantifying study findings-effect size calculation
 5.  Statistical analysis of effect size data 
 6.  Interpretation of results
___________________________________________


Identification of the Problem
As with traditional research one must first identify an area of investigation.  However, with meta-analysis it is important that the area in question has been researched to some degree.  This is the same as establishing a specific sample size (n) in traditional research for statistical power.  There is no set number of studies that are needed but one must remember in identifying an area of investigation that the purpose of the meta-analysis is to provide a consensus of past research.  It would be hard to build a case for investigating a subject that had limited research findings. 

Literature Search
An extensive literature search is necessary when conducting a meta-analysis.  Prior to this search the investigators need to establish inclusion and exclusion criteria.  An example of this in traditional designs is of one conducting research utilizing maximal tests excluding those with myocardial problems.  The same is true in terms of the selection of research that will be used in the meta-analysis.  For instance, if one were reviewing the effects of exercise on blood lipid profiles in aging males they might exclude those studies with pre-adolescent females.  This is an important step in the meta-analysis procedure as it provides considerable direction for the literature search and subsequent quantitative values used for analysis.  It should be noted that articles pertaining to the area of interest might still be used within the introduction of the paper as well as in the discussion and not meet inclusion criteria for analysis. 

Reading and Coding Studies
Establishing a coding sheet for research utilizing the meta-analysis design is extremely important.  The coding sheet is analogous to the apparati that are used in the collection of physiological data in traditional research.   A well thought out coding sheet not only provides guidance within the research but also establishes validity and power to the design.  For example: if one were coding for the controversial term “anaerobic threshold” what measures would the researcher seek in past research to quantify a physiological point of disproportionate change?  Careful reading and a knowledge base in exercise physiology would provide insight into this problem.  The term anaerobic threshold might be identified as the point of lactate inflection, ventilatory rate changes or heart rate deviations.  Therefore the coding sheet would allow for the recording of variables associated with the term anaerobic threshold.  In addition to the recording of specific values associated with the problem statement, coding can provide the project with validity in relation to measurement tools and power in relation to sample size.  A carefully planned coding sheet is essential in the meta-analytic procedure.  Done properly, this procedure does not randomly throw together experiments but groups studies carefully to collectively explain an outcome [7].  A partial example of a coding sheet can be found in Appendix I at the end of  this chapter.

Quantifying the Findings: Effect Size Calculations
One of the often-mentioned problems with the meta-analysis design relates to comparing variable findings of differing units of measure.  This has often been referred to as comparing apples and oranges.  Of course, if we were to compare these two variables one might look for a common unit of measure such as categorizing them as fruit or determining there kilocalorie values.  If we refer back to the aforementioned problem of anaerobic threshold one can immediately see that comparing lactate threshold to ventilatory threshold could result in trying to compare millimoles to liters per minute.  A standard unit of measure for continued statistical analysis.  In this instance, these measures are commonly recorded as percentages of maximal or as occurrences taking place at a percent of VO2max.  But what of research that may look at pre and posttests of subjects or compare an experimental group with a control group?  In either case the use of effect size calculations are used to reduce the values to standard deviation units (10).  The effect size calculation can be seen as follows:

 ES= EM – CM/ CSD     eq. 1
Where ES= effect size, EM= experimental mean, CM=control mean, and CSD=control standard deviation.  This can also be written in a design utilizing pre and post measures within a sample as the following:
 ES= PM -  PrM/ PrSD     eq. 2
Where PM= post mean, PrM=pre mean, and PrSD=pre standard deviation.  An analysis of effect sizes can then be determined through traditional statistical procedures or through the interpretation of the effect size.  Thomas & Nelson note that an ES greater that .8 is considered large, an ES at approximately .5 is moderate and an ES of .2 or less is considered small.  With the standard unit of measure established, the argument of “apples and oranges” is addressed.

Statistical Analysis and Interpretation of Results
As with traditional research the meta-analytic research to date utilize a variety of statistical methods.  As mentioned above, with the established standard unit of measure one can then run comparative statistical procedures in the same way as traditional studies.  The interpretation of results is strengthened with this procedure. Mann [7] notes that clear-cut findings often emerge from studies whose previous findings were literally scattered.  He points out that there are very few cases in which a well-conducted meta-analysis had produced erroneous findings.  Because of the quantitative nature of a meta-analytic review, its application of a scientific method allows for a greater objective analysis and less bias than the more traditional review process [1].

META-ANALYSIS IN EXERCISE PHYSIOLOGY
Current and Future Studies
An electronic search when “meta-analysis” was entered discovered 70 research articles in a sports medicine search engine.  If one were to investigate the sub-disciplines of sports medicine they would discover that within their selected field it is an effective design for research.  The procedure has increased in its use in the last few years. For instance, in the aforementioned search there were articles associated with epidemiology, athletic training, orthopedic surgical procedures, biomechanics, sport psychology and exercise physiology.  Why has this procedure increased across disciplines?

One possible answer to that question is that this procedure requires no laboratory or permission from the university internal review board for humans as subjects.  With the many internet search engines one can easily access research for conducting a meta-analysis.  More importantly, its popularity has grown because of the need.   One only need look into the literature to discover that in many areas of research there are conflicting findings.   Williams [12] investigated physical fitness and activity as a separate risk factor for coronary heart disease.  Tran et al. [11] determined the effects of exercise on blood lipids and lipoproteins through a meta-analysis.  Hoffert [1] noted the influence of meta-analysis on clinical research policy.  The use and impact of meta-analysis in all areas of Exercise Science is notably gaining status.

The evidence is equivocal.  Meta-analysis is a powerful research design tool that is necessary in the field of exercise physiology.  Oakes [8] notes that in some traditional research designs, there are overwhelming methodological and logical difficulties.  These difficulties include the following: selection and quality of the trials, population bias and the specificity of the population to which inference is made.  To date, these problems continue to plague traditional research designs. 

Past and current traditional research findings are numerous yet conducted by reducing and separating variables.  With the amount of this separate, reductionist data available, the field is open to multivariable comparisons allowing for a greater understanding of in vivo responses during exercise.
 

References

1.  Hoffert, S.P. (1997).  Meta-analysiis gaining status in science and policymaking. http://www.thescientist.library.upenn.edu. September.
2.  Hopkins, W.G. (2001). A new view of statistics: meta-analysis.  http://www.sportsci.org.
3.  Hyllegard, R., Mood, D. and Morrow, J. R., Jr. (1996).  Interpreting Research in Sport and Exercise Science.  Mosby Publishers, St. Louis, MO.
4.  Krathwohl, D.R. (1993).  Methods of Educational and Social Science Research: An Integrated Approach.  Longman Publishing, New York, NY.
5.  Lokey, E.A., Tran, Z.V., Wells, C.L., Meyers, B.C. and Tran, A.C. (1991).  Effects of physical exercise on prgnancy outcomes: a meta-analytic review.  Medicine and Science in Sports and Exercise, 23(11): 1234-1239.
6.  Lubsen, J. (1996).  Mega-trials: is meta-analysis an alternative? European Journal of Clinical Pharmacology, 49 supplement1: S29-33.
7.  Mann, C. (1990).  Meta-analysis in the breech. Science, 249: 476-480.
8.  Oaks, M. (1993). The logic and role of meta-analysis in clinical research. Statistical Methods in Medical Research, 2(2): 147-160.
9.  Schmidt, F.L. (1992).  What do the data really mean?  American Psychologist, 47 (10): 1173-1181.
10.  Thomas, J.R. and Nelson, J. K. (1996).  Research Methods in Physical Activity, Third Edition.  Human Kinetics Publishers, Champaign, IL 
11. Tran, Z.V., Weltman, A., Glass. G.V. and Mood, D.P. (1983).  The effects of exercise on blood lipids and lipoproteins: a meta-analysis of studies.  Medicine and Science in Sports and Exercise, 15(5):393-402.
12. Williams, P.T. (2001).  Physical fitness and activity as separate heart disease risk factors: a meta-analysis.  Medicine and Science in Sports and Exercise, 33(5):754-761.
13.  Wyatt, F.B. (1999).  Comparison of lactate and ventilatory threshold to maximal oxygen consumption: a meta-analysis.  Journal of Strength and Conditioning Research, 13(1): 67-71.
 


APPENDIX I

CODING SHEET
Cholesterol and Lipid Profiles: Association with Cardiovascular Disease in ESRD

1.  ARTICLE NUMBER:

2.  TITLE OF PAPER:

3.  AUTHOR(S):

4.  JOURNAL/DATE/PAGES:

5. A. PUBLISHED:

a.  refereed journal
b.  nonrefereed journal 
c.  conference proceedings
d.  other:
5. B.  UNPUBLISHED:
a.  Dissertation
b.  Thesis
c.  Unpublished manuscript
Other:
6.  INTERNAL VALIDITY:
Weak
Moderate
Strong
a)  Control Group? Y / N
b)  Random Techniques? Y/ N         if Y, type:
c)  Appropriate Instrumentation? Y / N
d)  Subject retention/dropout limited? Y / N
e)  Subjects blinded to treatment? Y / N
f).  Researcher blinded to treatment? Y / N
g).  Prospective versus Retrospective?Y/N
7.  TIME/DURATION OF EXPERIMENT (days):

8.  BLOOD PROFILE (i.e., cholesterol, LDL, HDL, Triglycerides):

9.  NUMBER OF SUBJECTS:

 a.  Control:
 b.  Experimental:
10.  NUMBER OF COMORBID DISEASES (besides Renal Failure)
(1)  Cancer
(2)  Diabetes Mellitus
(3)  Left Ventricular Hypertrophy
(4)  COPD
(5)  HIV
(6)  Coronary Heart Disease
(7)  Vascular Disease
(8)  Other:____________________________________
11.  EXPERIMENTAL GROUP CHARACTERISTICS:
a.  Gender: 
b.  Ethnicity: 
c.  Mean Age: 
d.  Dialytic Age: 
e.  Age Range: 
f.  Previous Experience with Task: Y / N
g.  Mean Height: 
h.  Mean Weight: 
i.  BMI: 
j.  Smoking:Y/N
k.  Myocardial Infarction: Y/N
l.  Anemia: Y/N   if yes, level of Hb: 
m.  Diabetes: 
n.  Other: 
12.  CONTROL GROUP CHARACTERISTICS:
a.  Gender: 
b.  Ethnicity: 
c.  Mean Age: 
d.  Age Range: 
e.  Mean Height: 
f.  Mean Weight: 
g.  BMI: 
h.  Smoking:Y/N
i.  Myocardial Infarction: Y/N
j.  Anemia: Y/N   if yes, level of Hb: 
k.  Diabetes: 
l.  Other: 
13.  EXPERIMENTAL EXERCISE INFORMATION:
a.  Mode of Exercise: 
b.  Intensity of Exercise: 
c.  Duration of Exercise (per session): 
d.  Frequency of Exercise (per week): 
e.  Other (i.e., stretching, type of resistance used, etc.): 
14.  VARIABLES OF INTEREST:
a.  Administered/Blood:
1)   TOTAL CHOLESTEROL: Y /N
2)   TRIGLYCERIDES: Y / N
3)   LDL: Y/N
HDL: Y/N
VLDL: Y/N
LDL PARTICLE CONCENTRATION: Y/N
LDL PARTICLE SIZE: Y/N

a.  Measures:     Experimental             Control

1) TOTAL CHOLESTEROL: Y /N 
2) TRIGLYCERIDES: Y / N
3)  LDL: Y/N
4) HDL: Y/N
5) VLDL: Y/N
6) LDL PARTICLE CONCENTRATION: Y/N
7) LDL PARTICLE SIZE: Y/N
8)  OTHER: Y/N