Laman utama
[ACSESS Publications] Applied Statistics in Agricultural, Biological, and Environmental Sciences ...
[ACSESS Publications] Applied Statistics in Agricultural, Biological, and Environmental Sciences  Chapter 9: Analysis of Covariance
McCarter, Kevin S.Sukakah Anda buku ini?
Bagaimana kualitas file yang diunduh?
Unduh buku untuk menilai kualitasnya
Bagaimana kualitas file yang diunduh?
Tahun:
2018
Bahasa:
english
DOI:
10.2134/appliedstatistics.2016.0006
File:
PDF, 1,36 MB
Tag Anda:
 Silakan masuk ke akun Anda dulu

Butuh bantuan? Silakan baca instruksi pendek cara mengirim buku ke Kindle
Selama 15 menit file akan dikirim ke email Anda.
Selama 15 menit file akan dikirim ke kindle Anda.
Catatan: Anda perlu memverifikasi setiap buku yang ingin Anda kirim ke Kindle Anda. Periksa email Anda untuk yakin adanya email verifikasi dari Amazon Kindle.
Catatan: Anda perlu memverifikasi setiap buku yang ingin Anda kirim ke Kindle Anda. Periksa email Anda untuk yakin adanya email verifikasi dari Amazon Kindle.
Daftar buku terkait
0 comments
Anda dapat meninggalkan komentar Anda tentang buku dan berbagi pengalaman Anda. Pembaca lain tertarik untuk mengetahui pendapat Anda tentang buku yang telah Anda baca. Terlepas Anda suka atau tidak buku itu, jika Anda menceritakan secara jujur dan mendetil, orang dapat menemukan buku baru buat diri mereka, buku yang sesuai dengan minatnya.
1

2

Published online August 23, 2018 Chapter 9: Analysis of Covariance Kevin S. McCarter* Abstract This chapter presents analysis of covariance from an applied perspective. A descriptive overview of the analysis of covariance model is provided, and its relationship to analysis of variance models and regression models is discussed. A series of examples illuminates certain problems that can occur when analysis of variance, based only on classification treatment effects, is used to analyze experimental data, and how analysis of covariance can be used to alleviate these problems and improve model performance and the quality of analyses by increasing power, improving precision, and reducing bias. From these examples, this chapter develops a general approach for performing analysis of covariance. Analysis of covariance is then used to analyze data from an actual field trial. Introduction This chapter presents the modeling methodology known as analysis of covariance. The chapter begins with an overview of the analysis of covariance model and a discussion of its relationship to analysis of variance models and regression models. The analysis of covariance is a member of the general class of linear models; the chapter identifies the features it shares with these models, and uses these features to provide a description of the model’s general structure as well as to help motivate useful interpretations of it. Methods for model parameter estimation and procedures for hypothesis testing are identified. Then, a sequence of examples demonstrate how analysis of covariance can improve statistical inference and avoid certain problems that can occur when simpler analysis of variance models or regression models are used. From these examples, a general procedure for performing analysis of covariance is identified. In the final example, the analysis of covariance procedure is applied to the analysis of data from an actual experiment. The presentation in this chapter is aimed at researchers who are not statistical experts but who d; o possess knowledge of analysis of variance and regression. For background on analysis 161 Martin D. Woodin Hall, Department of Experimental Statistics, Louisiana State University, Baton Rouge, LA 70803. *Corresponding author (mccarter@lsu.edu) doi:10.2134/appliedstatistics.2016.0006 Applied Statistics in Agricultural, Biological, and Environmental Sciences Barry Glaz and Kathleen M. Yeater, editors © American Society of Agronomy, Crop Science Society of America, and Soil Science Society of America 5585 Guilford Road, Madison, WI 537115801, USA. 235 236 McCa rter of variance, please refer to Chapter 2 (McIntosh, 2018); for background on linear regression, refer to Chapter 6 (Richter and Piepho, 2018); and nonlinear regression is covered in Chapter 15 (Miguez et al., 2018). This chapter expands on that information by focusing on model characteristics and analysis issues that help distinguish analysis of covariance as a distinct methodology. The methods and interpretations presented aim at the analysis of data from designed experiments rather than observational studies. Description of the ANCOVA Model The analysis of covariance (ANCOVA) model is related to analysis of variance (ANOVA) models and regression models, each of which has been presented in other chapters in this book. This chapter builds on the understanding that the reader has obtained from those expositions. Analysis of covariance models possess features of both ANOVA models and regression models. The features of these two types of models will be briefly reviewed to facilitate a discussion of the ANCOVA model. Traditional ANOVA and regression models consist of a continuous response variable, the expected value of which is a function of one or more explanatory variables. In an ANOVA model, the explanatory variables are of the qualitative type, while in a regression model, the explanatory variables are quantitative. Analysis of variance models and regression models are special cases of the more general class of models known as linear models. A linear model is the sum of two modeling components: a deterministic component, which models the expected value, or mean, of the response, and a random component, which provides for the inevitable deviations of the response from its expected value (Graybill, 1976). In this socalled linear model, the relationship between the mean response and the explanatory variables can be quite complex. The explanatory variables may be linearly related to the mean response, but that is not always the case. It is common to have regression models with nonlinear polynomial effects and ANOVA models containing nonlinear, crossproduct interaction terms. The name "linear model" derives from the fact that the deterministic part of the model is a linear function, not of the explanatory variables, but of the model parameters it contains (Graybill, 1976). For additional discussion of this class of models, see Chapter 16 (Stroup, 2018). The ANCOVA model is also a type of linear model; hence, an understanding of the linear model, and of the ANOVA and regression models in particular, provides a useful conceptual framework with which to understand the ANCOVA model. Model parameter estimation and hypothesis testing for the ANCOVA model flow immediately from the general theory of linear models (Graybill, 1976; Milliken and Johnson, 1992). From a practical standpoint, this means that any generalpurpose statistical software capable of performing analysis of variance and regression can be used to perform analysis of covariance. Structurally, the deterministic component of the ANCOVA model combines the characteristics of both the ANOVA model and the regression model. The deterministic component of the ANCOVA model consists of at least one classification variable and at least one quantitative explanatory variable, or regression variable. In the context of ANCOVA, a regression variable is often called a covariate. The ANCOVA A n a ly s i s of Covarian ce model can be complex. As in a pure regression model, the regression part of an ANCOVA model can have polynomial effects and interactions between covariates. The ANOVA part of the model can have more than one classification variable and can include their interactions. In addition, the model can include interactions between classification variables and regression variables. See Chapter 7, Vargas et al. (2018), for an indepth discussion of interactions. This flexibility allows for a wide variety of possible models. As will be seen, how one goes about performing an analysis of covariance depends on the structure of the model. A common goal in many experiments is to compare the effects of two or more treatments on the mean of a measured response variable. Treatments are randomly assigned to experimental units in some fashion. The method of assignment determines the design structure of the experiment, which must be properly accounted for in the statistical model to obtain valid inferences about the treatments. Ideally, experimental units receiving a given treatment would be as homogeneous as possible. In reality, experimental units often exhibit heterogeneity with respect to characteristics that could have an effect on the response variable of interest. If unaccounted for, this heterogeneity inflates the variation associated with error terms. This has the effect of diminishing the power of treatment comparisons and reducing the precision of estimates of treatment effects and their differences. In addition, heterogeneity across treatment groups that is not accounted for can result in biased estimates of treatment effects and their differences. If these underlying heterogeneous characteristics could be measured, the researcher could account for the concomitant variability in the response variable by inclusion of the underlying heterogeneous characteristics in the statistical model. Treatment comparisons would then be more powerful, and estimates of treatment effects and their differences could be made with greater precision and less bias. As in ANOVA and regression settings, ANCOVA can be performed for experiments involving complex design structures. These design structures are typically accounted for by the inclusion of random effects or the specification of nonindependent covariance structures for the error terms (Milliken and Johnson, 1992; Littell et al., 2006). Because of the wide variety of possible design structures, and because the defining characteristic of the class of ANCOVA models is a deterministic component consisting of both classification variables and regression variables, the focus in this chapter will be on presenting ANCOVA in the context of relatively simple design structures. Extension of the ANCOVA model to the more complex design structures that one may encounter practice will be similar to the extension of basic ANOVA and regression models to these more complex situations. Analysis of covariance models extended by the inclusion of random effects or the specification of nonindependent error covariance structures are called linear mixed ANCOVA models. The same issues involved in other types of mixed models apply to linear mixed ANCOVA models. Excellent information regarding these extensions can be found in Milliken and Johnson (2002) and Littell et al. (2006). The ANCOVA model combines features of both the ANOVA model and the regression model. Like an ANOVA model, it contains classification variables that account for the treatment structure of the experiment. This includes treatment main effects and can also include interactions among treatments. Like a regression model, 237 238 McCa rter it contains one or more quantitative explanatory variables, or covariates, and can contain polynomial covariate effects as well as interactions among the covariate terms. One interpretation of such a model is as an ANOVAtype model in which covariates account for heterogeneity among the experimental units. Another interpretation is as a regression model that fits the same relationship between the mean response and the covariates for all treatment groups, but which allows the regression models for different treatment groups to be offset from one another (Littell et al., 2002; Milliken and Johnson, 2002). The deterministic part of an ANCOVA model can also include interactions between the two types of explanatory variables, the classification variables and the covariates. This greatly increases the flexibility of the model. From an ANOVA standpoint, such a model can be thought of as one in which treatment effects depend on the values of the covariates. From a regression standpoint, the model can be thought of as one in which the relationship between the mean response and the covariates is allowed to vary across treatment groups. This model fits nonparallel regression lines and/or surfaces, one for each treatment group (Milliken and Johnson, 2002). The interpretation one chooses may depend on the nature of the experiment and the goals of the research, but quite often both interpretations can be useful for a given experiment. Regardless of the interpretation, inclusion of both classification effects and regression covariates increases the flexibility of the model, which improves the model’s ability to account for variation in the response beyond that which would be possible by using a model containing either classification effects or regression covariates alone. The resulting reduction in unexplained variation in the response increases the power of treatment comparisons and the precision with which treatment effects and their differences are estimated. In this chapter, the use of ANCOVA will be investigated via five examples. The first four examples use simulated data and were designed to illustrate several issues that can arise when performing an ANOVA and how ANCOVA can address these issues and improve the analyses in various ways. To help keep focus on the ANCOVA issues, the same simple experimental design is used across the first four examples. These examples will help readers to organize the various steps that have been utilized into a rough framework for performing ANCOVA in their own experiments. The last example applies the ANCOVA approach that has been developed to actual experimental data involving the comparison of the effect of soybean (Glycine max [L.] Merr.) variety on the growth of fall armyworm (Spodoptera frugiperda [J.E. Smith]) larvae. All statistical analyses will be performed using SAS software. Analysis of Covariance Example 1 Consider an experiment to compare the effects of two treatments, denoted A and B, on the mean of a response variable Y. Twenty experimental units were available for the study. Ten experimental units were randomly assigned to Treatment A and the rest assigned to Treatment B. The experimental design consists of a oneway treatment structure and a completely randomized design structure. A single covariate X was measured on each experimental unit before its assigned treatment was applied. The response variable Y and the covariate X were expected to be positively correlated. The primary objective of the study was to determine whether a difference in 239 A n a ly s i s of Covarian ce the mean response existed between treatments, and if so, to quantify the difference via point and confidence interval estimates. The data for this example are presented in Table 1. In looking at the data, it appears that the values of the response Y tend to be larger under Treatment B than under Treatment A, although the observed difference between the two samples is not large enough to draw a conclusion by visual inspection alone. It also appears that the covariate values tend to be larger for larger values of the response, which is consistent with the stated expectation of a positive correlation between response and covariate. Do the data provide sufficient evidence to allow a firm conclusion to be drawn about whether the treatment means differ? If so, what is the estimate of this difference and how precise is the estimated difference? Can the information provided by the covariate be used to help improve the conclusions? To answer these questions both ANOVA and ANCOVA will be performed and the results compared. Prior to performing formal statistical analysis, it is a good idea to perform exploratory analysis of the data using graphical and numerical summaries to look for general trends and to check for violations of expected relationships and potential problems in the data. This is especially true when performing analysis of covariance. The SAS statements to enter the data into a SAS dataset are given in Fig. 1. Figure 2 gives the SAS statements to produce univariate summary statistics and boxplots of the response variable and the covariate for each treatment, as well as a scatter plot of the response versus the covariate for the two treatments, superimposed on the same plot. Sidebyside boxplots of the response variable Y are shown in Fig. 3. From these boxplots it can be seen that the responses under Treatment B tend to be higher than under Treatment A. In particular, the sample mean response under Treatment B is larger than the sample mean response under Treatment A. The variation of the two samples appears to be similar, with Sample B showing only slightly less variation than sample A. Table 1. Dataset for Example 1. Treatment A Y X 71.3 12.7 63.4 13.3 55.0 8.6 54.0 7.3 Y X 63.0 7.6 80.9 13.4 78.7 10.3 85.1 14.8 54.6 8.2 47.7 6.4 49.1 7.0 88.1 14.2 59.4 8.6 70.5 10.7 73.0 13.5 53.0 5.3 76.3 9.9 68.7 9.4 84.9 14.2 Treatment B data example_1; input grp $ y x @@; datalines; A 71.3 12.7 A 63.4 13.3 A 47.7 6.4 A 49.1 7.0 B 63.0 7.6 B 80.9 13.4 B 73.0 13.5 B 53.0 5.3 ; run; A A B B 78.5 13.6 55.0 8.6 A 54.0 7.3 A 88.1 14.2 A 59.4 8.6 A 78.7 10.3 B 85.1 14.8 B 76.3 9.9 B 68.7 9.4 B Fig. 1. SAS statements to create the SAS dataset for Example 1. 54.6 8.2 70.5 10.7 78.5 13.6 84.9 14.2 240 McCa rter title “Summary of the Dataset”; proc means data=example_1 mean std; var y x; by grp; run; title “Boxplots of Response Variable Y for Each Group”; proc sgplot data=example_1; vbox y / group=grp extreme ; run; title “Boxplots of Covariate X for Each Group”; proc sgplot data=example_1; hbox x / group=grp extreme ; run; title “Scatter Plot of Response Y vs Covariate X”; proc sgplot data=example_1; scatter y=y x=x / group=grp; run; Fig. 2. SAS statements to compute summary statistics, boxplots, and scatter plot for Example 1. Fig. 3. Boxplots of the response Y for Example 1. A n a ly s i s of Covarian ce Boxplots for the covariate are given in Fig. 4. Based on these boxplots, the two samples appear to be similar with respect to the distribution of the covariate. The covariate mean of the Treatment B assignment group is slightly larger than that of the Treatment A assignment group. The level of variation of the covariate appears to be similar in the two assigned groups. Summary statistics for both the response and the covariate are given in Table 2. The sample mean response under Treatment B is greater than the sample mean response under Treatment A by 12.9 units. As measured by the standard deviation, the variation in the response is similar under the two treatments. With a sample size of ten experimental units in each treatment group, the estimated standard error of the difference between sample means would be approximately five units, so the observed difference looks to be statistically significant, although perhaps not highly significant. The sample means and the sample standard deviations of the covariate are similar in the two samples. This suggests that the two treatment groups are similar with respect to characteristics which could affect the response variable, as measured by the covariate. As stated previously, the primary objective is to determine whether a treatment difference exists, and if so, to quantify that difference. To do this, both an ANOVA as well as an ANCOVA of the response Y will be performed. The results from these analyses will then be used to compare the effectiveness of the two approaches. In addition, an ANOVA on the covariate X will be performed to formally test whether the covariate means differ across the two treatment groups. The SAS statements to perform these analyses are provided in Fig. 5. The results of the ANOVA of Y are given in Table 3. The observed significance level of the test is p = 0.0203. The observed difference between treatment means is Fig. 4. Boxplots of the covariate X for Example 1. 241 242 McCa rter Table 2. Summary statistics for treatments A and B for Example 1. Treatment A B Variable Mean Standard Deviation Y 61.3 12.4 X 9.7 2.8 Y 74.2 10.1 X 11.2 3.2 title “ANOVA To Compare Mean of Y Across Groups”; proc mixed data=example_1; class grp; model y = grp ; lsmeans grp / pdiff cl ; run; title “ANCOVA To Compare Mean of Y Across Groups, Adjusting for X”; proc mixed data=example_1; class grp; model y = x grp ; lsmeans grp / pdiff cl ; run; title “ANOVA To Compare Mean of Covariate X Across Groups”; proc mixed data=example_1; class grp; model x = grp ; run; Fig. 5. SAS statements to perform ANOVA and ANCOVA for Example 1. Table 3. ANOVA results for Example 1. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model Error Corrected Total 1 18 19 832.050 2310.478 3142.528 832.050 128.360 6.48 0.0203 Source DF Type III SS Mean Square F Value P>F GRP 1 832.050 832.050 6.48 0.0203 Type III Tests Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 12.90 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 2.26 23.54 243 A n a ly s i s of Covarian ce significant, but not highly significant. Using a 5% significance level, the conclusion is that a treatment difference exists. The point estimate of the difference between the mean response under Treatment B and the mean response under Treatment A is 12.90 units. The 95% confidence interval estimate of the difference is [2.26, 23.54]. Because the interval does not contain zero, it could be used by itself to conclude that a difference exists between treatment means at the 5% level. Note that the width of this confidence interval is 21.28 units. Finally, note that from the ANOVA table, the point estimate of the variance of the error distribution is 128.360 units squared. The magnitude of the error variance is important to consider, as it affects the overall significance of the model and the precision of confidence intervals. The error variance is a measure of the uncertainty with which the response is measured. If the model does not capture variation in the response, it contributes to the estimate of the error variance. To the extent that we can account for variation in a response variable with a model, we can increase the power of hypothesis tests and increase the precision of confidence interval estimates. Therefore, if we can add a variable to the model that accounts for significant additional variation in the response, then we should be able to increase the power and precision of our analysis. That is the reason for adding a covariate to the model and forming an ANCOVA model. The results of the ANCOVA are given in Table 4. The only structural difference between this ANCOVA model and the ANOVA model described above is the inclusion of the covariate X in the model. Notice first what has happened to the error variance compared with that in the first model. The estimated error variance has been reduced from 128.360 to 29.258. Thus, the inclusion of the covariate X has resulted in a reduction in the estimated error variance of over 75%. Such a reduction has a very positive inferential impact on the analysis, increasing power as well as precision. The pvalue for the overall model is now < 0.0001. The pvalue of the covariate is < 0.0001, indicating that the linear relationship between the response and the covariate is highly significant. Having accounted for the variation in the response associated with the covariate, the pvalue for the treatment effect has been reduced from 0.0203 to 0.0057. This is a 72% reduction in the treatment effect pvalue compared with that obtained from the ANOVA model. The result of this improvement is that the treatment effect is much more significant now that the covariate has Table 4. ANCOVA results for Example 1. ANOVA Table Source DF Model Error Corrected Total 2 17 19 Source DF X GRP 1 1 Sum of Squares 2645.139 497.389 3142.528 Mean Square F Value P>F 1322.569 29.258 45.20 < 0.0001 Mean Square F Value P>F 1813.089 291.926 61.97 9.98 < 0.0001 0.0057 Type III Tests Type III SS 1813.089 291.926 Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 7.90 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 2.62 13.18 244 McCa rter Fig. 6. Scatter plot of the response variable and covariate, with the fitted ANCOVA model for Example 1. been added to the model. The estimate of the difference between the mean response under Treatment B and the mean response under Treatment A is 7.90 units. The 95% confidence interval estimate of the difference is [2.62, 13.18]. The important point to note here is that the width of this interval is 10.56 units, which is much shorter than the interval produced by the ANOVA model. Adding the covariate to the model has greatly improved precision by reducing the width of the confidence interval estimate of the treatment difference by 50%. To gain insight into why this ANCOVA model provides greater power and better precision than the ANOVA model, it is helpful to compare the boxplots of the response variable in Fig. 3 to the plot in Fig. 6. The boxplots can be loosely thought of as a graphical representation of what the ANOVA model is fitting. In particular, the estimated difference in treatment means obtained from the ANOVA model corresponds to the difference in the location of the mean symbols in those boxplots. The explanatory variable in the ANOVA model is the classification variable identifying treatment group. The ANOVA model sees only the vertical axis. It is therefore capable of fitting only the means of the two treatment groups (or equivalently, the differences of the treatment means from the overall mean). Figure 6 gives a scatter plot of the response Y versus the covariate X for both treatment groups, along with parallel regression lines running through the points from each treatment group. This plot is a graphical representation of the model that is fitted by the ANCOVA model. The explanatory variables of the ANCOVA model include the covariate as a continuous variable along with the treatment classification variable. The continuous covariate accounts for the variation in the response that is due to the linear relationship between the response variable and the covariate, while the treatment classification variable accounts for the overall difference in means 245 A n a ly s i s of Covarian ce between the two treatments. This graphic corresponds to the conceptualization of the ANCOVA model as the simultaneous fitting of separate parallel regression lines for each treatment group described previously. The estimated difference between treatments, estimated to be 7.90 units, corresponds to the vertical difference between the fitted parallel regression lines. Unaccountedfor variability corresponds to the squared deviations of the points within each treatment group from that treatment group’s regression line, rather than from the treatment group’s overall mean. The added perspective provided by the horizontal covariate axis makes it much easier to visually detect the difference in treatment means relative to the unaccountedfor variation than is possible with the boxplots, and the ANCOVA model sees the same sharper distinction. The result is that the ANCOVA model provides greater power to detect the treatment difference and greater precision in estimating that difference than does the ANOVA model. Table 5 gives the results of an analysis of variance to compare the means of the covariate values across the two treatment assignment groups. The pvalue of the test is 0.2799, and so there is insufficient evidence to conclude that a difference exists in the covariate mean across treatment groups. Hence, while the covariate values vary within each sample, overall the distribution of covariate values are similar in the two samples. We conclude that while there is heterogeneity within treatment assignment groups, experimental unit characteristics do not differ significantly across the groups. Let’s think conceptually about how the ANCOVA approach works. Variation in the response comes from two main sources: variation due to application of the treatments, and variation due to heterogeneity in characteristics of the experimental units that can affect the response. In the ANOVA model, variation due to the treatments is accounted for by the treatment classification variable. The rest of the variation is unaccounted for, and therefore ends up contributing to the variability of the error term. In a given situation, there may be a number of characteristics that vary from one experimental unit to another and that have an effect on the response. If all of these characteristics could be measured prior to treatment application, then they could be used to account for variation in the response that they induce. It is probably impossible to identify, let alone measure, all such characteristics. We do not need to do that, however. Suppose we can identify and measure a covariate that is highly correlated with the response variable. Then the variability in the response due to heterogeneity in the experimental units could be assessed indirectly by measuring the covariate before application of the treatment. Since the response and covariate are strongly correlated, the relationship between the two variables can be used to measure the underlying variation in the response prior to application of the treatment and then account for this variation in the model, preventing it from contributing to the estimate of the error variance. In this way, heterogeneity in the experimental material that causes variation in the response is extracted and accounted for, leading to more accurate and precise Table 5. ANOVA comparing mean covariate values across treatments for Example 1. ANOVA Table Source DF Model Error Corrected total 1 18 19 Sum of squares 11.25 163.18 174.43 Mean square F value P>F 11.25 9.06 1.24 0.2799 246 McCa rter estimates of treatment effects and more powerful hypothesis tests. The resolving power of the model is therefore substantially enhanced by the inclusion of a quality covariate, one that is highly correlated with the response variable and is measured prior to application of the treatment. Why must the covariate be measured prior to application of the treatment? The reason is that we want the covariate to extract only that variability in the response that is due to heterogeneity in the experimental units prior to application of the treatment. We do not want the covariate to extract variation that is the result of application of the treatment; treatment effects are included in the model for that purpose. If a covariate could be affected by the treatment, and if it were measured after the treatment had been applied, then the variation in the covariate would come from both the pretreatment experimental unit heterogeneity and also the effect of the treatment. Its inclusion in the model would extract the treatment effect that we are trying to detect and isolate with the classification effects in the model. The measured treatment effect would be attenuated to the degree that the covariate is affected by the treatment, which would decrease power to detect differences and would also bias estimates of treatment effects. To prevent this, the covariate must be such that it cannot be affected by the treatment being applied, or it must be measured prior to application of the treatment. The safest approach for obtaining a covariate is to always measure the covariate before treatments are applied. This need for independence of the covariate and the treatment is the reason why the ANCOVA procedures being presented in this chapter are better suited for data from designed experiments than for data from observational studies, as was indicated in the introduction to this chapter. In designed experiments, treatments are randomly assigned to experimental units, and so if covariates are measured before application of treatments, independence of treatment and covariate is guaranteed. On the other hand, in observational studies, rather than being randomly assigned, measured characteristics are typically inherent characteristics of the experimental units themselves and are often correlated. Hence, measured characteristics considered treatments and those utilized as covariates cannot be guaranteed to be independent. Results of ANCOVA from such data can be difficult to interpret correctly. The misuse of covariates in such cases have apparently led many journals in the behavioral sciences to not allow their use when treatments are not randomly assigned (Freund et al., 2010). Lessons Learned from Example 1 The deterministic part of the ANOVA model includes terms for the treatment effects only, and therefore possesses no way to account for heterogeneity in the experimental material. As a result, any variability due to heterogeneity in the experimental units is left unaccounted for by the deterministic part of the model and ends up in the error terms. This has the effect of inflating the estimated error variance, which diminishes both power and precision. On the other hand, the ANCOVA model includes a covariate as an explanatory variable and therefore is able to account for variability in the response that is associated with heterogeneity in the experimental material. Unaccountedfor variation in the response is reduced, which in turn reduces the estimated error variance. The result is an increase in the power of tests 247 A n a ly s i s of Covarian ce (Chapter 4, Casler, 2018) as well as an increase in the precision of estimates. This increase in power results in a reduction in the Type 2 error rate for any given significance level used (Chapter 1, GarlandCampbell, 2018). In this example, based on the ANOVA, the treatment effect was significant, although not highly so. Using the ANCOVA model the treatment effect became highly significant, with the pvalue for this effect being 72% smaller that the corresponding pvalue from the ANOVA. In addition, the ANCOVA model provided a more precise estimate of the treatment difference, the 95% confidence interval being 50% shorter than the interval obtained from the ANOVA model. Note that because a 5% significance level was used, both models were significant. However, if a 1% significance level had been used, for example, then the treatment effects would still be significant in the ANCOVA model, but the treatment effect would not be significant in the ANOVA model. From the standpoint of both power and precision, including the covariate in the model had a positive inferential impact on the statistical analysis of the data. Analysis of Covariance Example 2 This example uses the same experimental setup and goals as in Example 1. An experiment is conducted to compare the effects of two treatments, denoted A and B, on the mean of a response variable Y. Twenty experimental units were available for the study, ten randomly assigned to each of the two treatments. A single covariate X was measured on each experimental unit before treatments were applied. The response Y and the covariate X are expected to be correlated. The data for this example are presented in Table 6. Perusing the data it is evident that the response Y tends to be larger under Treatment B than under Treatment A. In addition, the values of the covariate appear to be larger for larger values of the response. Also, the values of the covariate are larger under Treatment B than under Treatment A. These observations are consistent with the expectation of a correlation between the response and covariate. The SAS statements to create a dataset containing these data, produce graphical and numerical summaries, and perform ANOVA and ANCOVA analyses are similar to those in Example 1, the only difference being the substitution of the data values for this example. Sidebyside boxplots of the response variable Y for the two treatments are shown in Fig. 7. The boxplots show a large difference between the means of the two samples relative to the variation in the samples, with the mean response much higher under Treatment B than under Treatment A. Note that there is little overlap between the two samples; in fact, there is almost complete separation. A difference between response means is clear in these boxplots. The variation in the two samples appears to be similar. Table 6. Dataset for Example 2. Treatment A Y 69.2 59.4 70.2 52.3 61.0 73.9 57.1 64.9 68.2 75.1 X 10.8 10.7 13.1 6.6 9.6 13.4 7.3 10.0 13.8 14.8 Treatment B Y 89.9 101.3 73.2 96.4 86.4 74.8 81.2 97.3 99.4 79.3 X 17.4 21.2 13.4 20.1 17.6 14.8 17.2 20.2 21.9 13.9 248 McCa rter Fig. 7. Boxplots of the response Y for Example 2. Table 7. ANOVA results for Example 2. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model 1 2596.921 2596.921 31.32 < 0.0001 Error Corrected Total 18 19 1492.257 4089.178 82.903 Source DF Type III SS Mean Square F Value P>F GRP 1 2596.921 2596.921 31.32 < 0.0001 Type III Tests Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 22.79 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 14.24 31.34 Results of an ANOVA to compare the response means under the two treatments are given in Table 7. The pvalue for comparison is < 0.0001, providing very strong evidence of a difference in the response means under the two treatments. The point estimate of the difference between the mean response of Treatment B and the mean response of Treatment A is 22.79 units. The 95% confidence interval estimate of the difference is [14.24, 31.34], the length of which is 17.10 units. Note also that the estimate of the error variance is 82.903 (units squared). In Example 1, we saw that including the covariate in the model increased power and precision, and a significant treatment effect as assessed by an ANOVA became even more significant when the covariate was added to the model forming an ANCOVA model. Can we expect the same results in this example? Results 249 A n a ly s i s of Covarian ce of an ANCOVA for these data are given in Table 8. Note first that the error variance estimated by the ANCOVA model is 10.480. This is an 87% reduction in the estimated error variance of 82.903 from the ANOVA model. As in Example 1, inclusion of the covariate in the model has substantially reduced the amount of unexplained variation in the response. This will result in an increase in the power of statistical comparisons and also in an improvement in the precision of estimates. The pvalue in the ANCOVA table tests the overall null hypothesis that none of the explanatory variables in the model are significant. Specifically, it is testing that no linear relation exists between the response and the covariate, and that there is no treatment effect. The pvalue of this test is < 0.0001. Hence, there is very strong evidence that this null hypothesis is false. We conclude that either a linear relationship exists between the response and the covariate, that a treatment effect exists, or both. This is expected based on a visual inspection of the raw data and the boxplots, which together suggested both a linear relationship between response and covariate and a pronounced difference in response means across treatment groups. The section on Type III tests gives results for each of these two hypotheses. The pvalue for the hypothesis of no linear relationship between response and covariate is < 0.0001, so there is very strong evidence that a linear relationship exists. On the other hand, the pvalue for the test of no treatment effect is 0.2191. Based on this test, there is insufficient evidence to conclude that a treatment effect exists. The point estimate of the difference between the response mean of Treatment B and the response mean of Treatment A is 2.92 units. The 95% confidence interval estimate of the difference is [1.91, 7.75]. Since the interval includes zero, it could be used instead of the test above to conclude that the observed difference in mean response between treatment groups is not statistically significant at the 5% significance level. Note that the width of this interval is 9.66 units. Compared with the width of the interval based on the ANOVA model of 17.10 units, this is a 44% reduction in width. These results show that inclusion of the covariate in the model has once again improved precision as well as the power of tests. Table 8. ANCOVA results for Example 2. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model 2 3911.053 1955.527 186.63 < 0.0001 Error 17 178.125 10.480 Corrected Total 19 4089.178 Type III Tests Source DF Type III SS Mean Square F Value P>F X 1 1314.132 1314.132 125.42 < 0.0001 GRP 1 17.061 17.061 1.63 0.2191 Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 2.92 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 1.91 7.75 250 McCa rter Fig. 8. Boxplots of the covariate X for Example 2. The boxplots of the response and the ANOVA indicated strong evidence of a difference in response means across the two treatment groups; but according to this ANCOVA model, there is insufficient evidence to conclude that a treatment effect exists. Shouldn’t the ANCOVA provide more power in making this comparison? A look at how the covariate X is distributed in each assigned treatment group will shed some light on what is going on. Figure 8 gives boxplots of the covariate for each treatment group. From the boxplots, it is clear that the covariate tends to be larger for experimental units assigned to Treatment B than for experimental units assigned to Treatment A. An ANOVA comparison of the covariate means for the two samples is given in Table 9. Based on this analysis, there is strong evidence that the covariate means are different in the two groups. Note that because the covariate was measured prior to application of the treatment, any differences in the distribution of the covariate in the two samples would be the result of the randomization process. Additional insight can be obtained by looking at scatter plots of the response variable and covariate for the two treatment groups in Fig. 9. The ANOVA models the mean of the response Y (along the vertical axis) as a function of the treatment group only, ignoring the covariate X (along the horizontal axis). Projecting the points onto the vertical axis provides the same perspective seen in the boxplots in Fig. 7. It is clear why the ANOVA sees a difference in the mean response for the two groups, for indeed the values of the response variable are larger under Treatment B than under Treatment A. Projecting the points onto the horizontal axis provides the same perspective seen in the covariate boxplots in Fig. 8. Again, it is clear from this scatter plot that the covariate values are greater in the sample assigned to Treatment B than 251 A n a ly s i s of Covarian ce in the sample assigned to Treatment A. Finally, from the scatter plot as a whole, it is clear that a strong linear relationship exists between the response and the covariate. This is why the linear effect was highly significant in the ANCOVA model. So why is the treatment effect not significant in the ANCOVA model? Figure 10 shows the same scatter plot of the response variable and covariate superimposed with the model fit by the ANCOVA. In the ANOVA model, the estimated treatment difference corresponds to the difference between the means of the values projected onto the vertical axis, whereas in the ANCOVA model the estimated treatment difference corresponds to the vertical distance between the fitted regression lines, as indicated in the discussion of this type of plot in Example 1. Hence, it is clear why the treatment difference is not significant in the ANCOVA model. There is relatively little vertical separation between the regression lines fitted by the ANCOVA model. The significant difference in treatment means detected by the ANOVA model was driven mainly by heterogeneity in characteristics of the experimental units that affected the response, which registered as a difference in both the distribution of the response and the distribution of the covariate values across the two samples. This heterogeneity of experimental units across treatment groups Table 9. ANOVA comparing mean covariate values across treatments for Example 2. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model 1 228.488 228.488 27.03 < 0.0001 8.452 Error 18 152.130 Corrected Total 19 380.618 Fig. 9. Scatter plot of the response variable and covariate for Example 2. 252 McCa rter Fig. 10. Scatter plot of the response variable and covariate, with the fitted ANCOVA model for Example 2. biases the estimate of the difference in treatment means by an amount that is proportional to the difference in covariate means. In addition to increasing power and precision, in this case addition of the covariate to the model has the added benefit of reducing, if not eliminating, this bias. Once this variation has been accounted for by the ANCOVA model, there is little variation remaining that is due to the treatments. Lessons Learned from Example 2 In this example, the ANOVA model detected a highly significant difference between the treatment group means. However, this difference was not the result of the actual treatment effects, but instead was driven by differences between samples with respect to characteristics of the experimental units that affected the response variable. Because the covariate is correlated with the response, these differences were registered in the covariate, but because the ANOVA model does not take the covariate into account, it is not able to utilize this information. The estimate of the treatment difference was therefore biased, the amount of bias related to the difference in covariate means. On the other hand, because the ANCOVA model does incorporate the covariate, it is able to differentiate variation in the response due to heterogeneity in the underlying characteristics of the experimental units and variation due to the treatment. In other words, it removed the bias from the estimate of the treatment effects. The ANCOVA model therefore provides a more accurate assessment of the significance of the observed difference, as well as a more accurate estimate of the treatment difference. In addition to removing the bias, including the covariate also improved the power and precision of the analysis. This is evidenced by the 87% reduction in the estimated error variance and the 44% reduction in the 253 A n a ly s i s of Covarian ce width of the confidence interval estimate of the treatment difference compared with those obtained from the ANOVA model. Analysis of Covariance Example 3 This example uses the same experimental setup and goals as in the first two examples. An experiment is conducted to compare the effects of two treatments, denoted A and B, on the mean of a response variable Y. Twenty experimental units were available for the study, ten randomly assigned to each of the two treatments. A single covariate X was measured on each experimental unit before treatments were applied. The response Y and the covariate X are expected to be correlated. The data for this example are presented in Table 10. The statements to create a SAS dataset, produce graphical and numerical summaries, and perform ANOVA and ANCOVA are similar to those given in Example 1. As we inspect the data visually, it does not appear that the treatment groups differ substantially with respect to the distribution of the response variable Y. This is confirmed by the sidebyside boxplots in Fig. 11, where the means look to be almost identical and the levels of variation are similar. Summary statistics for the response variable Y are provided in Table 11. From these values we see that the means and standard deviations are nearly identical in the two groups. Based on these summaries there appears to be no evidence of a difference in the two treatment groups. To quantify this assessment with a formal test, Table 12 gives the results of an ANOVA to compare the mean responses across treatment groups. The pvalue of 0.9802 confirms that based on the measured response variable Y alone, there is no evidence of a difference in means across treatments. The point estimate of the difference between the mean responses under Treatment B and Treatment A is 0.12 units. The 95% confidence interval estimate of the difference is [9.88, 10.12] units. Since the confidence interval contains zero, it can be used to draw the conclusion that no difference exists between treatment means. The width of this confidence interval is 20.00 units. In Example 2, use of the ANOVA and ANCOVA models resulted in two qualitatively different conclusions. This difference in conclusions was driven by a difference in the distribution of the covariate across the two samples, which was unaccounted for by the ANOVA model but accounted for by the ANCOVA model. Before drawing final conclusions in this example, we will look at the distribution of the covariate data. We will then perform an ANCOVA and compare results with those of the ANOVA model. Table 10. Dataset for Example 3. Treatment A Y 81.2 58.7 47.4 49.4 66.1 72.5 71.1 53.5 62.2 68.5 X 14.1 8.8 5.7 5.5 9.1 14.6 12.7 6.2 8.0 12.4 Treatment B Y 56.6 57.5 75.6 68.5 58.0 57.7 62.6 73.9 77.0 44.4 X 15.9 17.9 21.6 19.9 14.1 15.1 16.8 20.6 20.9 13.0 254 McCa rter Fig. 11. Boxplots of the response Y for Example 3. Table 11. Summary statistics for the response variable Y for Example 3. Treatment Mean Standard deviation A 63.06 10.88 B 63.18 10.40 Table 12. ANOVA results for Example 3. ANOVA Table Source DF Sum of squares Mean square F value P>F 0.0006 0.9802 Model 1 0.072 0.072 Error 18 2038.540 113.252 Corrected Total 19 2038.612 Type III Tests Source DF Type III SS Mean square F value P>F GRP 1 0.072 0.072 0.0006 0.9802 Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 0.12 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 9.88 10.12 255 A n a ly s i s of Covarian ce Figure 12 shows boxplots of the covariate in the two samples. The covariate sample means and medians for the two samples are very far apart relative to the variation in the two samples. In fact, the interquartile intervals do not overlap. The variation in the two samples appears to be about the same, as measured by both the range and interquartile range. Table 13 gives summary statistics for the covariate for the two samples. The observed difference between sample means is 7.87 units. The standard deviations are close. Table 14 shows the results of an ANOVA to compare the covariate means across assigned treatment groups. The pvalue is < 0.0001, so the observed difference is highly significant. The covariate distributions clearly differ with respect to location in the two groups. Since the ANOVA does not take into account the variation in the response due to the covariate, the conclusion drawn from the ANOVA is highly suspect. As was seen in Example 2, an unaccountedfor difference in covariate distributions can result in biased estimates of treatment effects. Hence, we will perform an ANCOVA to account for the covariate. Results of the ANCOVA are shown in Table 15. Note first that the estimate of the error variance is 14.486, which is 87% smaller than the estimate of 113.252 for the error variance from the ANOVA model. The pvalue from the overall model is < 0.0001; the model is highly significant. The conclusion is that either the response and Fig. 12. Boxplots of the covariate X for Example 3. Table 13. Summary statistics for the covariate X for Example 3. Treatment Mean Standard Deviation A 9.71 3.49 B 17.58 3.06 256 McCa rter Table 14. ANOVA comparing mean covariate values across treatments for Example 3. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model 1 309.685 309.685 28.72 < 0.0001 10.781 Error 18 194.065 Corrected Total 19 503.750 Table 15. ANCOVA results for Example 3. ANOVA Table Source DF Sum of squares Mean square F Value P>F Model 2 1792.342 896.171 61.86 < 0.0001 14.486 F Value P>F Error 17 246.270 Corrected Total 19 2038.612 Source DF Type III SS Mean square X 1 1792.270 1792.270 123.72 < 0.0001 GRP 1 1090.785 1090.785 75.30 < 0.0001 Type III Tests Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 23.80 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 29.58 18.01 covariate are linearly related, or a treatment effect exists, or both. Results of the individual tests for these effects are given in the section on Type III tests. The test of the null hypothesis that no linear relationship exists between the response and covariate, given that the treatment effect is in the model, has a pvalue of < 0.0001, so there is strong evidence of a linear relationship between the response and the covariate. The test of no treatment effect, given that the linear effect is in the model, also has a pvalue < 0.0001, so there is strong evidence of a treatment effect as well. This is perhaps surprising, since the sample means were so close, differing by only 0.12 units, and since the ANOVA comparison of the treatment groups was so nonsignificant. Note that the point estimate of the difference between the mean responses under Treatment B and Treatment A is 23.80 units, which is a much larger difference in magnitude than that estimated by the ANOVA model. The 95% confidence interval estimate of the difference is [29.58, 18.01]. The width of this interval is 11.57 units, which is 42% smaller than the width estimated using results of the ANOVA model. It is clear that the ANCOVA model provides results that differ substantially from those provided by the ANOVA model. Figure 13 gives a graphical representation of the fitted ANCOVA model superimposed over a scatter plot of the data. The vertical distance between the regression lines is equal to the estimated treatment difference. With this graph, it is easy to visually distinguish between the two regression lines; this is why the treatment difference is so significant in the ANCOVA model. On the other hand, the ANOVA ignores the covariate. Graphically, this is equivalent to ignoring the horizontal axis and projecting all of the points onto the vertical axis. When this is done, the A n a ly s i s of Covarian ce observed treatment difference becomes very small and insignificant relative to the variability in the responses along the vertical axis. This illustrates clearly why the ANCOVA has so much more power than the ANOVA to detect the treatment difference in this type of situation. Lessons Learned from Example 3 In this example, the treatment comparison was very nonsignificant based on the ANOVA model, but became highly significant when the ANCOVA model was used. This lack of significance of the ANOVA was the result of unaccountedfor heterogeneity in experimental units across samples with respect to characteristics that affect the response. This unaccountedfor heterogeneity induces extra variation in the response that was unaccounted for by the ANOVA model. This unaccountedfor variability in the response introduced bias in the estimated treatment effects and their difference. In this particular case, this bias substantially offset the actual treatment difference. The unaccountedfor variability in the response also resulted in an inflation of the estimated error variance, which decreased the power of the test and the precision of the confidence interval estimate. Because the response variable and the covariate are correlated, the extra variation in the response variable that is induced by heterogeneity in the experimental units is also registered in the covariate. The ANCOVA model accounted for this heterogeneity in the experimental units indirectly by including the covariate in the model. Inclusion of the covariate reduced this heterogeneityinduced bias, reducing the estimated error variance by 87%, and reducing the length of the confidence interval estimate of the treatment difference by 42%. The result is a model that provides Fig. 13. Scatter plot of the response variable and covariate, with the fitted ANCOVA model for Example 3. 257 258 McCa rter Table 16. Dataset for Example 4. Treatment A Y 55.1 67.1 73.6 64.6 76.4 45.5 47.3 57.4 78.5 61.9 X 8.2 11.0 14.8 9.3 13.8 5.3 6.4 6.6 12.9 8.4 Y 61.7 56.3 54.2 68.0 58.5 59.2 60.8 68.2 52.7 78.5 X 9.5 11.3 12.0 9.8 12.6 10.5 10.7 9.8 13.1 6.4 Treatment B Fig. 14. Boxplots of the response Y for Example 4. more power in detecting the treatment difference and a more accurate and precise estimate of that difference. Analysis of Covariance Example 4 This example uses the same experimental setup and goals as the previous examples. An experiment is conducted to compare the effects of two treatments, denoted A and B, on the mean of a response variable Y. Twenty experimental units were available for the study, ten randomly assigned to each of the two treatments. A single covariate X was measured on each experimental unit before treatments were applied. The response Y and the covariate X are expected to be correlated. The data for this example are presented in Table 16. Boxplots of the response variable under the two treatments are given in Fig. 14. From the boxplots we see that the observed response means are nearly equal. The range and interquartile range are both somewhat smaller under Treatment B than under Treatment A. 259 A n a ly s i s of Covarian ce Summary statistics by treatment group for the response variable are given in Table 17. As can be seen, the sample means are close, and the standard deviation of the response is slightly smaller under Treatment B than under Treatment A. The pvalue of the test of equal variances is p = 0.2588, so there is insufficient evidence to conclude that variation in the response differs under the two treatments. Note that the pvalue for this comparison of variances is not produced as part of the ANOVA, but rather was obtained by hand using the Ftest for comparing two population variances that is found in introductory statistical textbooks. Results of an ANOVA to compare the response means across treatment groups are given in Table 18. The pvalue of the comparison is 0.8352, so there is insufficient evidence to conclude that a difference exists between response means. The point estimate of the difference between Treatment B and Treatment A is 0.93 units. The 95% confidence interval estimate of the difference is [10.18, 8.32] units. The confidence interval contains zero, which also implies that there is insufficient evidence to conclude that a difference in response means exists at the 5% significance level. The width of the confidence interval is 18.50 units. Examples 2 and 3 demonstrated that ANOVA can lead to wrong conclusions if differences exist in the covariate distribution across assigned treatment groups, and how ANCOVA can account for such differences and improve decision making. It is important to know if such differences exist, so we will now look at the distribution of the covariate. Boxplots of the covariate are given in Fig. 15. From the boxplots we see that the covariate means are similar. The range of covariate values is similar as well, although the interquartile range is quite a bit smaller in the group assigned to Treatment B. Summary statistics for the covariate are given in Table 19. As can be seen, the covariate means and standard deviations are numerically similar across the two Table 17. Summary statistics for the response variable Y for Example 4. Treatment Mean Standard deviation A 62.74 11.54 B 61.81 7.80 Table 18. ANOVA results for Example 4. ANOVA Table Source DF Model 1 Sum of Squares Mean Square F Value P>F 4.325 4.325 0.04 0.8352 97.020 Error 18 1746.363 Corrected Total 19 1750.678 Source DF Type III SS Mean Square F Value P>F 1 4.325 4.325 0.04 0.8352 Type III Tests GRP Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 0.93 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 10.18 8.32 260 McCa rter Fig. 15. Boxplots of the covariate X for Example 4. Table 19. Summary statistics for the covariate X for Example 4. Treatment Mean Standard deviation A 9.67 3.31 B 10.57 1.91 assigned treatment groups. An Ftest of the hypothesis of equal variances resulted in a pvalue of 0.1170. Results of an ANOVA to compare the covariate means across assigned treatment groups is given in Table 20. The pvalue is 0.4662, and hence there is insufficient evidence to conclude that the assigned treatment groups are different with respect to the covariate means. Even though the covariate means do not appear to differ, as we have seen in each of the previous examples including a quality covariate in the analysis can increase power and precision by accounting for variation in the response that is due to heterogeneity in the experimental units. Results of an ANCOVA to compare the response means across treatments are given in Table 21. The pvalue for the overall ANCOVA model is 0.1965. Interestingly, the overall ANCOVA model is not significant. This result is perhaps a bit surprising. While we may have expected the treatment effect to be nonsignificant based on the summary statistics and results of the ANOVA, given that the response and covariate were expected to be correlated, we would have expected the covariate to be significant, as it has been in previous examples. However, the covariate is not significant at the 5% significance level. In previous examples, including the covariate has resulted in a significant decrease in the estimated error variance and in the width of the confidence interval 261 A n a ly s i s of Covarian ce estimate of the treatment difference. In this example, the estimated error variance in the ANOVA model is 97.020 and for the ANCOVA model it is 85.040, so they are about the same. Based on the ANOVA model, the width of the confidence interval estimate of the treatment difference is 18.50, while for the ANCOVA model it is 17.66, again about the same. In this example, including the covariate in the model has not resulted in much of a decrease in unexplained variation or in the precision of the confidence interval estimate. Why is this? The answer can be obtained by careful inspection of Fig. 16, which gives the graphical representation of the fitted ANCOVA model superimposed over a scatter plot of the response and covariate values. Notice that in this case the two parallel regression lines do not fit the data well. As in previous examples, the assumption that the regression lines are parallel is implicit in this ANCOVA model. Looking back at the ANCOVA plots of previous examples, we can see that the assumption of parallel lines has been appropriate in those cases. In this example, however, it is clear from the ANCOVA plot that while the relationship between the response and the covariate does appear to be linear in each group, the linear relationships are not the same. In particular, in Treatment Group A there is an increasing relationship between the covariate and the response, whereas in Treatment Group B the relationship is decreasing. In this case, therefore, the ANCOVA model that imposes a parallel lines assumption is too restrictive and can therefore lead to erroneous conclusions. For these data, a more flexible ANCOVA model that allows the linear relationship between the response and the covariate to be different in the two treatment groups is needed. Table 20. ANOVA comparing mean covariate values across treatments for Example 4. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F 0.55 0.4662 Model 1 4.050 4.050 Error 18 131.542 7.308 Corrected Total 19 135.592 Table 21. ANCOVA results for Example 4. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model 2 304.999 152.499 1.79 0.1965 85.040 F Value P>F Error 17 1445.679 Corrected Total 19 1750.678 Source DF Type III SS X 1 300.674 300.674 3.54 0.0773 GRP 1 25.453 25.453 0.30 0.5914 Type III Tests Mean Square Least Squares Means for Effect GRP Point Estimate of TRT B – TRT A 2.29 95% Confidence Interval for TRT B – TRT A Lower CL Upper CL 11.12 6.54 262 McCa rter Fig. 16. Scatter plot of the response variable and covariate, with the fitted parallellines ANCOVA model for Example 4. title1 "ANCOVA To Compare Mean of Y Across Groups, Adjusting for X"; title2 "Allowing for Separate Slopes"; proc mixed data=example_4; class grp; model y = x grp x*grp ; lsmeans grp / pdiff cl ; run; Fig. 17. SAS statements to perform ANCOVA (nonparallellines model) for Example 4. Fortunately, it is straightforward to extend the ANCOVA model and give it the flexibility to allow the slopes to vary across treatment groups. To do this, we include a term for the interaction between treatment group and the covariate. In such a model, the main effect for the covariate fits an overall average slope, and the interaction term fits a slope deviation from this average slope for each group. Fig. 17 shows the SAS statements for this extended model. The code also contains an LSMEANS statement with options for comparing treatment groups. Table 22 shows the results of this analysis. Note that while the previous model was not significant, this model is highly significant. The pvalue associated with the X×GRP interaction tests the null hypothesis that the slopes of the regression lines are the same in the two treatment groups. With the pvalue < 0.0001, there is strong evidence that the slopes are not equal. The main effect for the covariate is not significant (pvalue = 0.5173), whereas the main effect for treatment is significant (pvalue < 0.0001). Because the interaction between treatment group and covariate is significant, however, as advised by Vargas et al. (2018) in Chapter 7, we will refrain from interpreting the main effects and their pvalues and focus on the interaction when making comparisons. 263 A n a ly s i s of Covarian ce Because the slopes can be different, this model provides a much better fit to the data than the previous ANCOVA model. This improvement in fit can be assessed visually in the ANCOVA plot in Fig. 18, which provides a graphical representation of this fitted ANCOVA model superimposed over a scatter plot of the response and covariate. From Table 22, we see that allowing the slopes to differ has resulted in an 83% reduction in the estimated error variance, which has decreased from 85.040 in the previous ANCOVA model to 14.060 in the current model, at the cost of a single error degree of freedom. This improvement in model fit results in more powerful tests and more precise estimates of treatment effects and their difference. The pvalue for the comparison of treatment means is 0.6858. The point estimate of the difference between the mean of Treatment B and the mean of Treatment A is 0.71 units, with the 95% confidence interval estimate being [4.33, 2.92]. According to these results, there is insufficient evidence to conclude that a difference exists between treatment means. This may seem a bit surprising at first because the main effect for treatment was significant, but as stated previously, we need to be careful interpreting comparisons involving main effects when the model includes a significant interaction term. Another look at the ANCOVA plot will provide insight into what is going on. Recall from discussions of the ANCOVA plot in previous examples that the vertical difference between the fitted regression lines is equal to the estimated treatment difference. From the ANCOVA plot in Fig. 18, it is clear that the vertical distance between fitted regression lines depends on the value of the covariate at which the vertical distance is computed. Hence, to compare treatment means through a hypothesis test or by estimating their difference, we need to specify the value of the covariate at which the comparison is to be made. Looking back at the LSMEANS statement in Fig. 17, note that no such specification was explicitly made, and yet SAS produced a test comparing the treatment means and estimated their difference. At what value of the covariate was this comparison made? Table 22. ANCOVA (nonparallel slopes) results for Example 4. ANOVA Table Source DF Sum of Squares Mean Square Model 3 1525.722 508.574 Error 16 224.955 14.060 Corrected Total 19 1750.678 F Value P>F 36.17 < 0.0001 Type III Tests Source DF Type III SS Mean Square F Value P>F X 1 6.165 6.165 0.44 0.5173 GRP 1 1093.234 1093.234 77.76 < 0.0001 X*GRP 1 1220.724 1220.724 86.82 < 0.0001 Least Squares Means for Effect GRP TRT B – TRT A TRT A 64.21 TRT B 63.51 Pvalue 0.6858 Point Estimate 0.71 95% Confidence Interval Lower CL Upper CL 4.33 2.92 264 McCa rter Fig. 18. Scatter plot of the response variable and covariate, with the fitted nonparallellines ANCOVA model for Example 4. It turns out that the default behavior in SAS is to make such comparisons at the overall mean value of the covariate. To make a comparison at another value of the covariate, the AT option of the LSMEANS statement can be used. In this example, the overall mean of the covariate is 10.12. Figure 19 gives the code that explicitly instructs SAS to make the comparison of treatments at the covariate mean value of X = 10.12. This code will produce the same results obtained above using the code which did not specify the value at which to make the comparison. To make the comparison at a different value of the covariate, simply substitute that value in place of 10.12 in the AT statement. Table 23 gives the results of comparing the two treatments at various values of the covariate. The results of the comparison depend on the particular value of the covariate at which the comparison is made. For example, using a 5% significance level, the treatment means would not be determined to be different at covariate values of 9.5, 10.0, or 10.5, but would be determined to be different at covariate values of 8.0, 8.5, 9.0, 11.0, and 11.5. When slopes are different across treatment groups, the researcher should explicitly choose the covariate values at which to make treatment comparisons, rather than relying on the default behavior of the software. The covariate values at which to make comparisons must be determined by the researcher in each particular situation and will typically be driven by the research questions being investigated. Lessons Learned from Example 4 When performing ANCOVA it is imperative that the covariate portion of the model be specified correctly (Milliken and Johnson, 2002). The relationship between the 265 A n a ly s i s of Covarian ce title1 "ANCOVA To Compare Mean of Y Across Groups, Adjusting for X"; title2 'Allowing for Separate Slopes"; proc mixed data=example_4; class grp; model y = x grp x*grp ; lsmeans grp / pdiff cl at x=10.12 ; run; Fig. 19. Performing comparisons at a particular covariate value using the AT keyword. response and covariate must be determined and correctly specified to draw accurate, reliable conclusions from an ANCOVA. The first thing that must be determined is the nature of the relationship between the response and the covariate. Is the relationship linear, or does some higher order relationship exist? Scatter plots of the response and covariate can help determine the nature of the relationship. The next thing to determine is whether the relationship between the response and covariate is the same in each treatment group. Again, scatter plots can help answer this question. This can also be addressed formally via modeling by including a term for the interaction between the treatment classification variable and the covariate. This is an important step that should be routinely performed. The pvalue for the interaction term tests the null hypothesis that the relationship is the same across all treatment groups. If the interaction term is significant, it can be retained, resulting in a model that fits nonparallel lines or surfaces. In this case, the researcher should explicitly specify the values of the covariate at which treatment comparisons are to be made. If the interaction term is not significant, then it can be removed and a parallel lines or surfaces model can reasonably be used. In this case, the software can be allowed to make treatment comparisons at the default value of the covariate, since the difference will be the same at every value of the covariate. In either case, it is only after the covariate part of the model has been correctly specified that treatment comparisons should be made. Summary of Lessons Learned from Example 1 through Example 4 From the examples considered thus far, we have gleaned several things about ANCOVA. A quality covariate can have a very positive inferential impact on an analysis to compare treatment effects. Specifically, a quality covariate can increase the power of Table 23. ANCOVA comparison of treatments at specified values of the covariate for Example 4. TRT B – TRT A Value of Covariate 8.0 TRT A 57.28 TRT B 71.50 pvalue Point Estimate 95% Confidence Interval Lower CL < 0.0001 14.22 9.01 Upper CL 19.43 8.5 58.92 69.61 0.0002 10.70 6.03 15.36 9.0 60.55 67.73 0.0023 7.18 2.97 11.38 9.5 62.18 65.84 0.0613 3.66 0.20 7.51 10.0 63.82 63.96 0.9364 0.14 3.51 3.79 10.2 64.21 63.51 0.6858 0.71 4.33 2.92 10.5 65.45 62.07 0.0651 3.38 7.00 0.24 11.0 67.09 60.19 0.0013 6.90 10.66 3.14 11.5 68.72 58.30 < 0.0001 10.42 14.47 6.36 266 McCa rter hypothesis tests and increase the precision of estimates by reducing unaccountedfor variation in the response variable that is due to heterogeneity in experimental units with respect to characteristics that affect the response. Including a covariate can also reduce or eliminate bias in estimates of treatment effects and their differences in situations where the assigned treatment groups vary with respect to such characteristics. As has been demonstrated, not including a covariate in such a case can lead to qualitatively incorrect conclusions regarding treatment effects, as well as biased estimates of treatment effects and their differences. To be a quality covariate, the covariate must be correlated to the response variable. The stronger the correlation, the better. To avoid artificially diminishing treatment effects, the covariate should either be such that it cannot be affected by the treatment, or it must be measured before application of the treatment to the experimental units. The safest approach is always to measure the covariate before treatments are applied to the experimental units. Before making treatment comparisons in the context of an ANCOVA, it is imperative that the relationship between the response and covariate be determined and correctly specified. As part of this, it must be determined whether the relationship between response and covariate is the same across all treatment groups. This can be formally tested by including appropriate interaction terms in the model and determining whether the interactions are statistically significant. Only when the covariate part of the model has been determined to be adequate should treatment comparisons be performed. If the relationship between response and covariate is determined to differ across treatment groups, the researcher should explicitly choose the covariate values at which to make treatment comparisons. ANCOVA Example 5: Do Fall Armyworm Larvae Grow Better on Some Soybean Varieties than Others? Researchers performed an experiment to determine whether fall armyworm (Spodoptera frugiperda J.E. Smith) larvae grow better on some soybean (Glycine max [L.] Merr.) varieties than others. Forty fall armyworm larvae were used in the experiment, with ten larvae randomly assigned to each of four soybean varieties. The experiment therefore consisted of a oneway treatment structure with four levels and a completely randomized design structure. On day zero of the experiment, each larva’s initial weight was obtained for use as a covariate. Then for each armyworm larva, one leaflet was removed from that larva’s assigned soy variety and used to feed the larva for two days. The primary response variable was final armyworm weight, which was measured at the end of the experiment on day two. It was believed that final weight may depend on initial weight. One way to account for variation in final armyworm weight due to variation in initial larval weight is to include initial weight as a covariate in an ANCOVA. To the extent that final weight and initial weight are correlated, this should improve the fit of the model and the performance of the resulting analyses. This was the approach used in this example. Boxplots of the final larvae weights are given in Fig. 20. The levels of variability in the samples look similar. There is some variability in the sample means, with the biggest observed difference being between the weights of the larvae assigned to the Davis and Braxton varieties. 267 A n a ly s i s of Covarian ce An ANOVA to compare the mean final weights across the four treatments is given in Table 24. The pvalue for the comparison is 0.1284, so at the 5% significance level there is insufficient evidence to reject the null hypothesis that the mean weights are the same across all four soybean varieties. Because the overall ANOVA test is not significant, we will not perform pairwise comparisons. Note that for comparison with the subsequent ANCOVA analysis, the common width of the 95% confidence intervals for differences in mean larvae weights between soybean variety means is 12.30 units. Note also that the estimated error variance based on this ANOVA model is 49.78. Recall from previous discussions that the degree to which a covariate will improve an analysis is dependent on the strength of its relationship with the response variable and on the degree to which treatment groups differ with respect to the distribution of the covariate. To get an idea of the improvement that we might expect from the covariate in this analysis, we will next look at both of these factors. Table 25 gives correlations between the initial and final armyworm weights, by Fig. 20. Boxplots of armyworm final weights from each soy variety. Table 24. ANOVA comparison of final armyworm weights across varieties. ANOVA Table Source DF Model 3 Sum of Squares Mean Square 301.77 100.59 49.78 Error 36 1792.19 Corrected Total 39 2093.96 F Value P>F 2.02 0.1284 Type III Tests Source DF Type III SS Mean Square F Value P>F VARIETY 3 301.77 100.59 2.02 0.1284 268 McCa rter Table 25. Correlation between final and initial armyworm weights. Correlation Estimates Variety Point estimate 95% Confidence Interval Lower CL Upper CL Asgrow 0.63 0.03 0.90 Braxton 0.63 0.03 0.90 Davis 0.78 0.24 0.94 William 0.60 0.08 0.89 All Varieties Combined 0.61 0.36 0.77 Fig. 21. Boxplots of initial larvae weights. variety as well as over the entire sample. Within varieties, the correlation estimates range from 0.60 to 0.78. The 95% confidence interval estimates all have a broad range of overlap, and hence the assumption of a common correlation across all groups is reasonable at this point. The point estimate of the correlation obtained from the complete sample is 0.61. The 95% confidence interval estimate based on the complete sample is [0.36, 0.77]. Note that since this confidence interval does not contain zero, at the 5% significance level the conclusion is that the correlation between final and initial weights is nonzero, and in particular positive. Figure 21 shows boxplots of the covariate. Based on the boxplots, the distributions of initial larval weights appear to be similar in the four assigned samples, both with respect to variability and mean values. Results of an ANOVA to compare the initial weight means across the four varieties are given in Table 26. The pvalue of 0.9762 reinforces the observation based on the boxplots that there is no evidence of a difference in covariate means. 269 A n a ly s i s of Covarian ce The correlation between initial and final weights is significant, so we may expect that the inclusion of the covariate in the model may improve its power and precision. Because this correlation is smaller than the correlations in the previous examples, however, the degree of improvement may not be as great as seen in those models. In addition, since the distribution of the covariate does not appear to differ across samples, the covariate will not be correcting for any bias that would result from such differences. Results of an ANCOVA that accounts for initial larvae weight in comparing the mean final larvae weights across the four soybean varieties are given in Table 27. Note first that the estimated error variance is 29.74, which is a 40.3% reduction from the estimate of 49.78 provided by the ANOVA model, at the cost of only a single degree of freedom. This should increase the power of hypothesis tests and improve precision. The overall model is highly significant, with a pvalue < 0.0001. Because it is significant, we can evaluate the significance of each term in the model. The pvalue associated with the covariate initial weight is < 0.0001, so the linear relationship between the response and covariate is highly significant. The pvalue associated with the variety treatment effect on larva weights is 0.0357, and so we conclude that soybean variety significantly affected final weight means of larvae. Because of this, we will evaluate the pairwise comparisons. Based on the leastsquare means comparisons, we see that the mean final armyworm weights are different under the Braxton and Davis varieties (pvalue = 0.0047). The observed difference between the Asgrow and Davis varieties is almost significant at the 5% significance level (pvalue = 0.0549), and the observed difference between the Braxton and William varieties is almost significant at the 10% significance level (pvalue = 0.1094). Note that the common width of the 95% confidence interval estimates of differences between means is 9.92 mg, which is a reduction of 22.6% from the width provided by the ANOVA model. Figure 22 shows the fitted ANCOVA model with its four regression lines superimposed over a scatter plot of the final and initial weights. Each regression line represents expected mean final larvae weight as a function of initial larvae weight, one regression line for each of the four soybean varieties. Because this model does not include an interaction between soybean variety and initial larvae weight, it forces the fitted lines to be parallel. A formal test of the assumption that the parallellines model is adequate was performed by fitting an ANCOVA model containing the interaction between soybean variety and initial larvae weight. The pvalue associated with the interaction term was 0.8369, so there was insufficient evidence to conclude that the lines were not parallel. The Braxton and Davis varieties, which are the only soybean varieties significantly different at the 5% level, are represented by the upper and lower regression lines, respectively, and the estimated difference Table 26. ANOVA comparing mean initial armyworm weights across varieties. ANOVA Table Source DF Sum of Squares Mean Square F Value P>F Model 3 0.332 0.111 0.07 0.9762 Error 36 57.935 1.609 Corrected Total 39 58.267 270 McCa rter Table 27. ANCOVA comparison of final armyworm weights across varieties adjusted for initial weight. ANOVA Table Source DF Model 4 Sum of Squares 1053.03 Mean Square 263.26 29.74 F Value P>F 8.85 < 0.0001 F Value P>F Error 35 1040.93 Corrected Total 39 2093.96 Source DF Type III SS INITIAL_WT 1 751.26 751.26 25.26 < 0.0001 VARIETY 3 284.15 94.71 3.18 0.0357 Type III Tests Mean Square Least Squares Means for Effect VARIETY VARIETY 1 – VARIETY 2 VARIETY 1 VARIETY 2 Pvalue Point estimate 95% Confidence interval Lower CL Upper CL Asgrow Braxton 0.3087 2.52 2.43 7.47 Asgrow Davis 0.0549 4.85 9.80 0.11 Asgrow William 0.5442 1.49 6.45 3.46 Braxton Davis 0.0047 7.37 12.32 2.41 Braxton William 0.1094 4.02 8.98 0.95 Davis William 0.1790 3.35 1.61 8.31 between the mean final larvae weights for these two varieties is equal to the vertical distance between these lines. Summary of Example 5 An ANCOVA model, with its explanatory covariate, resulted in an analysis that improved power and precision compared with the ANOVA model, which included only the classification treatment effect as an explanatory variable. In the overall sample, the correlation between the response variable final weight and the covariate initial weight was 0.61, a moderate correlation. The ANOVA model was not significant at the 5% level. Inclusion of the covariate in the ANCOVA model accounted for additional variability in the response and decreased the estimated error variance by 40%. This reduction in unaccountedfor variation increased the power of the hypotheses tested by the model. The overall ANCOVA model was highly significant, as was the linear relationship between the response and the covariate. Having accounted for the variability in the response associated with the covariate, the variety effect on final larvae weight became significant, and larvae fed on two of the four varieties differed significantly in mean final weight. In addition, inclusion of the covariate increased the precision of the confidence interval estimates of differences between variety means by reducing their width by 22.6%. By including initial larvae weight as a covariate in an ANCOVA, we improved the overall analysis by increasing the power of treatment comparisons and improving the precision of estimates of treatment effects and their differences. A n a ly s i s of Covarian ce Summary This chapter investigates analysis of covariance. It does so primarily through use of five examples designed to illustrate issues that can arise in the context of designed experiments where comparison of treatment effects is the purpose. One such issue is heterogeneity in the experimental units within treatment groups with respect to characteristics that affect the response. This heterogeneity induces variation in the response, which if not accounted for, decreases the power of significance tests and decreases the precision of estimates of treatment effects and their differences. Another potential issue is differences in such characteristics across assigned treatment assignment groups. Such differences bias estimates of treatment effects and their differences and can lead to qualitatively incorrect inferences. The examples in this chapter have demonstrated how using a quality covariate can reduce unaccountedfor variation in the response, reduce or eliminate bias when it exists, and as a result, provide more powerful hypothesis tests and more accurate and more precise estimates of treatment effects and their differences. This chapter presents ANCOVA in the context of a simple design structure to focus on the essential features of this modeling approach that distinguish it from both ANOVA and regression. Like both ANOVA and regression, ANCOVA can be utilized to analyze data from a wide variety of experimental situations, from experiments with simple designs to those with complex designs. Random effects or nonindependent error structures are typically utilized in statistical models to accommodate more complex experimental designs. As long as the statistical software being used can correctly accommodate the design features of an experiment, that software can be used to perform ANCOVA utilizing that design structure. Fig. 22. Scatter plot of final weight versus initial weight, with the fitted ANCOVA model. 271 272 McCa rter Since ANCOVA models are contained within the class of linear models, the issues involved in applying the ANCOVA model to more complex experimental designs are common to the issues involved in extending the class of linear models to the class of linear mixed models. In particular, they are similar to the issues faced when applying ANOVA models and regression models to data from complex experimental designs. Resources specializing in mixed model analysis and ANCOVA can be found in the reference section of this chapter. Key Learning Points ·· Unaccountedfor heterogeneity in experimental material that affects a measured response increases variation in the response. This reduces the power of hypothesis tests and decreases the precision of parameter estimates. Unaccountedfor heterogeneity can also bias estimates of treatment effects. ·· ANCOVA models extend ANOVA models by accounting for heterogeneity in experimental material through the inclusion in the model of one or more covariates that have been measured on the experimental units. ·· To be effective, covariates should be related to the response, either overall or within treatment groups. ·· To avoid attenuating estimates of treatment effects, covariates should be measured before assigned treatments are applied, or be such that they cannot be affected by the treatment being applied. The safest approach is to measure covariates before treatments are applied whenever possible. ·· The relationship between the response variable and the covariate must be determined and correctly specified in the ANCOVA model to draw valid inferences from the model. ·· It is very important to determine whether the relationship between the response and the covariate is the same in all treatment groups, or whether the relationship varies from one group to another. This is accomplished in the modeldevelopment process by including a term for, and testing the significance of, the interaction between the treatment classification variable and the covariate. ·· If the relationship between the response and the covariate differs across treatment groups, then treatment differences will depend on the value of the covariate. In order for treatment comparisons to be meaningful, the researcher must specify the values of the covariate at which treatment comparisons are made. Review Questions 1. True or False: In an ANCOVA, the covariate should either be measured before application of the assigned treatment, or be such that it cannot be affected by the treatment if it is measured after the treatment is applied. 2. True or False: A covariate is effective only when a statistically significant difference A n a ly s i s of Covarian ce exists in the covariate means across the treatment assignment groups. 3. True or False: A covariate will be effective only when it has a statistically significant correlation with the response variable over the entire dataset. 4. True or False: When developing an ANCOVA model, the nature of the relationship between the response and the covariate must be determined. 5. True or False: When developing an ANCOVA model, one should check to see whether the relationship between the response and the covariate differs across treatment assignment groups, and to accommodate such a difference in the model if it exists. 6. In some of the examples considered in this chapter, the issue that ANCOVA has been used to address has been the result of an imbalance between treatment assignment groups with respect to the values of the covariate. In other words, one of the treatment assignment groups is comprised mostly of experimental units with large values of the covariate, while the other is comprised mostly of those with small values of the covariate. The good news is that large imbalances are unlikely when randomization is used to allocate experimental units to treatment groups. For example, suppose that twenty experimental units are available for an experiment to compare the effects of two treatments. Ten experimental units are to be randomly assigned to one treatment group, and the remaining ten to the other treatment group. Suppose that the covariate has already been measured prior to treatment assignment, and consider the subset of the experimental units with the ten largest values of the covariate and the subset with the ten smallest values of the covariate. The various randomization outcomes that can occur can be combined into the following six scenarios, from most balanced to most unbalanced with respect to these two subsets: A. Both treatment groups receive five of the experimental units with the ten largest covariate values; B. Either one of the treatment groups receives six of the ten experimental units with the largest covariate values, and the other treatment group receives four; C. Either one of the treatment groups receives seven of the ten experimental units with the largest covariate values, and the other treatment group receives three; D. Either one of the treatment groups receives eight of the ten experimental units with the largest covariate values, and the other treatment group receives two; E. Either one of the treatment groups receives nine of the ten experimental units with the largest covariate values, and the other treatment group receives one; F. Either one of the treatment groups receives all ten of the experimental units with the largest covariate values, and the other treatment group receives the experimental units with the ten smallest covariate values. a) Find the probability associated with each of the randomization scenarios, 273 274 McCa rter A through F, above. b) What is the probability that a randomization results in either of the two most balanced scenarios, A or B? c) What is the probability that a randomization results in either of the two most unbalanced scenarios, E or F? d) Even if a randomization does not result in an unbalanced scenario, can ANCOVA still provide a better analysis than ANOVA? If so, discuss the ways in which ANCOVA can provide improvement over ANOVA. e) What do these results suggest about the importance of randomization in the treatment assignment process? 7. In this chapter interpretation of ANCOVA results has been based on the assumption that either the covariate has been measured prior to application of the treatment, or that it is impossible for the treatment to affect the value of the covariate. In some situations, however, neither of these conditions can be guaranteed to be satisfied. For example, in observational studies where ANCOVAlike models are often utilized, it is common for measured categorical factors to be used as “treatments” and other measured characteristics to be used as covariates. In such situations where application of the treatment is not under the control of an experimenter, the treatments and the covariates are likely to be related. Discuss how the results of an ANCOVA are to be interpreted in such situations, and how these interpretations differ from those in this chapter, where by virtue of how the experiment is performed, the value of the covariate is not affected by the treatment. Data Analysis Exercises 1 Through 5 The five tables below give data for a series of data analysis exercises. In each case, the data come from hypothetical experiments to compare the effects of two treatments, denoted A and B, on the mean of a response variable, Y. Twenty experimental units were available for each study, ten being randomly assigned to treatment A and the remaining ten to treatment B. A single covariate, X, was measured on each experimental unit before its assigned treatment was applied. The response variable Y and the covariate X were expected to be related. Once all five data analysis exercises have been completed, write up a discussion of the various ways in which the use of ANCOVA has improved the analyses and led to better inferences in these exercises. Questions To Be Considered For Each Data Analysis Exercise 1. Ignoring the covariate, X, perform an ANOVA to compare the mean of the response variable Y for the two treatments. As part of this analysis, also construct a 95% confidence interval estimate of the difference between the response variable means for the two treatments. What is the conclusion based on this analysis? Is there sufficient evidence to conclude that a difference exists between treatments with respect to the response variable means? A n a ly s i s of Covarian ce 2. Are the distributions of the covariate the same for the two treatments, or are they different? Perform an ANOVA on the covariate X to compare the covariate means across the two treatment assignment groups. 3. Are the response variable Y and the covariate X related? Construct scatter plots of Y versus X to assess the nature and strength of their relationship graphically. Are they linearly related, and hence correlated? Is their correlation approximately the same in the two treatment groups? To answer these questions, perform a correlation analysis of Y and X to estimate their correlation and to test the null hypothesis that their overall correlation is zero, and construct a 95% confidence interval estimate of the correlation. Do this for the whole dataset, and then separately for each treatment group. 4. Perform an ANCOVA to compare the mean of the response variable Y under the two treatments, taking the covariate X into account. Is a parallellines model adequate, or is a nonparallellines model necessary? Perform the appropriate test in the context of the ANCOVA in making this determination. What are the conclusions based on this ANCOVA analysis? Is the covariate significant? Is there sufficient evidence to conclude that a difference exists between the treatments with respect to the response variable means? Construct a 95% confidence interval estimate of the difference between response variable means based on this analysis. 5. Discuss the impact of including the covariate on the estimate of the error variance. Is the mean squared error (MSE) smaller for the ANCOVA than for the ANOVA? If so, by what percentage does it change? Also, what happens to the width of the confidence interval for the difference of response variable means when the covariate is included in the model? Is the confidence interval width smaller for the ANCOVA model than for the ANOVA model? If so, by what percentage does it change? Finally, what happens to the point estimate of the difference in response variable means when the covariate is included in the model? Based on these considerations, what can be said about the impact of the inclusion of the covariate on the power, precision, and accuracy of the analysis? 6. Were the overall conclusions from the ANOVA and the ANCOVA the same, or were they different? If they were different, which analysis would you use, and why? 275 276 McCa rter Data for Analysis Exercise 1 Treatment A Y 49.4 72.2 61.6 63.4 71.0 49.6 56.6 61.1 66.1 71.9 X 5.2 13.2 9.2 11.3 12.3 7.2 8.3 6.7 9.7 12.0 Y 95.3 96.7 75.8 102.6 78.5 101.4 100.8 88.2 79.1 108.0 X 20.2 20.5 12.5 21.9 15.3 20.8 21.3 17.2 14.4 21.2 Treatment B Data for Analysis Exercise 2 Treatment A Y 56.7 46.3 71.2 69.0 59.7 59.8 60.1 66.8 72.5 57.6 X 7.0 5.7 14.6 11.9 10.8 10.2 8.1 10.8 14.5 7.9 Y 70.3 78.0 63.1 57.1 68.7 70.4 73.3 61.7 73.7 77.4 X 10.5 12.8 9.4 6.6 11.3 9.5 9.4 6.8 11.9 11.6 Treatment B Data for Analysis Exercise 3 Treatment A Y 54.8 54.6 52.5 68.2 X 8.9 8.3 10.0 12.6 59.6 76.8 56.3 71.2 62.0 68.0 9.4 13.2 7.9 15.0 9.2 13.8 Treatment B Y 54.4 55.8 69.4 51.7 67.2 43.9 71.6 66.7 42.4 55.7 X 14.7 19.1 20.3 15.3 21.3 12.2 21.6 20.5 13.4 15.9 Data for Analysis Exercise 4 Treatment A Y 73.9 46.7 48.9 83.0 X 14.4 6.1 6.3 14.6 80.7 73.1 54.8 53.4 79.3 60.6 15.0 12.1 7.2 7.3 13.5 11.5 Treatment B Y 68.6 66.9 53.5 79.6 82.0 72.4 67.6 77.4 70.6 75.4 X 9.6 9.9 5.0 13.3 13.2 12.2 9.9 13.4 10.0 11.1 Data for Analysis Exercise 5 Treatment A Y 70.9 71.8 61.0 71.4 56.9 55.8 62.1 80.8 54.8 50.8 X 12.6 13.5 10.9 13.9 6.4 9.1 7.4 13.2 8.8 5.6 Y 52.4 57.9 68.9 62.7 62.4 78.5 55.0 79.2 82.6 51.6 X 12.3 10.7 11.1 11.6 10.2 6.5 14.8 6.8 5.5 13.7 Treatment B Acknowledgments Data for Example 5 kindly provided by Dr. Michael Stout, Department of Entomology, Louisiana State University, Baton Rouge, Louisiana. A n a ly s i s of Covarian ce References Casler, M.D. 2018. Power and replication–Designing powerful experiments. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. Freund, R.J., W.J. Wilson, and D.L. Mohr. 2010. Statistical methods, Third ed. Academic Press/ Elsevier, Amsterdam, the Netherlands. GarlandCampbell, K. 2018. Errors in statistical decision making. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. Graybill, F.A. 1976. Theory and application of the linear model. Wadsworth & Brooks/Cole, Pacific Grove, CA. Littell, R.C., G.A. Milliken, W.W. Stroup, R.D. Wolfinger, and O. Schabenberger. 2006. SAS for mixed models, Second ed. SAS Institute Inc., Cary, NC. Littell, R.C., W.W. Stroup, and R.J. Freund. 2002. SAS for linear models, Fourth ed. SAS Institute Inc., Cary, NC. McIntosh, M. 2018. Analysis of variance. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. Miguez, F., S. Archontoulis, and H. Dokoohaki. 2018. Nonlinear regression models and applications. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. Milliken, G.A., and D.E. Johnson. 1992. Analysis of messy data, Volume I: Designed experiments. Chapman & Hall, London, UK. Milliken, G.A., and D.E. Johnson. 2002. Analysis of messy data, Volume III: Analysis of covariance. Chapman & Hall/CRC, London, UK. Richter, C., and H.P. Piepho. 2018. Linear regression techniques. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. Stroup, W. 2018. Analysis of nonGaussian data. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. Vargas, M., B. Glaz, J. Crossa, and A. Morgounov. 2018. Analysis and interpretation of interactions of fixed and random effects. In: B. Glaz and K.M. Yeater, Applied statistics in the agricultural, biological, and environmental sciences. ASA, CSSA, SSSA, Madison, WI. 277 278 McCa rter