Nonparametric tests for interaction in two-way ANOVA with balanced replications

Factorial designs allow researchers to explore main effects and interactions. Particularly, detecting interaction is crucial to conducting data analysis as the presence of interaction in the data can influence whether a factor is significant or not. In a parametric setting, the usual test for interaction is an $F$-test (Montgomery, 2020). However, for data collected in various fields and industries, errors tend to not follow the normal distribution. The use of tests with known issues provides continued motivation for developing appropriate nonparametric tests for interaction in the two-way layout with multiple replications per cell. We consider the model for the general two-way layout and some different types of interaction. We focus on a balanced design with an equal number of replications per cell.

In the case of a two-way layout with balanced replications per cell, the general model is

where two factors $U$ and $V$, having $I$ and $J$ levels respectively, are being investigated and $K$ is the number of replications per cell. The $Y_{ijk}$’s are $IJK$ observations which are mutually independent random variables. The error terms, denoted by the $\varepsilon_{ijk}$’s, are assumed to have a common median $\theta$. For notation, $\alpha_{i}$ is the effect of the $i^{\text{th}}$ level of factor $U$, $\beta_{j}$ is the effect of the $j^{\text{th}}$ level of factor $V$, and $\gamma_{ij}$ is the effect of the interaction between the $i^{\text{th}}$ level of factor $U$ and the $j^{\text{th}}$ level of factor $V$.
The hypotheses for the typical test of interaction, with no restrictions on $\alpha_{i}$ and $\beta_{j}$, are

Stating that not all $\gamma_{ijk}$ values are zero is only one way of stating that interaction is present. Multiple definitions of interaction have been developed, and some are only applicable to the location-family model, while others are more general.

In this paper, we propose new aligned rank-based tests ($APCSSA/APCSSM$) for interaction in the general two-way layout with balanced replications per cell. We start with a review of existing test procedures for interaction. Then, we set forth our procedures. Through Monte Carlo simulation power studies with seven competing tests for interaction, we demonstrate the consistent performance of $APCSSA/APCSSM$ in various settings. We conclude with a discussion of our findings and thoughts on future work in this area.

Researchers have developed a variety of procedures to test for interaction in the general two-way layout. The classical test procedure for determining the presence of interaction (as defined by non-zero $\gamma_{ijk}$’s) in a parametric setting is the standard ANOVA $F$-test (Montgomery, 2020). When the error terms in (1) are normally distributed, the two-way analysis of variance sum of squares identity partitions the variability and leads to an $F$-test for our null hypothesis of no interaction. The $F$-test assumptions of normality and common variance may not be met in all situations, however. When the assumptions necessary for performing an $F$-test are not met, a nonparametric test should be used.

Multiple nonparametric tests for interaction exist in the literature. Conover and Iman (1976) suggest using the rank transform approach ($RT$) in factorial settings. $RT$ uses joint ranks (averages for ties correction), and then uses the parametric $F$-test on the ranks. Another nonparametric test for interaction based on the rank transform approach is the aligned rank transform ($ART$). With the $ART$ procedure, an alignment is performed in the rows and columns before the joint ranking of the response variables. Wobbrock, Findlater, Gergle, and Higgins (2011) align the data before performing analysis like $RT$ to study individual effects. Alignment is performed by computing the row and column averages and subtracting each from all corresponding entries in the individual rows and columns. After obtaining the joint ranks on the aligned data, the $F$-statistic is calculated on the joint ranks.

The aligned rank transform has been found to perform better than the rank transform, but it still has problems with elevated Type I error rates and with nonnormal error terms (Richter, 1999; Luepsen, 2017). Mansouri and Chang (1995) performed a comparative analysis between the $F$-test, rank transform, and two versions of the aligned rank transform. They conclude that the aligned rank tests are preferred over the other test procedures. They also note that all the test procedures they considered performed poorly in the Monte Carlo power study with errors drawn from a Cauchy distribution. In our power studies, we use the art procedure from the $ARTool$ package in $\mathsf{R}$ (Kay et al., 2021).

Salazar-Alvarez, Tercero-Gómez, Cordero-Franco, and Conover (2014) conduct a literature review and conclude that most of the techniques were based on $RT$. In order to provide a more complete comparison of nonparametric tests for interaction, we will consider methods that do not stem from Conover and Iman’s $RT$.

A different testing approach was proposed by De Kroon and Van Der Laan (1981), based on their definition of rank interaction. Their test statistic was developed to detect the presence of rank interaction in the two-way layout with multiple replications per cell. As noted by De Kroon and Van Der Laan (1981) and Hartlaub et al. (1999), the proposed statistic only works well in detecting a rank interaction of type $U^{*}(V)$ if no main effect for $U$ is present (and vice versa for detecting type $V^{*}(U)$, which works well where no main effect for $V$ is present).

Another way to detect interaction in factorial design is through the lens of linear regression, specifically rank-based linear models. Kloke and McKean (2012) proposed a rank-based analysis for all three hypotheses including the main effects and interactions based on a reduction of dispersion from the reduced to the full model. For the computations in this paper, we have utilized the raov function in Rfit package to compute their test statistic for interaction (Kloke and McKean, 2012).

Lastly, Hartlaub, Dean, and Wolfe (1999) developed procedures to test for interaction in the two-way layout with one observation per cell. An invariance problem with their statistics was solved by Lehman, Wolfe, Dean, and Hartlaub (Lehmann et al., 2001) who proposed symmetrized procedures. The symmetrized procedures, $S-SA$ (symmetrized statistics aligned by averages) and $S-SM$ (symmetrized statistics aligned by medians) are based on the statistics $CRA$ and $RCA$, and $CRM$ and $RCM$, respectively. In short, these procedures align within the rows or columns using averages or medians, and then rank within the other dimension. Reversing the aligning and ranking to create two statistics is similar to checking for both row and column concordance. The ranks are then combined in a cross-comparison framework to form appropriate statistics to test for interaction. A challenge for this procedure is that null means and variances for the statistics must be computed before the significance of the test statistic may be determined.

Salazar-Alvarez, Tercero-Gómez, Cordero-Franco, and Conover (2014) recommend developing new nonparametric methods that are not based on $RT$ to detect interaction. The test statistics proposed by Hartlaub, Dean, and Wolfe (1999) were found to perform well in the two-way layout with a single replication per cell. Thus, we propose an extension of their technique for the case of balanced replications per cell. Our proposed statistics use the technique of crossed comparisons (Tukey, 1991) to detect the interactions. Initial investigations summarizing cell information into a single statistic (such as a median or mean) and then applying the methods from Hartlaub, Dean, and Wolfe (1999) did not perform as well as these extended methods where all possible comparisons were examined.

Our proposed statistics generalize the comparison idea with all possible comparisons. We eliminate nuisance effects by aligning with averages or medians to remove one of the nuisance effects (row or column) and ranking within the columns or rows to remove the other. We call the proposed statistics $APCSSA$ and $APCSSM$. The names of the statistics come from the idea that they are extensions of $S\text{-}SA$ and $S\text{-}SM$ from Hartlaub, Dean, and Wolfe 1999, where we have added APC to stand for all possible comparisons.

Next, we describe how our proposed statistics, $APCSSA$ and $APCSSM$, are computed. We begin with $APCSSA$, the all possible comparisons extension of $SSA$. $APCSSA$ is the maximum of two standardized statistics, so we begin by outlining their calculation.

Step 1. Calculate $APCCRA$, which stands for all possible comparisons (APC), column alignment (C), row ranking (R), using averages for the alignment (A). Again, the name of the statistic just reflects that we are aligning within the columns using column averages, and then ranking within the rows, and then using all possible comparisons to create the statistic (below). We compute the $J(J-1)/2$ crossed comparisons denoted $V_{jj\;^{\prime}}$.

$APCCRA$ is the maximum of the $V_{jj\;^{\prime}}$’s.

Step 2. Calculate $APCCRAD$, which is just a scaled version of $APCCRA$. Divide $APCCRA$ by $K^{4}I(I-1)/2$, the number of summands in $APCCRA$, to obtain this scaled version of the crossed comparisons for the maximum column comparison. That is,

Step 3. Calculate $APCRCA$, which stands for all possible comparisons (APC), row alignment (R), column ranking (C), using averages for the alignment (A). Repeat Step 1, with alignment in the rows and ranking in the columns. $APCRCA$ is computed by taking the maximum of $I(I-1)/2$ possible row comparisons.

Step 4. Calculate $APCRCAD$. $APCRCAD$ is computed by dividing $APCRCA$ by $K^{4}J(J-1)/2$.

Step 5. Standardization. $APCCRAD$ and $APCRCAD$ are further standardized by subtracting the appropriate null mean and dividing by the appropriate null standard deviation. In order to do this, one must find

where $E_{0}(APCCRAD)$ and $E_{0}(APCRCAD)$ are the null means of $APCCRAD$ and $APCRCAD$ respectively, and $V_{0}(APCCRAD)$ and $V_{0}(APCRCAD)$ are the null variances of $APCCRAD$ and $APCRCAD$ respectively. These null means and variances are computed via simulation and their values are available on Github at https://github.com/tranbaokhue/Rank-based-InteractionTest. Note that if $I=J$, then by symmetry, the null means and null variances are equal and only one set needs to be computed.

Step 6. Calculate $APCSSA$. $APCSSA$ is the maximum of $APCCRAD^{*}$ and $APCRCAD^{*}$.
Alternatively, using medians to align the data instead of averages in our procedure yields $APCSSM$. The common ties correction of using average ranks for ties should be used during the ranking process.

Our simulation studies have verified previous work that $DEKR$ suffers from the introduction of row main effects and is not recommended for interaction detection in the balanced replications per cell setting. With Conover’s $RT$ approach, the resulting statistic does not compete well with the $F$-test and suffers elevated Type I error rates when the error terms come from nonnormal distributions. Although Kloke and McKean’s raov and $ART$ are much more powerful than $DEKR$ and $RT$, their rejection rates in various settings can be twice the significance level of 0.05. Our proposed statistics $APCSSA$ and $APCSSM$ perform well in settings with exponential and Cauchy error terms respectively. Additionally, when using $APCSSA$, not much power is lost compared to the $F$-test when error terms are from the normal or uniform distributions and $APCSSM$ is undeniably the best procedure for detecting interactions for data with Cauchy errors. In conclusion, we are able to recommend using $APCSSA$ and $APCSSM$ to detect interaction in the two-way layout with balanced replications per cell.

While we have demonstrated that our statistics work well to detect interaction in these settings, future work in this area remains. Further validation of these results with the power comparisons from settings with more replications per cell may be enlightening. Additionally, we have work in progress to develop $\mathsf{R}$ code so that the new statistics, which are computationally intensive, may be easily accessed by interested researchers. It is also natural to consider extending these techniques to the general two-way layout with an unequal number of replications per cell.

Acknowledgments

We are grateful to Jessica Jeong for her meticulous verification of $\mathsf{R}$ code and scripts. We would like to thank Amherst College and Kenyon College for supporting our summer research projects.