Signorella & Hayes (2012, April). Dependent Variables, Designs, and Data Analytic Decisions: Empirical Evidence on the Effects of Single-Sex Schools. Paper presented at the American Educational Research Association conference, Vancouver, BC.

According to the National Association for Single Sex Public Education (NASSPE, 2011), there will be 116 single-sex public schools and 390 coeducational public schools with single-sex classes in the United States in the 2011-2012 academic year, and that number is expected to increase (Salomone, 2006). It is important, therefore, that scholars carefully examine the costs and benefits of separating students by gender. The No Child Left Behind Act approved funding for innovative education programs, including single-sex schools and single-sex programs within coeducational schools. Districts were to conduct self-evaluations of their single-sex classes at least every two years and ensure that a “substantial relationship” exists between the single-sex nature of the classes and achievement of the schools’ educational objectives. The empirical literature, however, has produced no consensus (e.g., Bracey, 2006; Haag, 1998; Marsh, 1989; Mael, Alonso, Gibson, Rogers, & Smith, 2005). We will provide an overview of the methodological and data analytic issues in this disputed literature. The first major shortcoming concerns the methodologies typically employed in the field. The “gold standard” for assessing causal effects of school programs is random assignment and blind assessment, and no studies employing these techniques exist. Federal regulations require that enrollment in single-sex settings be voluntary and thus truly randomized designs are impossible to implement now in the US. Nonetheless, the weakness of non-random assignment might be addressed via methodological procedures and/or statistical controls. The use of such designs and controls are crucial because much of the reported success of single-sex education may be attributable to selection and school quality effects. First, students who elect to attend single-sex schools may differ systematically from those students who do not attend. Second, those applicants who are selected by the administrators to attend single-sex schools may differ systematically from those who are not selected. We will review recent empirical evidence that points to the presence of both types of selection operating in public single-sex educational contexts. There are also weaknesses in the methods used to review and summarize the data on school gender composition. Mael et al. (2005) concluded that a meta-analysis was not possible due to Department of Education rules on using only experimental studies, and instead used a type of vote-counting that has major problems. One of the reasons meta-analysis has gained support as a literature review technique is that narrative reviews can be interpreted differently by different readers and may rely on those misleading vote-counting methods. We will give examples of this type of problem in the single-sex reviews to date. The third major shortcoming of the literature on single-sex schooling is the lack of theoretical grounding. The majority of extant research on single-sex education focuses on the outcomes of such education without attention to the mechanisms underlying the purported effects. Only a few have tested reasonable hypotheses for why educating boys and girls separately might make a difference for students’ outcomes. We will discuss possible mechanisms underlying any observed outcomes in single-sex contexts, and make suggestions for future research.