What and Who Do We Trust?
While many conversations of this type are heard among parents and students and amateur musicians with personal biases, I have to admit that we as directors of show choir competitions have not done an acceptable job in providing students and teachers with valuable and reliable feedback from a standardized score sheet. From week to week, show choirs are being evaluated on different score sheets with different point systems and categories, and few if any attempt to qualify scores in any sort of objective manner. While adjudicators strive to be as consistent as possible, it is difficult when many score sheets do not qualify what kind of performance is deserving of 6s and 7s, rather than 9s and 10s.

Furthermore, why do we often use a ten-point scale with categories that are all worth the same number of points? Wide gaps in scores between judges could suggest that they may be unqualified to adjudicate, but more likely indicates the lack of any sort of validity or common standard. In the fall of 2016, I decided to develop a rubric as a part of a doctoral project, in the hopes of improving adjudicating measures in my own state of Mississippi.

Weighty Matters: Inconsistency Breeds Frustration
In order to create a weighted rubric, I first had to determine what categories should receive the most weight. I developed a survey that asked raters to rank the typical subcategories in order of importance, most to least. The only exception was the Overall Effect category, in which respondents chose the most important of the three subcategories. The survey link was emailed to show choir directors, choreographers, and adjudicators within the Southeastern region of the United States. A link was also posted on my personal Facebook page, as well as the group pages, “Show Choir Directors and Choreographers” and “I’m A Choir Director.”

Of the seventy-four people who responded to the survey, 75% reported that they were currently teachers in a K-12 music program and 74% reported that they currently direct a competitive show choir. The teachers were equally distributed between the South and Midwest regions of the U.S. Teachers were given the opportunity to answer an open-ended question regarding the parts of the current scoring methods with which they are most dissatisfied. Some general themes were revealed in this question. The most common source of dissatisfaction is the inconsistency of scoring rubrics between competitions and the lack of a more universal scoring method.

In addition, teachers feel that the current methods do nothing to attempt to alleviate adjudicator bias and subjectivity. Another common theme was the qualification, experience, and training level of the adjudicators. Teachers were also asked to rank in order of importance the categories from a typical score sheet.

Knowing It When We See It: Defining the “It”
Though the survey results certainly informed decisions made regarding category weights, my own experience as an adjudicator and director also contributed to the development of the rubric. I also spent time interviewing trusted colleagues—fellow directors and adjudicators—in person and through email.

Those reviewing the rubric generally agreed upon the resulting rubric, although interesting discussions and arguments were made for and against the inclusion and weights of some categories. Furthermore, the operational definitions for each subcategory and qualifications of each were more difficult than anticipated. Though we “know what it is when we see it,” putting that into a text narrative that is detailed in what differentiates a 9-10 performance from a 7-8 performance was time-consuming and thought-provoking.

Exploring Outcomes and Implications
This project began as a way to improve the adjudication procedures of show choir contests. The finished product, however, has implications for use other than that of its original intent. As discussed earlier, competitive show choir is a wonderful environment for exploring educational outcomes. Students who participate in this particular arena have the potential to become not only outstanding musical performers, but also educated critics of musical performances. Evaluative tools are useful outside of the role of the adjudicator.

In trying to improve adjudication methods, the educational value of scores and feedback should be kept at the forefront of the discussion. A more reliable and less subjective scoring process can ensure that students receive valuable and consistent feedback on how to improve, even if they do not receive a trophy. Keeping education at the center of adjudication makes the competitive experience meaningful regardless of the final placement.

Unresolved Issues
While this rubric arguably moves toward improving standardization and fairness in show choir adjudication, it does not resolve all of the issues with show choir contest adjudication. Most of the survey respondents were from two regions of the United States, the South and the Midwest. Show choir in these regions is similar in nature, but for East and West Coasts, values and scoring methods may differ greatly. This rubric may not be representative of the aspects of performance valued in those geographic regions.

The issue of adjudicator training cannot be resolved with this rubric. Many survey respondents suggested that a main problem in the realm of competitive show choir is the lack of qualified and adequately-trained judges. The hiring and training of judges is the responsibility of each contest producer. Opinions on this matter vary widely and would be a good subject for future research.

The most obvious limitation to the rubric is that it is reflective of my own personal philosophies. Though the survey certainly informed the creation of the rubric, I placed a higher value on some aspects than what the survey results indicated. As musicians, we have personal preferences that are reflective of our own beliefs and experiences with music. These constructs played an influential role in the creation of this rubric.

Stepping in the Right Direction
While further research and discussion is needed to continue to improve adjudication measures in the realm of competitive show choir, improved assessment tools are a step in that direction. Rubrics provide specific feedback that allow for subjectivity while providing reliable justification of objective numerical scores. This specificity allows for greater discussion in the classroom and more distinction between strengths and weaknesses of a performance. I am hopeful that as the rubric is put into practice, it will be discussed and edited to make it an effective tool that is widely utilized.