Why Its So Difficult To Measure Participation In The Arts

By Peter Linett, Senior Fellow, NORC at the University of Chicago and Co-Founder, Slover Linett at NORC

The report on findings from the 2022 Survey of Public Participation in the Arts (SPPA) calls attention to the growing proportion of Americans who say they’ve attended art forms or genres other than those listed in the survey, and it emphasizes the need to better understand those participation patterns in future waves of the study. The report’s authors envision commissioning qualitative research to more open-endedly identify the range of arts forms, genres, and experiences that Americans attend, thereby generating new items or response options to include in the next survey. That new research and revision of the questionnaire would extend the agency’s decades-long effort to widen the range of genres and participation modes measured in the SPPA. The earliest iterations of the survey, beginning in 1982, examined public engagement with jazz, classical music, opera, ballet, musicals and plays, and art galleries and museums—categories that came to be referred to as the “benchmark arts”(1)  — as well as literary reading. Viewed from our current vantage, with perhaps a more postmodern picture of culture (or rather, cultures) and certainly a greater alertness to racial, economic, and other kinds of marginali­zation in American systems, it’s astonishing that the early analysts of SPPA data could claim, unselfconsciously, that they were estimating “the size of the audience for the arts” full stop (italics added) on the basis of such a selective set of activities. (2)

That this selectivity went largely unnoticed at the time tells us a great deal about what the phrase “the arts” meant in that era, at least in a public policy context: the formal, institutional, professional arts, mostly derived from European forms and models; mostly subsidized by the federal government in the form of nonprofit status and charitable tax deductions; and mostly founded and led by (and still disproportionately serving) historically privileged socio-economic groups. In other words, the category was defined as the forms and settings of art that the dominant culture felt were vital enough to deserve support—and, perhaps, vulnerable enough to deserve protection and preservation. The earliest waves of the SPPA can be said to have been part of that apparatus of support and protection; we measure what we value.

In the years since then, though, the NEA’s research office has steadily widened the lens of the SPPA, adding new lines of inquiry to broaden both the what (i.e., art genres and forms) and the how (i.e., attending in person, viewing via media, actively creating yourself) measured in the survey. Questions have been added about arts education, some commercial genres, some community-based experiences, and even some adjacent forms of culture or creativity that may not be strictly artistic but which round out the picture of arts and cultural participation in the U.S. (e.g., reading nonfiction, touring historic sites, and coding software). Most recently, the 2022 question about in-person attendance at “other” music, dance, and theater performances, with its new prompts about hip-hop, country, rock, comedy, magic, and circus acts, seems intended to bring the study asymptotically close to comprehensiveness: to make all forms and genres of art count. (There’s a difference, however, between measuring a specific activity via a full survey item, on par with those benchmark art forms, and relegating it to an “other” response or follow-up prompt. And that difference is not merely analytical; it may convey to respondents something about the hierarchy of values underlying the research, subtly shaping their sense of the “right” or “best” answers to these already culturally freighted questions.)

But that progress toward an inclusive measurement frame for arts engagement raises additional questions about what is and isn’t covered. If attending live theater and watching or listening to plays on a device are both worth measuring, what about watching a well-crafted series on Netflix? If yes, which kinds of programs should count? Should all watching on streaming platforms count? Interestingly, Netflix has become a producer of live theater, first in London and now on Broadway, which makes the line even fuzzier: If the same production runs as a live play or musical and then as a show on Netflix, why should only one of those count as an arts activity in the SPPA?

Similarly, if weaving and quilting are worth measuring, what about creating authorized or unauthorized graffiti or street art; setting up a religious display in your apartment window or place of business; working with a tattoo artist to design your first ink; customizing and decorating your car, bike, or skateboard; or making a creative statement with your hair, jewelry, and fashion choices to express your personal style or cultural identity? Those examples all come from studies conducted by my research practice, Slover Linett, over the past two decades—studies in which we used qualitative methods to broaden existing programming or measurement frameworks of arts engagement for specific nonprofit cultural organizations or with support from national foundations. (3) In such studies, we (and presumably other arts researchers who have explored similar questions) have attempted to sketch a bottom-up picture or taxonomy of arts participation, in part as an empirical, demotic alternative to top-down, Western constructs of culture and the arts, with their histories of classism, racism, and colonialism.

It’s not coincidental that these studies tend to be conducted among populations in the United States whose heritage or identities have been marginalized by the country’s dominant racial, ethnic, and economic groups and systems. After all, such communities may have different arts and culture practices, priorities, interests, values, perceptions, and language-frames than those that have traditionally been taken into account in arts policy, practice, and measurement schemas. And that does seem to be the case, judging from the studies we’ve undertaken. Sure, (some) people do still (sometimes) attend concerts, plays, and museum exhibitions, but when we inquire open-endedly and agnostically about the range of arts and creativity experiences in their lives and communities, we tend to discover that those attendance-based forms of engagement are only part of—and not the most central, organic, or valued part of—the broader picture of participation.

These kinds of studies offer an important reference point for the NEA as it commissions the qualitative research mentioned above and considers adding categories to the SPPA. “Ideally,” the report’s authors write, “one would want to know—despite limited space on the NEA questionnaire, and risks of disrupting historical trend data—what percentages of adults engage with all the individual art forms currently excluded from the survey’s attendance module” [italics added]. That would be an ambitious undertaking in any case, and it is given added complexity and urgency by the NEA’s embrace of intersectional, instrumental, and community-centered work in and through the arts, as articulated at the “Healing, Bridging, Thriving” summit, held by the agency in January 2024 in partnership with the White House Domestic Policy Council.

What would a national arts research framework look like if it were constructed afresh with that mission and those values in mind, rather than the comparatively preservationist and traditionalist mission of the agency in the early 1980s? What kinds of arts experiences would need to be measured at those intersections and in those collaborative community settings? Would the notion of “attendance” still apply, and, if not, what would replace it? Perhaps most fundamentally, would it be enough to measure participation in such a framework, or would we also need to somehow measure the outcomes of that participation?

Part of the challenge is that such questions aren’t really empirical; they can’t be answered by just by asking Americans about their arts activities. Rather, they’re normative and therefore, inevitably, political. Take, for example, an arts category not discussed at that summit but very much in line with its arts-in-community and social-impact conversations: the ballroom scene. I’m referring not to the predominantly white partnered-dance tradition but to the African American and Latino queer community culture celebrated by Beyoncé in her 2022 album Renaissance and its subsequent tour and concert film, and by the current off-Broadway reimagining of the musical Cats. Gustavo Arruda, a young researcher with a health communications background who is now one of my colleagues at NORC, recently told me that the ballroom scene brings together dance competition, beauty pageant, rapping, DJing, fashion design, drag culture, and community health, all structured around informal but durably connected social groupings known as “houses.” (4) (Wikipedia describes houses as “chosen families of friends [who] form relationships and communities separate from their families of origin, from which they [as LGBTQ+ individuals] may be estranged.”) The health component often consists of free HIV and STI testing at the door, tables staffed by local nonprofit organizations offering counseling services, or other elements. According to Arruda:

Ballroom events, which arose as an alternative to the exclusion of LGBTQ+ people of color from traditional beauty pageants, later originated new art forms and aesthetics. “Balls”, as these events are also called, began to enable community exchanges of cultural, social, and even financial capital—since practicing together with one’s house and receiving health services or monetary prizes can extend the reach of ballroom events to everyday life. (5)

This intertwining of artistic performance, social services, community solidarity, self-expression, and what we might call political or cultural reclaiming isn’t unique to ballroom, of course; it’s characteristic of many community, Indigenous, and folk arts and cultural practices. But it is probably rare in most of the forms and genres of arts participation measured in the SPPA. Would participation in a ballroom event be captured by the SPPA’s existing questions about attending dance performances and social dancing? Perhaps, although for most people it’s not exactly “attendance,” and not everyone involved ends up dancing themselves—yet the people DJing, judging the competition, and staffing the nonprofit tables are very much participating in, indeed helping to create, this outcomes-rich arts event.

How important is it for future versions of the SPPA to measure those kinds of arts experiences not just obliquely, as “bycatch” in the larger net of arts categories, but directly, or even to name them along with other specific arts forms? If specific dance genres such as “ballet” and “salsa” are mentioned elsewhere in the survey, what does it mean that ballroom (in the above sense) and hip hop cyphers are not? Should there be questions about arts experiences that both emerge from and contribute to community, identity, and healing? The point is not to try to mention and measure every conceivable subgenre or setting of arts engagement; those constraints on questionnaire length are real, as are the imperatives of data comparability over time. The point is to take a step back from the received framework, which despite its many adaptations still carries the DNA of its origins in European aesthetic theory and 20th-century American cultural policy, and ask how the SPPA can more authentically represent both the lived experience of the arts in the lives and communities of all people in this country and the NEA’s strategic aspirations under its current leadership—including its new and future collaborations with other federal agencies and national stakeholders.

In addition to the qualitative research already envisioned, the agency may want to consider options such as experimental new survey modules, oversampling among members of marginalized communities, cognitive interviewing (ideally in multiple languages) as new survey items are developed, or commissioning one or more independent national surveys in parallel to the SPPA that would attempt to measure not just participation in, but also the perceived functions and outcomes of, arts experiences that may be community-based, identity-specific, issue-focused, or change-oriented. Such surveys could be methodologically “lighter” than U.S. Census Bureau-based data collection while still providing a rigorous, weighted population sample. They could even begin to measure related dynamics in the arts such as belonging, well-being, sociality, sense of place, and representation, either as benefits of or barriers to participation. More importantly, those parallel surveys would allow the Arts Endowment to build a data-picture that’s more illuminating and operationally useful, given its current activities, and perhaps a better advocacy tool for its dynamic vision of the arts in American lives and communities—all without interrupting the SPPA’s ongoing evolution toward greater inclusivity.

Perhaps the real question, then, isn’t just which genres of arts participation should be measured but also which purposes. How might the NEA work to broaden the range of intended impacts of arts experiences studied in the SPPA, just as it has broadened the range of arts forms, genres, and engagement pathways? How would that conceptual shift affect the kinds of questions asked in the survey? Could it help more Americans see themselves and their communities in the research? When I sat down to draft these reflections, I thought I would be writing about the SPPA’s “perimeter problem”: the question of where, exactly, we should draw a line around “the arts” for the purposes of this national study. What if it’s actually a lens problem, instead—a decision about how to look at arts engagement, how to view the engagers, and how to think about the broader human and social systems in which the arts operate? This may be an opportunity for the NEA to not just to update its flagship research project, but also to tighten the connections between research, policy, and practice for a new, more collaborative, communitarian, and intersectional era in the arts.

1. A decade of arts engagement: findings from the survey of public participation in the arts, 2002–2012 (NEA Research Report #58). 2015. National Endowment for the Arts. 

2. Robinson, John P. et al., Public participation in the arts: Final report on the 1982 survey. 1985. University of Maryland Survey Research Center for the National Endowment for the Arts.  

   The three studies referred to are:

   1. Ethnographic neighborhood study: Research findings and implications. 2009. Slover Linett Audience Research for Fleisher Art Memorial, with support from the Wallace Foundation.
   2. Exploratory qualitative research for the California arts participation survey development. 2013. Slover Linett Audience Research for the James Irvine Foundation and NORC at the University of Chicago.  
   3. Buyukozer Dawkins, M. et al. A place to be heard, a space to feel held: Black perspectives on creativity, trustworthiness, welcome and well-being—Findings from a qualitative study. 2021. Slover Linett Audience Research. Part of Culture + Community in a Time of Transformation: A Special Edition of Culture Track, with support from the Wallace Foundation, Barr Foundation, Institute for Museum & Library Services, and other funders. https://wallacefoundation.org/sites/default/files/2023-08/A-Place-to-Be-Heard-Black-Perspectives-on-Creativity-Trustworthiness-Welcome-and-Well-Being.pdf 
3.  Gustavo Arruda, 2024. Spoken communication with the author. See http://www.arrudafranco.com/.  
4. Arruda, 2024. Email communication with the author.
5. Arruda, 2024. Email communication with the author.