Qualitative research as public scholarship

Originally posted at www.21stcenturyscholar.org

At this year’s AERA conference, Bill Tierney and I presented a paper, “The Role of Ethnography as Ethical and Policy-Relevant Public Scholarship.” We had a great panel, including Rob Rhoads, Jessica Lester, Laurence Parker, and Yvonna Lincoln. Fellow blogger Antar chaired. Michelle Fine acted as discussant, providing great commentary.

The idea for the symposium developed after Bill sent a link to an article in The Chronicle about Goffman’s On the Run. If you remember, last year, I blogged about the book. Rather than focusing narrowly on Goffman’s research—many people have already critiqued her work—the session focused broadly on concerns of conducting ethnography as public scholarship.

What is public scholarship? Stated simply, it is a scholar’s engagement with multiple publics in order to inform social issues, provoke civic participation, and promote social justice. Typical examples include writing nonfiction books, appearing on NPR, and creating policy reports. This blog is a form of public scholarship. Unconventional acts—although, certainly not rare—include teaching courses, participating in local political movements, and conducting participatory action research. The difference between the first and second categories depends on the scholar’s level of engagement. A nonfiction book creates a one-way conversation from researcher to reader. Organizing a neighborhood-based planning committee or providing a summer outreach program involves collaborative engagement.

Qualitative research is particularly well-suited for more participatory examples of public scholarship. For years, the Pullias Center, based on research, provided mentoring services for students in Los Angeles. They even created and shared an infographic (another form of public scholarship). And, among other examples, they have developed apps and games to reach more students.

Public scholarship, as an abstract concept, seems noble and harmless. Why wouldn’t scholars want to advocate for social justice and facilitate deliberative democracy? In practice, it is considerably more complicated. While some (particularly those who believe in positive science) may disagree, all research is a political act. To engage with multiple publics draws attention to its political nature and makes researchers vulnerable to critique. That makes some uncomfortable.

For qualitative researchers, whose work is often misunderstood or dismissed, public scholarship poses even more risks. Think about two examples:

• A group of university researchers conduct a large-scale experimental study. Education Week writes about it. The study gains national attention. Policymakers use the findings to argue for reforms. Later, statisticians at a think tank contest the validity of the findings. Debates ensue about the researchers’ methodological decisions.
• A researcher conducts a five-year ethnography. The scholar publishes a book that becomes a NY Times best-seller. She performs a popular TED talk and appears on national news outlets. Journalists and scholars begin to critique her work. Discussions transition from the topic of the book to the qualifications of the researcher.

How do the two scenarios differ? There is often a degree of separation between quantitative researchers and their studies. Critics may read the methods section and think, “That’s a terrible design.” But, they typically don’t denigrate the researchers. The same separation does not exist for qualitative scholars. If critics perceive a problem, they often target the methods and the researchers. At a certain level, this makes sense. The researcher is the instrument. However, having witnessed enough controversies, the discussions often become personal, not professional. Instead of discussing flaws in the methods, critics target defects in the researcher.

Still, qualitative research as public scholarship is important and necessary. It contributes unique and grounded perspectives and contests deleterious stereotypes. It also has the ability to incite change.

The question then becomes, how might qualitative researchers improve the utility of their research as public scholarship and, relatedly, establish standards and techniques to enhance the quality of their work. These are a few of the issues Bill and I address in our paper. We plan to revise it this summer and invite feedback. If you have suggestions, let me know via comments, email, or Twitter.

Rush to relevance: Conducting research to improve policy and practice

“We need research to be more relevant” is a common clarion call in education. Most recently, John Easton, Director of IES, released a video for AERA in which he talks about different initiatives to improve relevancy.

During one of my first Ph.D. courses, Bill asked us about the three major responsibilities of academics: research, teaching, and service. In particular, he wondered about teaching, something that is too often overlooked at prestigious universities. As someone who just left the high school classroom, I boldly proclaimed—in a way only a first-year Ph.D. can—that all professors should be required to teach, and to teach well. Students pay a lot of money and deserve more than someone who views teaching as a chore to be completed before doing important work. As Bill pointed out then, and I have thought about since, academics have different skill sets. Some are unbelievably talented teachers, others researchers. Let’s propose the best possible scenario: Young scholars receive effective mentorship and professional development and work at universities that support and reward teaching and learning. Even then, some will never be effective teachers. What’s my point? Bold proclamations often fail to account for important nuances.

Just as I wanted all celebrity researchers to be all-star teachers, I think, at a certain level, all research should be relevant. But, the rush to relevancy has a few troubling side effects. First, it ignores the value of various forms of research. A new set of guidelines illustrates the types of methodologies—namely experimental and quasi-experimental—that IES favors (and funds) to produce relevant research. Large-scale, experimental studies are important. If I want to understand how growing up in a low-income neighborhood effects social mobility, neighborhood effects studies provide compelling, significant findings. What else do they provide? A lot of unanswered questions. The Moving to Opportunity (MTO) study, funded by HUD in the 1990s, was an innovative randomized experiment. Researchers sought to understand what happened to low-income families when they received housing vouchers. Scholars are still arguing about the study’s effects on variables like educational achievement, employment, and health. Even the best studies fail to capture the complex and dynamic social processes at work in low-income neighborhoods. While a life history of one or an ethnography of sixty may not generalize, they can provide valuable findings. Understanding social issues requires sophisticated, complimentary methodologies.

Second, the rush to relevance may diminish academic freedom. Foundations and think tanks like and support trendy issues. Schools of education like grant funding. Inevitably, these factors lead to subtle (or not-so-subtle) nudges to Ph.D. students and early career faculty: “Yes, you are free to study whatever you want, but we hope it will be relevant and also generate funding.” Considering fewer and fewer tenure-track positions, the pressures are magnified. Some scholars maintain balanced research portfolios, examining both mainstream and non-mainstream topics. But, even in those circumstances, we diminish the vibrancy of national debates by settling into familiar paths. 

Third, and related to the above point, relevance relies on ever-changing interests. Last year, we talked about MOOCs. This year, we are talking about Common Core. Too often, scholars sacrifice their own interests to pursue those of others. I want to be clear here: I am not arguing against forward-thinking scholars or the study of emerging, innovative issues. Rather, policy windows open and close quickly. Thinking of the most pressing, deeply rooted issues in education, neither scholarship nor meaningful reform benefit from that same hectic agenda. Instead of racing to relevance and the allure of fame and funding, researchers ought to use their own experiences and expertise to help define what is and is not important.

Open access and scholarly publishing

Napster operated from 1999 to 2001. Twenty-six million users joined the peer-to-peer file sharing service. From B-sides to bootlegs, the digital venue allowed music lovers to share rare songs and provided unprecedented access to diverse forms of music.

Record label executives rebelled. Users, they argued, were stealing music. They targeted the college student who, with no regard for the artists’ (and industry’s) labor, amassed a 500 GB music collection. And yet, by the time Metallica, Dr. Dre, and other wealthy industry figures shut the service down, the change had already happened. The top-down corporate structure could not hold. 

Napster represents the ethos of a tech-savvy, connected generation. New technologies, along with sharing tendencies, have challenged issues such as access and ownership. 

Changing production and distribution channels have influenced all forms of media. Apple and book publishers are now battling allegations of e-book price fixing. Microsoft and Sony have both announced next-gen gaming consoles. A key issue among gamers has been rumors about game sharing and “always on” consoles. Gamers worry that corporations will use technology—the same technology that has allowed mass sharing—to limit access. PlayStation, unsurprisingly, has taken the early lead by charging less, limiting restrictions, and supporting indie game developers.

In academia, open access journals have increasingly dotted the publishing landscape. AERA recently announced a partnership with Sage to launch AERA Open, a peer-reviewed open access journal. It will allow all online users to view and read articles. To defray costs, AERA Open will charge authors a fee: AERA members will pay $400 whereas non-members will spend $700.

AERA’s forward thinking decision to embrace open access recognizes the shifting dynamics of knowledge production and distribution and raises a number of important and interesting questions: How will analytics affect journal and article rankings? Will crowdsourcing become more influential? And, how will open access modify the ways in which people find articles and value research topics? 

Open access journals present opportunities and challenges for graduate students and early career faculty members. For individuals like myself, I am drawn to new forms of distribution and presentation. Some fear—like record industry execs did of Napster—that open access will ruin the knowledge economy and devalue the work of academics. I disagree. Higher education is changing. Fear of uncertainty is not an acceptable reason to uphold the status quo and limit free access to knowledge. Open access potentially allows wider readership and improves visibility. 

The new trend encompasses several risks. For tenure-track faculty, a level of uncertainty exists regarding rigor. Although faulty, impact factors are important to tenure. Scholars make their names by publishing in top-rated and topic-specific journals. Even though the peer review process makes open access far different from pay-to-publish journals, many deans may not be able to differentiate between the two. Authors, consequently, have to choose and advocate wisely. As Bill suggested during an AERA governance meeting, while impact factors may not be available, other metrics such as accept / reject rates provide solid evidence to tenure committees.

Overall, I believe the benefits of open access outweigh the risks. The primary challenge for journal editors will be to maintain high standards, provide a quick peer review process, and present articles in a compelling and logical manner. The test for academics will be to embrace new technologies and practices.