Over the last two years, I’ve spent a lot of time talking about MediaCommons, the digital scholarly publishing network that I’ve been developing with my collaborator Avi Santo — and in every conversation that I’ve had about the project, with scholars both senior and junior, both beginning and established, one question has repeatedly resurfaced: what are you going to do about peer review? I don’t want to dismiss these questions out of hand; concerns about peer review are understandable, given that peer review is in some sense the sine qua non of the academy. We employ it in almost every aspect of the ways that we work, from hiring decisions through tenure and promotion reviews, in both internal and external grant and fellowship competitions, and, of course, in publishing. The work we do as scholars is routinely subjected to a series of vetting processes that enable us to indicate that the results of our work have been scrutinized by authorities in the field, and that those results are therefore themselves authoritative.

But as authors including Michael Jensen of the National Academies Press have recently argued, the nature of authority is shifting, and shifting dramatically, in the era of the digital network (see Jensen). Scholars in media studies have avidly explored such shifts as they affect media production, distribution, and consumption, focusing on the extent to which, for instance, bloggers have decentralized and even displaced the authority structures surrounding traditional journalism, or the ways that a range of phenomena including mashups and fan vids have disrupted the previously assumed hierarchies that existed between media producers and media consumers, or the growing tensions in the relationship between consumers, industries, and industry regulators highlighted by file-sharing services and battles with the RIAA. These changes are at the heart of much of the most exciting and influential work in media studies today, including publications such as Siva Vaidhyanathan’s The Anarchist in the Library, Henry Jenkins’s Convergence Culture, and Yochai Benkler’s The Wealth of Networks, projects that have grown out of an understandable interest in the extent to which the means of media production and distribution are undergoing a process of radical democratization in the Web 2.0 era, and a desire to test the limits of that democratization. To a surprising extent, however, scholars have resisted exploring a similar sense in which intellectual authority might likewise be shifting in the contemporary world. One might see such a resistance manifested in the often over-blown academic response to Wikipedia, which often indicates a serious misunderstanding about the value of the project. Treating Wikipedia like any other encyclopedia, by consulting only the entries, as John Seely Brown has been heard to say, runs the risk of missing the point entirely, as the real intellectual heart of the project lies on the history and discussion pages, where one can see the controversies inherent in the production of any encyclopedia entry enacted in public, rather than smoothed over into an untroubled conventional wisdom (see Brown). Centralized projects like Citizendium, which seek to impose traditional, hierarchical modes of authority on a site like Wikipedia, ignore the fact that, first, the wiki is in its very architecture a mode of ongoing peer review, and second, that not only the results of that review but the records of its process are available for critical scrutiny. Failing to engage fully with the intellectual merits of a project like Wikipedia, or with the ways in which it represents one facet of a far-reaching change in contemporary epistemologies is a mistake that we academics make at our own peril. As one librarian suggests, “Banning a source like Wikipedia (rather than teaching how to use it wisely) simply tells students that the academic world is divorced from real-world practices” (Badke, qtd, in Regalado). The production of knowledge is of course the academy’s very reason for being, and if we cling to an outdated system for the establishment and measurement of authority at the very same time that the nature of authority is shifting around us, we run the risk of becoming increasingly irrelevant to the dominant ways of knowing of contemporary culture.

For this reason, what I am absolutely not arguing is that we need to ensure that peer-reviewed journals online are considered of equivalent value to peer-reviewed journals in print; in fact, I believe that such an equation is instead part the problem I’m addressing. Imposing traditional methods of peer review on digital publishing might help a transition to digital publishing in the short term, enabling more traditionally-minded scholars to see electronic and print scholarship as equivalent in value, but it will hobble us in the long term, as we employ outdated methods in a public space that operates under radically different systems of authorization. Instead, we must find ways to work with, to improve, and to adapt those new systems for scholarly use — but we must also find ways to convince ourselves, our colleagues, and our institutions of the value that is produced by the use of such systems. Such is the focus of the book-length project I’m currently working on: not the technical changes that I’d argue are necessary for ensuring that scholarship survives well into the twenty-first century, but rather the institutional and social changes that must precede such technological change in order for it to take root. In the process of writing the chapter that focuses on peer review, however, I’ve discovered that there’s been surprisingly little scholarly exploration of the history and function of peer review in the humanities, in contrast to the overflow of such studies in many of the social and natural sciences. To some extent, this discrepancy may have to do with methodological differences in these fields; studies of peer review often involve a form of data-gathering and analysis that isn’t part of the usual humanist skill set. But there may also be a more troubling factor involved in the dearth of work on intellectual authorization in the humanities; critical studies of the epistemological practices of peer review require a form of self-analysis that, as Donald Hall has argued in The Academic Self, many of us resist. Our resistance might suggest an underlying anxiety about the outcome of the analysis, a potential concern that the time-honored procedures and standards that guide our work might be shown to be flawed — and thus that the humanities might be even further marginalized within the academy’s mission of knowledge-production than they already are. However, as Hall argues, genuinely “owning” our careers and the ways in which we conduct them requires taking the risk of applying our critical skills to an examination of “the textuality of our own profession, its scripts, values, biases, and behavioral norms” (Hall xiv). In the academy — as goes the joke about defenders of tradition in many realms — too many attitudes may be summed up in a mere eight words: “we have never done it that way before.” The apparently intractable nature of the way things have always been done is precisely the kind of signal that, in other institutions, impels us to critical analysis; a refusal to turn such a critical eye on our own seemingly naturalized assumptions may create (or deepen) an atmosphere of intellectual oppression and stultification, as we allow systems in which we do not genuinely have faith to dictate our engagements with the world, and with one another. Opening up the basis of those engagements through a thorough reconsideration of peer review may be precisely what we need in order to allow our work to help shape the ways of knowing of the contemporary world.

Resistance to considering the merits of a mode of publishing freed from the gatekeeping function of peer review often runs something like that expressed — in, I assume, an intentionally hyperbolic fashion — by David Shatz: It is hard to say who would have the biggest nightmare were open review implemented: readers who have to trek through enormous amounts of junk before finding articles they find rewarding; serious scholars who have to live with the depressing knowledge that flat earth theories now can be said to enjoy “scholarly support”; or a public that finds the medical literature flooded with voodoo and quackery. Let us not forget, either, that editors and sponsoring universities would lose power and prestige even while their workload as judges would be eliminated. (16) The vehemence of such resistance reveals something about the nervousness of those who express it, and as in much psychotherapeutic discourse, it’s only after some initial projection and displacement that the real source of that anxiety comes out: the loss of “power and prestige.” Not only does peer review as it is currently practiced ostensibly ensure that only the best work makes it into circulation — already a debatable assumption, and one that has been subjected to much critique in other fields — but gatekeeping itself is a source of significant privilege.

It’s in this regard that Mario Biagioli, one of the few humanists to take on a thorough critique of assumptions about peer review, compellingly argues for an understanding of the practice as not simply productive of the borders of an academic discipline, but as itself a disciplinary technology in the Foucauldian sense, a mode of defining what is thinkable that is “simultaneously repressive, productive, and constitutive” of academic ways of knowing (11). He pertinently points out a key distinction between Foucault’s discplinary reference points in medicine and the prison, however, and the discipline of peer review, as only in the academy do we find “that the roles of the disciplined and the discipliner are often reversed during one’s career” (12). Peer review thus functions as a self-perpetuating disciplinary system, inculcating the objects of discipline into becoming its subjects. After all, those who manage the current system of peer review are of course those who have successfully negotiated it, granting an enormous inertia to the status quo. Of course, there have been a number of experiments with changing the structure of peer review, particularly within the sciences, including the extremely cautious open review trial conducted by Nature, which, as I’ve written elsewhere, seems to have been intentionally set up to fail (see Fitzpatrick). Alongside this trial, however, the editors of other journals such as Electronic Transactions on Artificial Intelligence and Atmospheric Chemistry and Physics described their long-standing review processes that take advantage of the interactive technologies of the web in order to blend open, collegial discussion of submitted papers with more traditional editorial vetting. These innovations have resulted in two-stage processes that serve both the quality-control purposes scholars expect of peer review and the crucial communication amongst peers that traditional review often elides.

I’d argue that it’s this communication, and the scholarly growth that can result from it, that must become the focus of web-native modes of peer review, allowing, as does Wikipedia, not just the results of our research and vetting processes but the very processes themselves to become an open, accessible part of the published record. Noah Wardrip-Fruin’s recent experiment with the community-based peer review of his book manuscript, “Expressive Processing,” for instance, suggests the value for authors in making such processes public. Wardrip-Fruin, in his reflections on the review process, noted that “the blog-based review form not only brings in more voices (which may identify more potential issues), and not only provides some ‘review of the reviews’ (with reviewers weighing in on the issues raised by others), but is also, crucially, a conversation” (Wardrip-Fruin). As a conversation, blog-based review can determine the status of scholarly work not through its successful navigation of a gatekeeping process, but rather through the networked interactions of its authors and readers, thus replacing the mere existence of the published text as the dominant sign of its authority with the reception and uses of that text.

The key to such new modes of authorization is a shift from traditionally understood peer review to peer-to-peer review. In conventional peer review, the “value” of texts is determined through a process of gatekeeping designed for an economics of scarcity, in which a limited number of pages, or journal issues, or monograph volumes can be published; these constraints require that publishers ensure that resources be reserved for only the very best material. Peer-to-peer review acknowledges that the internet exists instead within an economics of abundance, in which there is no upper limit on the number or size of texts that may be published. What has become scarce, instead, is time and attention, and what is thus needed is not gatekeeping but filtering, a community-based process in which groups of scholars determine for themselves the most important texts in their subfield. As Cory Doctorow has said of the digital sphere, it “isn’t a tragedy of the commons; this is a commons where the sheep s*** grass — where the more you graze, the more commons you get” (Tweney). The appropriate mode of dealing with such abundance isn’t fence-building, but rather careful community cultivation.

I do not claim to know exactly what a successful peer-to-peer review process might look like as yet; what seems certain, though, is that the community must precede, and therefore guide, the development of the process. This is our hope at MediaCommons: that a community of scholars within a particular field might come together to determine in an emergent fashion what its values and standards are, and how best to cultivate the field. The catch, of course, will be articulating those values and standards, as well as the processes by which they are determined, well enough that the various credentialing bodies to which we as scholars are subject — tenure and promotion committees perhaps most significantly among them — understand that the absence of conventional peer review’s binary model of quality does not imply the absence of quality, but rather the adoption of a more appropriate model of intellectual authorization for the network age, one that will allow scholarly work to interact with the digital public rather than hiding within the walled gardens created by traditional structures of authority.

Works Cited

Biagioli, Mario. “From Book Censorship to Academic Peer Review.” Emergences 12.1 (2002): 11-45.

Brown, John Seely. Panel presentation, Claremont Graduate University colloquium on Social Entrepreneurship and Design, 1 June 2007.

“CZ:About.” Citizendium. . Accessed 16 February 2008.

Fitzpatrick, Kathleen. “MediaCommons: Scholarly Publishing in the Age of the Internet.” MediaCommons (29 March 2007). .

Hall, Donald. The Academic Self: An Owner’s Manual. Columbus: Ohio State University Press, 2002.

Harnad, Stevan. “Learned Inquiry and the Net: The Role of Peer Review, Peer Commentary, and Copyright.” Learned Publishing 11.4 (1998): 283-292.

Jensen, Michael. “Authority 3.0: Friend or Foe to Scholars?” Journal of Scholarly Publishing 39.1 (October 2007): 297-307. Rpt. of “The New Metrics of Scholarly Authority,” Chronicle of Higher Education 53.41 (15 June 2007): B6.

Regalado, Mariana. “Research Authority in the Age of Google: Equilibrium Sought.” Library Philosophy and Practice (2007). .

Shatz, David. Peer Review: A Critical Inquiry. Lanham, Md.: Rowman & Littlefield, 2004.

Tweney, Dylan. “Q&A: Cory Doctorow.” SFGate (23 January 2003). .

Wardrip-Fruin, Noah. “EP Meta: Chapter Four.” Grand Text Auto (16 February 2008). .