What 'Counts'?

In private conversation humanities scholars increasingly give voice to a strange confession regarding digital scholarship: "Actually, I'm O.K. with it, but the institution is not." Such a stark opposition between an assertion of individual progressiveness and a hesitation about institutional entrenchment hides a more complex story.

As institutions become increasingly open to new approaches, resistance to digital work still emanates more from a traditionalism rooted in departmental lore. It’s hard to change cultures, but academic publishing currently confronts a major structural transformation, and contributors as well as evaluators seek advice on how to assess digital projects. What steps should scholars, especially younger ones, take with their digital work to ensure that it will "count" toward hiring, promotions and tenure?

When I advise doctoral students assembling their dossiers for humanities teaching positions, most report great excitement about their digital projects, but remain uncertain whether to mention these for fear that they won’t "count" or that they may even count against their applications. Some tell me their advisers have encouraged such projects, but also doubted that one can be hired or promoted solely on the basis of digital contributions.

Given the questions of many young scholars and evaluations committees several guide have recently appeared. In 2011 the Modern Language Association devoted an issue of Profession to the "Question of Digital Evaluation."  Also in an effort to advance greater understanding of digital scholarship, The Journal of Digital Humanities has devoted its fourth and final volume of its first year to the topic "Closing the Evaluation Gap," with suggestions for rethinking the conventional peer-review process and avoiding comparisons to traditional scholarly book publications.

Eager to discover how institutions may consider forms of scholarship beyond traditional formats, I attended a pre-convention workshop at the 2013 Modern Language Association convention in Boston, entitled "Evaluating Digital Work for Tenure and Promotion: A Workshop for Evaluators and Candidates. "

Participants in this session were asked to form simulated third-year review committees evaluating a digital project. A myriad of tough questions about the value of digital work arose: Does the project have a scholarly argument? Is it original? Can one expect such criteria from a database? Must the project have a text-based argument? What sort of arguments do databases or other visual media presentations offer? What about the collaborative nature of many digital humanities (DH) projects? What makes a DH project scholarly as opposed to a service (administrative) or teaching contribution? Is this distinction still viable? Where is the originality in a group effort? What are the origins of the project, and in the case of a bibliography, how can a committee determine if it has been brought to completion? If the work remains in progress, how should it be evaluated? Since every DH project has contextual metadata and transcends traditional categories, how does it compare as a tool to new patents in the sciences?

Alison Byerly, a former provost at Middlebury College and the president-elect of Lafayette College, recommended new approaches to the work process: "If you’re reviewing a faculty member or a new hire’s work that was co-authored with others, it can be difficult to assess individual contribution. Therefore it’s imperative that each contributor keep good records of work done. Like scientists, digital humanists should be prepared to document their effort.”

Byerly also raised the question of the traditionally conservative nature of committees, where the emphasis on critical evaluation of individual accomplishment contrasts sharply with the "collaborative culture of digital humanities, where people work in teams, and readily promote and retweet one another.” Committees may not feel confident in their ability to judge unfamiliar work, and need both candidates and outside reviewers to make that work legible to them. Is it possible to maintain a critical distance from new projects that seem at the moment to spring from a small community of experts? Byerly includes a presentation here that she uses to advise faculty on digital evaluation.

Also addressing the collaborative nature of DH Bethany Nowviskie, director of digital research and scholarship at the University of Virginia Library and president of the Association for Computers and the Humanities remarked "the most important new duty of tenure and promotion committees [is] to avoid looking for the finished products of solitary genii and simple print-culture equivalencies (this work equals a book, that work equals an article), and instead to appreciate quality in digital scholarship as a set of evolving and open processes."

For faculty with little experience with DH, the task of evaluation can be daunting. When I queried colleagues in fields of literature, philosophy and history, most replied they felt as yet unqualified to offer advice let alone evaluate digital scholarship, and suggest I seek advice from more digitally active colleagues. A few, however, articulated a guarded openness to the imminent changes as well as quite a few words of caution.  Brian Leiter, director of the Center for Law, Philosophy and Human Values at the University of Chicago, reflected that any change in academic evaluation is glacial, that in philosophy DH remains nascent and that "even in law, I have yet to see a case where only digital scholarship was the basis for an appointment or tenure."

At Stanford University, many colleagues expressed openness to digital scholarship that could clearly demonstrate intellectual contribution. Franco Moretti, professor of English and founder with Matt Jockers, of the Literary Lab, suggests the main difficulty is merely lack of experience in evaluating collaborative projects "the only difficulty I see has to do with the collaborative nature of many DH projects. But the natural sciences have learned how to deal with that issue generations ago. Literature departments should simply copy what chemistry departments do, and save all of us a few million hours of discussions."

Dan Edelstein, associate professor of French and principal investigator for "Mapping the Republic of Letters,"  offered his "advice to graduate students: we're in a hybrid world. The brave new digital future has not yet arrived, and the venerable analog past is still alive and kicking." Advising students "not to put their scholarly eggs in one basket," Edelstein cautions students to continue traditional publications while developing new digital skills. He also speculates that at "some point in the near future, we may even stop speaking about DH as a separate field, and just revert to describing an enriched field known as the humanities."

Debra Satz, senior associate dean for the humanities and arts, advises students to learn from successful efforts in conventional publications and translate these to digital work. She also warns that simply adding technology will not guarantee the value of a new contribution:

My own advice would be for students thinking of using digital methods to learn from cases where this has worked [just as I advise my students to learn from conventional successful research projects] and also to learn from cases where it has not worked. There are plenty of examples where the use of technology has not added anything of value… I think the main way we will open doors to using digital methods is by the fruits of this scholarship. If it generates valuable new knowledge, then traditional scholars will come around.

Surely, digital humanities encompass a wealth of new knowledge, including innovative technical skills and hybrid approaches to teaching. Legitimation appears to be less an issue of content than form, because, for now anyway, the dominant idea remains that only a scholarly book published by an academic press "counts." At the same time, however, many view traditional academic books as an increasingly antiquated form of scholarship, particular primarily to literature and language fields. William A. Ladusaw, dean of humanities at the University of California at Santa Cruz and a linguist, offered a provocative position at the MLA workshop: "Most fields don’t require a book for tenure, and even in the humanities a lot of empirical work counts as well. In fact, as far as quality and audience are concerned right now, I’m more comfortable with a digital project than the monograph."

Comfort with digital projects arises from an understanding of the web and its offerings. As more academics use digital materials in the research, they also open the door for newer scholars and scholarship. Matthew G. Kirschenbaum, associate professor of English at the University of Maryland at College Park and associate director of the Maryland Institute for Technology in the Humanities, comments: "Today's Web depends and thrives on measures of influence, impact, and reputation. These are the very marrow of the medium. Why shouldn't scholars (and tenure committees) begin educating themselves about the judicious and proportional application of native Web metrics?"

Kirschenbaum also suggests looking at the DH Collaborators' Bill of Rights.

With these comments and resources I returned to my colleagues who had originally demurred when I asked them for advice on "what counts" in hiring and promotion. Most were convinced on-campus workshops for administrators, faculty and students would be greatly helpful to get "everyone on the same page."

In the meantime, there are also several resources to consult including the MLA guidelines for digital evaluations. Adeline Koh offers some helpful advice in a blog post. In a future where some faculty may come up for tenure solely on digital humanities publications, these are the five most important steps that faculty need to take regarding the use of digital media:

• All levels of the university need to be educated about digital media, so that departments and administration understand the spectrum of digital projects they will encounter.
• Departments should to keep a set of sample tenurable projects to provide multiple models and analogies for new DH work.
• Enlist the expertise of library, administrative and technology staff to make all materials, technology, and technical language easily accessible and understandable to all potential reviewers.
• Just like their colleagues in the sciences, DH scholars need to keep clear records of their contributions to collaborative projects and write up comprehensive narratives of the project’s inception and fruition.
• New hires should ask that their initial employment contracts explicitly recognize the validity of DH work as a basis for future renewals and promotions.