MLA Profession 2011: On the Evaluation of Digital Media as Scholarship

My paper “On the Evaluation of Digital Media as Scholarship” has just been published online in MLA: Profession 2011 (pp. 152-168). The PDF is freely available. The abstract reads,

As more and more scholarship is digital, we need to develop a culture of conversation around the evaluation of digital academic work. We have to be able to evaluate new types of research, like analytic tools and hypermedia fiction, that are difficult to review. The essay surveys common types of digital scholarly work, discusses what evaluators should ask, discusses how digital researchers can document their scholarship, and then discusses the types of conversations hires and evaluators (like chairs) should have and when they should have them. Where there is a conversation around evaluation in a department, both hires and evaluators are more likely to come to consensus as to what is appropriate digital research and how it should be documented.

This is part of a collection put together by Susan Schreibman, Laura Mandell and Stephen Olsen about Evaluating Digital Scholarship. McGann and Bethany Nowviskie, among others, also have papers in this issue of Profession.

Issuu – You Publish

Thanks to Sharla I came across Issuu a site for publishing online magazine like documents. You get an account, you upload documents, and they create an interactive page-flipping e-publication out of it. When you “Click to read” a publication an application takes over your screen to give you a reading environment. They seem to have a lot of publications made available this way.

The Shape of Things to Come — Rice University Press

Jerome McGann and Rice have published the Proceedings of The Shape of Things to Come — Rice University Press in record time. The conference took place in March and the essays are now up on line and coming out in print. (See my blog entry and conference report on the original conference.)

“Certain questions,” Jerome McGann writes in his introduction, “are especially insistent: How do we sustain the life of these digitally-organized projects; how do we effectively address their institutional obstacles and financial demands; how do we involve the greater community of students and scholars in online research and publication; how do we integrate these resources with our inherited material and paper-based depositories; how do we promote institutional collaborations to support innovative scholarship; how do we integrate online resources, which are now largely dispersed and isolated, into a connected network?”

My paper is titled As Transparent As Infrastructure. I argued that there has been a turn to infrastructure as a way to get sustained funding for things, but that don’t really know what infrastructure is and we are tempted to turn into infrastructure things that are still being negotiated.

Google Book Search Settlement

The Google Book Search Settlement, if approved by Judge Chin, may be a turning point in textual research. In principle, if the settlement goes through, then Google will release the full 7-10 million books for research (“non-consumptive”) use. Should get even the 500,000 public domain books for research we will have a historic corpus far larger than anything else. To quote the Greg Crane D-Lib article, “What can you do with a million books?” and “What effect will millions of books have on the textual disciplines?”

There is understandably a lot of concerns about the settlement especially about the ownership of orphan works. The American Library Association has a web site on the settlement, as do others. I think we need to also start talking about how to develop a research infrastructure to allow the millions of books to be used effectively. What would it look like? What could we do? Some ideas:

  • To be only usable by researchers there would have to be some sort of reasonable firewall.
  • It would be nice if it were truly multilingual/multicultural from the start. The books are, after all.
  • It would be nice if there was a mechanism for researchers to correct the OCRed text where they see typos. Why couldn’t we clean up the plain text together.
  • It would be nice if there was an open architecture search engine scaled to handle the collection and usable by research tools.

Update: Matt pointed me to an article in the Wall Street Journal on Tech’s Bigs Put Google’s Books Deal In Crosshairs.

Clay Shirky: Newspapers and Thinking the Unthinkable

Thanks to Peter O I came across Clay Shirky’s excellent analysis of what’s going on with newspapers and the web, Newspapers and Thinking the Unthinkable . Some of the salient points:

  • Change has been so rapid that it has changed who is pragmatic and who is a fabulist. Newspapers are in denial about the realities of online content so those who describe what is happening (the pragmatists) are treated as fabulists.

Revolutions create a curious inversion of perception. In ordinary times, people who do no more than describe the world around them are seen as pragmatists, while those who imagine fabulous alternative futures are viewed as radicals. The last couple of decades haven’t been ordinary, however. Inside the papers, the pragmatists were the ones simply looking out the window and noticing that the real world was increasingly resembling the unthinkable scenario. These people were treated as if they were barking mad. Meanwhile the people spinning visions of popular walled gardens and enthusiastic micropayment adoption, visions unsupported by reality, were regarded not as charlatans but saviors.

When reality is labeled unthinkable, it creates a kind of sickness in an industry.

  • The economics of publishing have changed. It used to be that there was a tremendous upfront cost to set up a newspaper or broadcasting facility. Now the infrastructure of distribution is paid for by all so publishing is cheap.

With the old economics destroyed, organizational forms perfected for industrial production have to be replaced with structures optimized for digital data. It makes increasingly less sense even to talk about a publishing industry, because the core problem publishing solves — the incredible difficulty, complexity, and expense of making something available to the public — has stopped being a problem.

  • It is easy to describe life before or after an epochal shift. It is hard to describe the chaos of experiments during the shift. Shirky looks to The Printing Press as an Agent of Change as an example of the hard type of history.

What Eisenstein focused on, though, was how many historians ignored the transition from one era to the other. To describe the world before or after the spread of print was child’s play; those dates were safely distanced from upheaval. But what was happening in 1500? The hard question Eisenstein’s book asks is “How did we get from the world before the printing press to the world after it? What was the revolution itself like?”

  • Advertisers don’t want to pay for the costs of a full-featured newspaper (with international bureaus and investigative reporting.) They will move their money to where it connects with their (usually local) audience.

The competition-deflecting effects of printing cost got destroyed by the internet, where everyone pays for the infrastructure, and then everyone gets to use it. And when Wal-Mart, and the local Maytag dealer, and the law firm hiring a secretary, and that kid down the block selling his bike, were all able to use that infrastructure to get out of their old relationship with the publisher, they did. They’d never really signed up to fund the Baghdad bureau anyway.

  • Newspaper reporting provides a public service that will be missed, but knowing we will miss it doesn’t save it. We just don’t know how to fill the gap that will be left when daily papers dissappear in cities.

“You’re gonna miss us when we’re gone!” has never been much of a business model. So who covers all that news if some significant fraction of the currently employed newspaper people lose their jobs?

I don’t know. Nobody knows. We’re collectively living through 1500, when it’s easier to see what’s broken than what will replace it.

Actually I think there are ideas floating around as to what might fill the gap:

  • Blogs may take up some of the slack with various advocacy groups and NGOs providing investigative reporting in the areas that concern them. I think it is wrong to assume that amateurs will necessarily do a worse job than professional reporters. In fact, as most know, professionals are too busy to usually go into depth and whenever they write about something you know they get it wrong in all sorts of ways. A blog like Buckets of Grewal probably does a more indepth job of examining the Grewal controversy than any newspaper story. The difference is rather that the professionals are committed to breadth and they write better.
  • Publicly funded broadcasters like the BBC and the CBC will provide tax funded news reporting with foreign bureaus and so on. They don’t have to have make a profit and can invest in things perceived as useful for society.
  • There will always be some big and international newspapers like the New York Times or Reuters because there will always be a demand for that sort of news. The internet reduces diversity – every city doesn’t need a newspaper with a foreign bureau. All we need is a couple of news services with foreign bureaus.
  • Some companies have already figured out how to package news as analysis and get other businesses to pay for it. This will accelerate as newspapers fail. Companies like Oxford Analytica will meet the demand of multinational businesses who need access to strategic information. The sooner the newspapers fail the sooner we will see these companies come out of the woodwork and start selling their products to us.

To conclude with another quote from the Shirky essay, “Society doesn’t need newspapers. What we need is journalism.”

MIT: Open Access Mandate

From Twitter I found a link to Peter Suber, Open Access News where there is a story of MIT’s unanimous faculty vote to adopt an Open Access mandate that gives the University the right to archive and make available their articles. Like the Harvard mandate, faculty can opt out if they have a good reason.

Each Faculty member grants to the Massachusetts Institute of Technology nonexclusive permission to make available his or her scholarly articles and to exercise the copyright in those articles for the purpose of open dissemination.

Gary Hall: Digitize this book

A couple of weeks ago I posted a blog entry about Gary Hall’s book Digitize This Book! I noted that I couldn’t find a digitized copy of the book and asked if others knew of one. To my surprise Gary wrote me back and pointed to the items listed below. Now that is the Internet at work! He is trying to get the publisher to allow a digital copy to be posted online, but in the meantime pointed out online versions of what became chapters in the book:

(2003) ‘Digitise This’, Mediactive, Vol. 1, No. 1 (pp. 76-90); republished in (2004), The Review of Education, Pedagogy and Cultural Studies, Vol. 26, No. 1, January-March (pp. 23-46) at http://scm-rime.tees.ac.uk/VLE/DATA/CSEARCH/MODULES/CS/2006/03/0147/_.doc (MS Word Document)

(2007) ‘IT, Again: or, How to Build an Ethical Virtual Institution’, in Experimenting: Essays With Samuel Weber, edited by Gary Hall and Simon Morgan Wortham (Fordham University Press: New York) (pp.116-140) at http://scm-rime.tees.ac.uk/VLE/DATA/CSEARCH/MODULES/CS/2008/01/0740/_.doc (MS Word Document)

Gary Hall says that “Since the book came out I’ve also published a new piece on open access publishing and the humanities” at http://www.culturemachine.net/index.php/cm/issue/current. A video of him presenting it as a talk is available at Pirate Philosophy – Steal This! .

I take back any irony in my previous post. (Can one take back irony? Perhaps I can only apologize for being ironic to early.)

Hall: Digitize This Book!

Cover of BookDigitize This Book! by Gary Hall is an interesting book at the intersection of cultural studies and humanities computing. The book seems to be addressed mostly to the cultural studies crowd arguing that “do cultural studies writers, thinkers, and practitioners not also need to experiment with ways of being ‘militant’ in a positive, innovative, creative, and constructive fashion in their own situations, institutions, and places of work?” (p. 206) The book is a sustained defense of the Cultural Studies e-Archive (CSeARCH) and other computing projects that Hall has initiated. He is trying to make space in cultural studies for projects we would recognize as humanities computing projects. To do this he argues against “transcendental politics” which assume a commitment to a particular political analysis in order to open room for actions, like starting an open archive, that cannot be demonstrated a-priori to be in support of capitalism or not. He ends the book with,

A fixed, pure and incorruptible institution could only be a violent, transcendental, totalizing, and totalitarian fantasy. One could even argue, after Derrida, that it is precisely the structurally open and undecidable nature of the situation – the fact that an institution or archive can be used to facilitate the forces of capitalism and globalization – that gives it ethical and political force. (p. 214)

Now I tend to shudder when I read phrases like “the forces of capitalism”, partly because I don’t understand the tradition of thought that takes such things as givens, but I don’t, as many colleagues do, believe we should therefore shun cultural studies or other forms of post-modern thought. Hall is interested in something important and that is the ethics and politics of digital work. To avoid discussing the ethics and politics of what we do in the university or as developers of digital works is to ascribe to a naive and unexamined ethic. Many avoid politics because the discourse has been politicized by second rate cultural studies folk who think shaming others for not being militant is a form of engagement. Hall is trying to open room for a form of politics beyond politics (or hyperpolitics) where we can act without knowing for sure what the consequences of our actions will be. That is the heart of ethics for me, acting (or not, which in turn is a form of acting) in the face of insufficient knowledge or ability. We always do things without being sure, ethics is knowing that and trying to deal thoughtfully with the ignorance.

Part of what I am saying here, then, is that certain forms, practices, and performances of new media – including many of those associated with open-access publishing and archiving – make us aware that we can no longer assume that we unproblematically know what the “political” is, or what sorts of interventions count as political. (p. 196)

Hall in his actions (like CSeARCH and the Open Humanities Press) and in his writing is trying to reach out to those in open access circles and in computing circles. We who are too buried in the techne should reach back.


You can find earlier versions of sections on CSeARCH like The Cultural Studies E-Archive Project (Original Pirate Copy), but, ironically, I can’t, find a copy of Digitize This Book!. No one has bothered to digitize it, no doubt due to the copyright notice as the beginning (p. iv) that states,

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher. (p. iv)

Is there a contradiction between the injunction of the title (“Digitize This Book!”) and the copyright notice? What is the status of a title when it comes to rights? Should I digitize the book?

To be fair to Hall, the chapters of his previous book, Culture In Bits are available on CSeARCH and I assume he will make Digitize This Book! also available after a suitable interval. Perhaps someone knows him and can update me or point me to a digitized version already open.

Note: since writing this someone passed on a note to Gary Hall who kindly pointed me to online copies of other chapters. See my more recent blog entry with the links.


Hall makes an interesting move at the beginning of the book to position open access as a middle way for the university between the commercialization of the university and the (impossible elitist) return to whatever it is we think we were doing in the humanities in the good old days. I find it interesting that Hall believes “cultural studies has for some time now arguably been the means by which the university thinks about itself …” (p. 13). I’ve seen no evidence of this – cultural studies to me seems to want to position itself as outside the university critiquing it in the Socratic gadfly tradition rather than taking a role acknowledged by the university. It would probably come as a surprise to most university administrators that cultural studies is doing this for them and somehow represents the university’s institutionalized reflection. And therein lies the promise of Hall’s book – that there is type of creative activity we can all engage in, through which we can imagine the university by modeling it. We don’t need approval to set up open works. We can use the technology to become a way for the university to think about itself.

Ithaka: Sustainability and Revenue Models for Online Academic Resources

The Ithaka organization has released a report on the Sustainability and Revenue Models for Online Academic Resources with support from the Strategic Content Alliance and JISC in the U.K. The report deals with the difficult question of how to sustain all the free online resources we have built in the first enthusiasm of the web.

There is no single formula that Online Academic Resources (OARs) can apply to achieve sustainability, no ‘one-size-fits-all’ plan that any organization can follow to reach a point of financial stability. There are, however, a variety of processes and approaches that can help to improve the likelihood of entrepreneurial success. In an age when traditional content producers – including scholarly publishers and newspapers – struggle to maintain their financial footing in face of the challenges of the digital world, OARs cannot turn to lessons of the past to find their way, but must see themselves as nimble players in a quickly shifting field.

Part of the problem is that we think of the digital as if it were a grant project with a print outcome. You do the research, you develop the resources, you publish it and then you move on. Digital publication seems to be cheaper and faster than print, but the true cost is the sustainability. You can get it up faster, but then you have to maintain it forever. The report argues that the problem is that academics, as smart as they are, don’t know how to think like entrepreneurs.

Clearly the leaders of these initiatives are competent professionals; why do they not rely on processes that have proven effective in both commercial and not-for-profit contexts? We have concluded that a key reason for this is that academic researchers tend to approach these problems from a different perspective, and with a different mindset, than do commercial entrepreneurs. (Page 5)

For this reason the report presents an entrepreneurial start-up model which excludes academics who can’t focus soley on a project (which is most of us):

Running a start-up is a full-time job and requires full-time leadership. The mode of principal investigators, in which they divide their time between overseeing a variety of research grants, teaching courses, and other responsibilities, is not conducive to entrepreneurial success. New initiatives aiming for sustainability require fully dedicated, fully invested, and intensely focused leadership. If a principal investigator cannot provide it, he or she will have to retain a very capable person who can. (Page 7)

This is the second time in a week or so I have heard people calling for the professionalization of academic resource development (the other time being at the Tools for Data-Driven Research meeting where the view was voiced that tool development should be taken out of the hands of the academics.) Reading the report I wonder what the role of academics in scholarly resources is, if any? It reminds me of calls for MBAs to run universities rather than academics. I wonder what it would look like to apply the logic of this report to the university itself (as a type of institution.) I think it fair to say that the university has clearly proven to be longer lasting (more sustainable) than commercial enterprises. For that matter ask how many software companies still exist ten years later (see my blog entry on In Search of Stupidity, over 20 years of high-tech marketing disasters). To be fair I think the report is looking at models for large-scale academic resources like online journals and other non-profit resource organizations that are often run by professional staff already.  Hereis a list of their major points:

  1.  Most OAR projects should not assume ongoing support from initial funders.
  2. Sustainability plans must include and provide for resources to support future growth.
  3. OAR projects create value through the impact they have on users.
  4. Projects should think in terms of building scale through partnerships, collaborations, mergers, and even acquisitions.
  5. In a competitive world, strategic planning is imperative.
  6. OAR leaders must see both the needs of users and the competitive environment as dynamic and constantly changing.
  7. OAR leaders must become fully accountable both to their projects and to their funders.
  8. Catalysing a dynamic environment for agility, creativity, risk-taking, and innovation is imperative.

While I am skeptical of the entrepreneurial thinking the report starts with we can learn from these points about sustainability by looking at the issue from an entrepreneurial perspective still stands. We can and should think about the long term sustainability and we can learn from other perspectives.

The really useful part of the report is “Section 4: Revenue Generating Options for OAR  Projects” which systematically discusses direct and indirect ways of generating revenue including the much avoided approach of allowing ads into academic sites.

Information Overload and Clay Shirky

Peter sent me to Clay Shirky’s It’s Not Information Overload. It’s Filter Failure talk at the Web 2.0 Expo in New York which starts with a chart from a IDC White Paper showing the growth of digital information. His title summarizes his position on the issue of Information Overload, but on the way he made the point that we have been complaining about overload for a while. To paraphrase Shirky, “if the problem doesn’t go away it is a fact.” Shirky jokes that the issue comes up over an over because “it makes us feel better” about not getting anything done.

I, like others, have used the overload meme to start talks and am now wondering about the meme. Recently I was researching a talk for CaSTA 2008 that started from this issue of excess information and found that Vannevar Bush had used overload as the problem to drive his essay, “As We May Think” in 1945.

There is a growing mountain of research. But there is increased evidence that we are being bogged down today as specialization extends. The investigator is staggered by the findings and conclusions of thousands of other workers—conclusions which he cannot find time to grasp, much less to remember, as they appear. Yet specialization becomes increasingly necessary for progress, and the effort to bridge between disciplines is correspondingly superficial.

Professionally our methods of transmitting and reviewing the results of research are generations old and by now are totally inadequate for their purpose. (Vannevar Bush, As We May Think)

If Shirky is right that this is a fact, not a problem, and that we default to using it to leverage ideas as solutions, then we have to look again at the perception of overload. Some of the questions we might ask are:

  1. What is the history of the perception of overload?
  2. Is it something that can be solved or is it a like a philosophical problem that we return to informatics as a ground for discussion?
  3. Have structural changes in how information is produced and consumed affected our perception as Shirky claims? (He talks about FaceBook being a structural change for which our balancing filtering mechanisms haven’t caught up.)
  4. One common response in the academy is to call for less publishing (usually they call for more quality and less pressure on researchers to crank out books to get tenure.) Why doesn’t anyone listen (and stop writing?)
  5. What role do academics play in the long term selection and filtering that shapes the record down to a canon?