Are Academic Conferences Broken? Can We Fix Them?

I’d hoped to get through all of the SIGIR 2009 Industry Track before blogging about anything else (such as Yahoo! search going bada-Bing), but clearly I’m taking too long. So I’m following Daniel Lemire’s suggestion that I post a recent comment on Lance Fortnow’s blog (actually a response to his CACM column entitled “Time for Computer Science to Grow Up“) here at The Noisy Channel.

It’s nice to see this piece joining a growing chorus questioning the way we conflate the distinct concerns of disseminating knowledge, establishing professional reputation, and building community. This problem is not unique to computer science, but we are certainly in a position to lead by example in addressing it.

In age where distribution is nearly free, I agree that we should move the filtering role from content publishers to content consumers. There’s no economic reason today why scholarship (or purported scholarship) shouldn’t be published online. Of course, the ability to publish digital content for free (or close to free) does not imply anyone will (or should) read what you write. The blogosphere offers an instructive example: the overwhelming majority of blogs attract few (if any) readers. I . …

I’d hoped to get through all of the SIGIR 2009 Industry Track before blogging about anything else (such as Yahoo! search going bada-Bing), but clearly I’m taking too long. So I’m following Daniel Lemire’s suggestion that I post a recent comment on Lance Fortnow’s blog (actually a response to his CACM column entitled “Time for Computer Science to Grow Up“) here at The Noisy Channel.

It’s nice to see this piece joining a growing chorus questioning the way we conflate the distinct concerns of disseminating knowledge, establishing professional reputation, and building community. This problem is not unique to computer science, but we are certainly in a position to lead by example in addressing it.

In age where distribution is nearly free, I agree that we should move the filtering role from content publishers to content consumers. There’s no economic reason today why scholarship (or purported scholarship) shouldn’t be published online. Of course, the ability to publish digital content for free (or close to free) does not imply anyone will (or should) read what you write. The blogosphere offers an instructive example: the overwhelming majority of blogs attract few (if any) readers. I suspect that the same holds true for arXiv.org. Of course, peer-reviewed content may not fare that much better, particularly given the proliferation of peer-reviewed venues. Regardless, it makes no sense for publishers to act as filters in an age of nearly-free digital distribution.

That brings us to the question of how researchers should establish their professional reputation–and, in the case of academics, obtain tenure and promotion. Today, they have to publish in peer-reviewed journals and conferences. Even if we accept the weaknesses of the current peer-review regime, we should be able to separate content assessment from distribution. The peer-review process (and review processes in general) should serve to endorse content–and ideally even to improve it–rather than to filter it.

Finally, conferences should primarily serve to build community. I find the main value of conferences and workshops to be face-to-face interaction, and I’ve heard many people express similar sentiments. Part of the problem is that so few presenters at conferences invest in (or have the skills for) delivering strong presentations. But more fundamentally it’s not even clear that the presentations are the point of a conference–after all, an author’s main motive for submitting an article to a conference seems to be getting it into the proceedings.

Here are some questions I’d like to suggest we consider as a community:

What if presentation at a conference were optional, and an author’s decision to present had no effect on inclusion in the proceedings? Would there be significantly fewer presentations? Would those fewer presentation be of higher quality?

What if the process of peer-reviewing conference submissions required the submission of presentation materials rather than (or in addition to) a paper? Would the accepted presentations be of higher quality? Would researchers invest more in presentation skills? What would happen to strong researchers without such skills?

Can we update the traditional conference format to foster more productive interaction among researchers? For example, should we have more poster sessions and fewer paper presentations?

I’d love to see the computer science community take the lead in evolving what increasingly feel like dated procedures for disseminating knowledge, establishing professional reputation, and building community. I’ve tried to do my small part, co-organizing workshops on Human-Computer Interaction and Information Retrieval (HCIR) that emphasize face-to-face interaction and organizing the SIGIR 2009 Industry Track as a series of invited talks and panels from strong presenters. But I’m encouraged to see “establishment” types like Moshe Vardi and Lance Fortnow leading the charge to question the status quo.

Link to original post