In a telling exchange with Nature.com, Google Scholar co-founder Anurag Acharya cited academic publishers as the roadblock behind Google Scholar’s failure to offer a programmable interface.
Q [Nature]: Many people would like to have an API (Application Programming Interface) in Google Scholar, so that they could write programs that automatically make searches or retrieve profile information, and build services on top of the tool. Is that possible?
A [Acharya]: I can’t do that. Our indexing arrangements with publishers preclude it. We are allowed to scan all the articles, but not to distribute this information to others in bulk. It is important to be able to work with publishers so we can continue to build a comprehensive search service that is free to everybody. That is our primary function, and everything else is in addition to this.
In other words, don’t hold your breath waiting for Google to add powerful new functionality to Scholar anytime soon.
The good news is that this glaring weakness, that Scholar lacks a programmable interface, creates an opening for a different sort of scholarly-search engine that can both help fill this void and put pressure on academic publishers to open up their content.
Here’s how that might work:
STEP 1 – Scholar alternative [“Delphi”] is built
An alternative scholarly search engine – let’s call it “Delphi” – is introduced that aims to provide the programmable searchability that Google Scholar lacks. To be clear, Delphi is NOT designed to compete with Scholar. People will continue to go to Scholar whenever they have a specific scholarly search. Delphi’s advantage will instead lie in allowing scholars to conduct programmable searches on the subset of scholarship that Delphi indexes.
STEP 2A – Open-access journals adopt Delphi
Unlike Google Scholar, which attempts to automatically scan all scholarship, Delphi is an “opt-in” platform that only indexes a subset of scholarly outlets. In particular, it only indexes the articles of academic journals/repositories that have taken steps to opt into the Delphi platform. At first, only open-access outlets will choose to opt-in.
STEP 2B – Scholars adopt Delphi for programmable searches
Scholars have an incentive to use every tool at their disposal, no matter how limited, as long as it makes them more productive. Once an API exists to automatically search even a fraction of relevant academic articles, scholars will have an incentive to use that API to learn about recent papers that are of interest to them.
STEP 2C – Delphi creates new distribution channel, encouraging entry of innovative new journals
By enabling automatic scholarly search, Delphi cracks the traditional “chicken-and-egg problem” of academic publishing, that an outlet can only be successful once scholars in a field all know about it. With Delphi, a newly published paper that is highly relevant to some scholar will get their attention, even if they have never heard of the outlet where it has been published, as long as that outlet is indexed.
STEP 3A – Authors prefer to publish in Delphi-indexed publications
Publishing your work in a Delphi-indexed publication will get your research noticed by others working in your field. As a result, Delphi-indexed publications will generate more citations. This extra visibility will boost scientists’ careers and reputation, giving authors a reason to favor outlets whose publications can be automatically searched.
STEP 3B — More and more publishers opt to be indexed by Delphi
As Delphi-indexed publications receive more visibility and more citations, academic journals that submit to Delphi indexing will see their own reputations improve. The resulting “race to the top” will put pressure on academic publishers who insist on boycotting the Delphi platform.
STEP 4 – Paywall publishers change their position; now support a Google Scholar API analogous to Google Books API
The rise of Delphi will put pressure on paywall publishers to offer open programmable search of their content. Rather than joining Delphi’s relatively radical open model, however, these publishers’ natural move will be to retain as much control as they can, by pushing Google Scholar to change and become more like Google Books.
Google Books offers free snippets, fully searchable content, and a programmable interface, while still allowing publishers to charge for full access to content. Allowing a similar amount of access to their content would “take the heat off” paywall publishers by reducing their citation disadvantage relative to Delphi-indexed outlets, while still allowing them to keep their paywall-based business-model intact.
The end result?
In the end, paywall publishers will be compelled to loosen their grip, a bit, on the scholarship that they control. More than that, however, an openly-searchable research platforms like Delphi can empower an alternative publication model that leverages programmable search to distribute the latest scholarship and advance the frontier of knowledge faster than ever before.
I think that’s a vision worth working towards.
Update: This post featured on ICIS blog (Innovating Communication in Scholarship)