Article vs journal

The incidental journalist.

By Albert Cardona, June 2nd, 2016.

Scientific journals have solved the many-to-many distribution problem for several centuries. Before that, academics would pen letters to each other; an approach to scholarly communications that is pretty much the opposite of broadcasting. Back then, the only way to get an update on new scientific findings was to become a member of a scientific society and to attend its meetings. The written proceedings of such society meetings might have been among the first scientific publications created for the purposes of broadcasting and archiving academic work [1].

The concept of a bunch of papers bundled together as a unit of distribution transformed science communications, and enabled both scientists and enthusiasts to receive updates at regular intervals, even if these spanned a whole year or more. An often overlooked side effect of the emergence of journals is the filtering: the power to limit which works get included in the periodical issue. This is of enormous consequence, and is tied together with the arrival of a new actor: the editor.

Granted that, before editors, there were other gate keepers. For example, whichever figure controlled membership to the society, or regulated the number of attendees to a meeting [2].

The role of the editor was a positive one: to organize the proceedings. Whether that originally included any type of fact-checking and proofreading I don't know. In considering that not even the English language itself was standardized, proofreading seems unlikely. What it most certainly did not include was formal peer-review [3]. One can imagine how, at the very least, peer-review existed in the form of verbal exchanges during the society meetings, and in the copious penned letters that scientists exchanged. The time scale for scientific communication was far larger than nowadays.

Fast-forward to modern times, the editor has remained a powerful figure, perhaps even more so than in ages past. With power comes corruption, and the desire to preserve the status quo. Resistance is futile; the environment enables and therefore it is only a matter of time until someone does something questionable at the expense of scientific progress.

As the scientific practitioners expanded and the disciplines fragmented, more and more journals emerged to serve the needs of the new communities. Why multiple journals emerged to serve the same field of research might be rooted in different causes for each case, but clearly, the old versus the new generation is an old theme in humanity, summarized by the famous pithy saying that science advances one funeral at a time (attributed to Max Plank).

With a diversity of journals serving the same field, the opportunity for ranking emerged. The game for becoming top dog started. Some journals gathered a larger readership and came to be regarded as more prestigious. All this before the advent of the impact factor, which merely provided a specific metric to optimize for, with a veneer of academic legitimacy far more appealing than merely counting subscribers.

The academic publishing ecosystem was set. Inevitably, parasitism emerged. There were resources that could be harvested by freeloaders.

During my postdoc years, my adviser organized a journal club on "development and disease". Every other week we would read a paper--or rather, scrutinize--and then someone would present it and we would all comment. The natural mistake was to pick articles from well-regarded journals; we are all guilty of that. The eye-opening experience for me was that, over several years of bi-weekly meetings, not one published paper survived the joint scrutiny of a dozen smart graduate students and postdocs. Not one. Most papers had a number of minor issues, and a few had several major ones, which clearly flew through peer review. Science self-corrects eventually, but the major message here is that there is a need for post-publication peer review.

Some papers defied explanation. I recall in particular a paper that presented negative results, and not very interesting ones, and yet had been published in one of the top-3 most prestigious journals. It occurred to me to look up the senior author, and lo and behold, was part of the editorial board of that journal. It turned out that the first author had no other papers after 5 years of postdoc, and basically, we speculated that the editor must have used insider knowledge to craft a manuscript that could become published in the journal, successfully associating the first author with the perceived glamour and prestige of the journal. This is parasitism.

How much parasitism can a journal sustain before noticing a dent in its metrics for impact? A lot. The impact factor is a mere average, heavily affected by strongly cited papers, and not much at all by weakly cited ones, given the non-linear distribution of paper citations. The h-index similarly ignores the long tail of weakly cited papers. There is a lot of room for parasites to join the bandwagon.

Which brings me to the point of this essay: papers versus journals. Whoever in their right mind thought that assessing the impact of a journal was worth their time? Turns out, it was the logical thing to do, given that subscription costs are per journal, not per article, and all that Eugene Garfield was aiming for in devising the impact factor metric was to optimize the library's budget allocation.

The right approach to the measurement of scientific impact might be individual article-level metrics, as endorsed by the DORA declaration. But these are not without problems. For one, each field of research has a different number of practitioners, and therefore the opportunities for citation will vary. And fields are not sufficiently separable to be able to conclusively estimate the denominator for the normalization. Additionally, some types of papers are more citable than others, for example reviews, in particular when faced with space limitations. Far more important is that some papers create new fields all by themselves, and their impact will only become evident decades afterwards.

The one winning aspect of emphasizing papers over journals is the death of parasitism [4]. No more riding on the wings of others.

An additional, crucial advantage of papers over journals is that the paper is the unit of publication. Some journals do not seem to understand this, offering in their web pages to download "just this PDF" or "this PDF plus all others in this issue". I do not know of anybody who ever selected the latter option other than by accident.

To further remark that I don't know anybody who reads an "issue" anymore. The search engine, the machine-learning based automatic suggestions, and the content aggregators such as social media feeds have killed that concept for good. Only relics, in the form of printed magazines sitting around discarded, offer the opportunity on an idle moment for a serendipitous encounter with a fascinating paper we would have otherwise likely never have read. Twitter and other social media, though, nowadays play this role quite well.

The paper as the unit of publication brings us back to the origin of journals. Is the need for a journal still here, to solve the all-to-all distribution problem? The internet has solved the distribution aspect, and the problem has mostly transmuted into one of discoverability: finding the right paper in an ocean of publications.

What role, then, is left for a modern journal? Primarily the moderation of private peer-review. This could best be achieved not by journals as they exist today, but by the Reviewer Entities proposed by Yann LeCun. These entities have also been called overlay journals, because they operate by selecting papers from public preprint repositories such as the arXiv and bioarXiv, and publish public reviews referencing the papers, rather than the papers themselves. The curated lists of reviewed papers, grouped by community or scientific discipline, could entirely replace existing journals [6]. These entities would therefore be providing the necessarily private yet moderated peer-review. And also some amount of discoverability to papers, similarly to how the "Insight" or "News and Views"-style write ups of prestigious journals contextualize research, highlight their impact and make it more accessible to outsiders or the uninitiated.

The Reviewer Entity model, in my view, is the future. It decouples publication (immediate, with a preprint) from evaluation (which occurs concurrently in many places) and, crucially, from the glamour, which originates in the choosiness of the editors/reviewers and is based on the public reviews rather than the publication server. But perhaps even more important, this decoupling enables tackling a major issue with the current journal model of scientific publication: it enables manuscript versioning, making obsolete the mostly invisible erratum of today. The arXiv and bioarXiv are already plucking this low-hanging fruit to great utility [5].

With the disappearance of the bundle of papers, the paraphernalia of "issue", "number" and "pages" would disappear, in favor of a system that exists today: the digital object identifier. The first page of a paper should always be page 1.

The well-intending figure of the editor ended up creating the opportunity for corruption and parasitism in the evaluation of science. With modern technology--the internet--and several centuries of hindsight, there are clear ways forward to improve scientific communication dramatically. Of course, the existing system constraints the launching of the next, particularly given its undeniable strong ties to academic promotion and success in grant applications. Relinquishing power is not something humans are good at.

May we have the strength to stop judging books by their covers and start reading the papers.

Notes

  1. The Philosophical Transactions of the Royal Society in the UK is credited with being the first scientific journal, publishing its first issue in 1665.
  2. The definition of an "in" group relative to an "out" group runs deep in the ancestral organization of human societies, the tribe.
  3. Formal peer-review moderated by editors as it is understood today might have arisen around the early 1900s, famously raising the fury of Einstein who was pissed that an editor elected to send his manuscript to a colleague/competitor of his for comment without his permission.
  4. At least of this particular type of parasitism. Where there is an energy gradient, someone will find a way to exploit it.
  5. Much more could be done in manuscript versioning, a la github, but not all scientists know or need to know the kind of fine-grained version control generally applied in software writing.
  6. The move from traditional journals with printed issues to online overlay journals implies a fundamental change in the position of a paper in the publishing structure: instead of the paper being a leaf in the tree (the journal is a tree, the issue is a branch, the paper is a leaf), the paper would then be a multiply tagged item, each tag defining one of the many collections interested in the paper. Think labels in gmail, with which an email can belong simultaneously to many different lists. In other words, the paper would move from a fixed position in a rigid hierarchy to a fluid position in multiple overlapping collections. The latter seems far more appealing towards better defining new communities of scientists, in particular multidisciplinary ones.