We share our thoughts on various events surrounding the working paper by Sabyasachi Das, erstwhile faculty member at Ashoka University.

The debate around it has now turned from the correctness of the paper to the issue of academic freedom. It has been argued by a number of academics, rightly so we believe, that academic freedom needs to be preserved from external pressures.

It is important because the freedom to ask uncomfortable or unconventional questions is essential to lead researchers to new discoveries and/or improve our understanding of some phenomena.

As any freedom is required to be exercised responsibly, the world of academia has evolved the system of peer-review to validate the results and conclusions claimed by researchers.

Under this system, a researcher submits a new result to a journal, which is then reviewed and validated by a set of domain experts, and if found correct is published in the journal.

With the advent of the Internet, however, this process has undergone some changes. Researchers now upload their work on web portals, in addition to submitting to journals, which makes the work immediately available to the worldwide community. If the work is potentially of sufficient importance, the community does not wait for journal review process, and immediately sets about collectively validating the result to establish its correctness.

Of course, the journal review process still retains its pre-eminence. However, some journals are now experimenting with more open and web-centric review process (eLife is an example).

Another important aspect of the process is that the credentials of the researcher hardly matter; rather, it is the quality and correctness of result that counts.

The review process generally works well. We give four examples of this spanning over nearly five decades.

In 1970, one of the authors solved a key problem in control theory. At the time, he had just finished his PhD and therefore, was relatively unknown. In the pre-internet days, communication was through post, and he sent his preprint to Professor Charles Desoer at UC Berkeley, one of the stalwarts of the domain. Within a couple of weeks Professor Desoer responded praising the result and making some suggestions. The paper was peer-reviewed and published in a leading journal after a year.

Our second example is from 2002, when the internet age had begun. Another of the authors solved an important problem in number theory and mailed a preprint to a few people including a couple of well-known number theorists, Professors Hendrik Lenstra and Carl Pomerance.

The author was as outsider to the number theory community, and the two number theorists who received the preprint had never heard of him. Yet, within a couple of days they not only read and validated the result, but also informed their colleagues, and within a week the result was widely acknowledged as a significant achievement. The actual peer-review happened much later and the journal version of the paper appeared four years later!

The third example is from 2010, when Dr. Vinay Deolalikar, a researcher at HP Laboratories, shared a preprint with some prominent researchers that claimed to settle the long-standing question of whether or not P equals NP (one of the "Millennium Problems" for the solution of which, a prize of $1 million is offered).

Immediately, the research community got to work, and within four weeks or so, serious errors were found in the work, and the preprint was ignored thereafter.

Our final example is the very recent claim of superconductivity at room temperature in a material dubbed as 'LK-99'. The arXiv preprints appeared on 22 July, 2023. A number of labs across the world went ahead with attempts to replicate the claimed results, and the scientific community reached a consensus by mid-August, or within about three weeks from the initial claim. The consensus was that "it is an insulator in pure form, and not a superconductor at any temperature".

It is important to note that, in the last two cases, the reputations of the scientists involved did not suffer much, if at all. Scientists recognize that failures pave the way to an ultimate success. Moreover, unless there was outright fraud involved, the general reaction of the research community can be summarized as, "Better luck next time."

Let us now examine the work of Dr Das in light of the above discussion.

Das claimed that the BJP had engaged in "registration manipulation," i.e., the manipulation of electoral rolls, as a part of its victory in the 2019 Lok Sabha polls. This is quite a sensational claim, deserving wider attention since it, if true, undermines faith in the electoral process – a key pillar of a democratic system.

One would have expected the Indian economics (and statistics) community to critically examine the analysis in the preprint (called a working paper in economics community) and check whether it is correct. Nothing of the sort happened.

Instead, the first serious attempt to critically examine the work was done by an anonymous X (formerly Twitter) handle by the name "Saiarav,” who raised a whole host of questions about the work, including the fact that the data used and/or the code used to do the computations were not made publicly available.

Motivated by this critique, Dr Mudit Kapoor, an economics professor in ISI Delhi, clinically dismantled the work by showing that statistically, one cannot establish that there was any malpractice.

However, several economists continued to argue that until the peer-review process is completed, one cannot dismiss the work. As a preprint takes 3-4 years in economics domain, this means that the results cannot be rejected for several years!

Faced with this situation and intense public scrutiny demanding quicker resolution, Ashoka university first put out a public statement and then setup a committee to advice the university on how to proceed (sources from the university state that both were done in consultation with Dr Das and several others). This led to the resignation of Dr Das and an uproar on academic freedom from fellow economists.

The are several important conclusions one can draw from the above episode.

First, the Indian economics community, with some honourable exceptions, abdicated its responsibility in providing a thorough and timely review of the preprint.

Second, many in the community seemingly have let their political preferences influence their assessment of the work.

Third, in the present internet age, it is completely unrealistic to expect that tentative results are not rejected based on the preprint alone. So the stand of many in the economics community is out of step with the times.

Fourth, and most importantly, the compact between society-at-large and the academic world on academic freedom rests on the academic community undertaking and completing its self-review at a speed commensurate with times. If there is a mismatch, society at large will draw its own conclusions.

Eventually this might lead to a curtailment of academic freedom, which will be a highly undesirable outcome. The responsibility for avoiding such an eventuality primarily rests with the academic communities.

Manindra Agrawal and M. Vidyasagar

Get Swarajya in your inbox.