How can we combat scientific fraud?

The rise in the number of publications during the health crisis and increasing pressure on researchers have severely tested scientific integrity. In doing so, they have revealed the extent and diversity of the phenomenon of fraud in all its different forms. Often with the help of new digital tools, scientists, publishers and research bodies are counterattacking – showing at least as much ingenuity as the fraudsters themselves!

Scientific fraud in all its diversity

From methodological errors to blatant trickery, scientific misconduct comes in different forms and has varying degrees of seriousness.

Duplicated data, misleading reports, improbable results and, finally, incorrect conclusions about ways to reduce the risk of bone fractures… Between 1996 and 2013, Japanese doctor Yoshihiro Sato engaged in massive fraud. And there were repercussions, since his dubious clinical trials influenced medical advice for years. Those trials were also the basis for many papers published in international journals, sixty of which have since been ‘retracted’ – in other words withdrawn by their publishers. Even so, a retracted paper often remains available online, if only in the context of research into fraud. Yoshihiro Sato was a perfect example of a repeat offender, guilty of every type of misconduct. What misconduct? Firstly, fraud in the strictest sense, i.e. plagiarism and the deliberate fabrication or falsification of data. Then ‘questionable’ practices: embellishing or concealing data, republishing a paper, using erroneous quotations or references, and incorrectly attributing responsibility for research – for instance by describing a head of department as the co-author of a paper solely on the basis of their status as line manager. Finally, ‘bad science’ or poor-quality science, which is often unintentional. It results from methodological errors or misinterpreted data and is actually thought to be a minority cause of retraction. Assessed case by case, misconduct can lead to the correction or retraction of a paper, or even disciplinary action, depending on its seriousness. As for fraud, it is comparable to forgery and can result in criminal charges for those responsible.

Scientific integrity challenged by the pandemic

Weighing the need to inform against the necessity of good quality control, researchers were dealing directly with decision-makers and the media during the pandemic.

At the peak of the health crisis, it was difficult to achieve the right balance between rapid access to new scientific conclusions and the risk of dubious clinical decisions based on inaccurate papers. Given the conflicting demands at the time, did scientific fraud increase during the SARS-CoV-2 pandemic? The French Office for Scientific Integrity (OFIS) feels it is too early to judge: papers may have been studied – and so filtered – more than usual. But during that period, research – whether reliable or dishonest – was carried out and shared at an unprecedented pace. So much so that papers submitted to prepublication sites (prior to acceptance by a journal’s editorial committee) were prematurely disclosed to the public by communicators and journalists hungry for news; for example nearly 4,000 papers on Covid-19 were pre-published on the medRxiv site in June 2020. As for the publishers of the scientific press, they had to relax their assessment procedures to deal with both the emergency and the vast number of Covid-19 papers submitted – for instance by bringing forward their deadlines for review. That in itself was enough to harm the quality of the process – but only to a certain extent, it seems. Nearly 230 publications related to the coronavirus were retracted during the pandemic – most of them in 2021 – but that only represents 7% of the 3,200 retractions recorded in the Retraction Watch site database for 2021.

 A recent regulatory initiative in France

France recently introduced a national legal definition of scientific integrity.

Fraud must now be reported to enable monitoring of the phenomenon. While it was previously a question of ‘good practice’ left to the discretion of each establishment, scientific integrity is now a legal concept in France. The Research Programming Law (LPR) of December 2020 made it part of the Research Code. Then the decree of 3 December 2021 set out clear obligations for public bodies and foundations whose main activity is public research. Among those requirements, the appointment of an officer for scientific integrity (RIS) in each establishment heads the list. Almost all research institutions have an RIS today, but there were only four of them in 2018. Their role is to participate in the drawing up of scientific integrity policies, coordinate awareness and training initiatives, and investigate breaches and report them to the head of the organisation, among other duties. Another new measure is designed to improve our still incomplete knowledge of fraud: every two years, each establishment must pass on the scientific integrity data it has collected (reporting, instructions, penalties, etc.) to the ministry responsible for research and the French Office of Scientific Integrity (OFIS). Set up in 2017, OFIS also acts a national monitoring body and updates the RIS directory. In addition, it provides resources and assistance for those who play a role in scientific integrity, and works on long-term planning. So scientific integrity problems are no longer solely a matter for individual establishments, but are now a national issue.

Researchers taking action

The apparent upsurge of fraud is above all a sign of the scientific community’s improved self-regulation.

A hundred million hours! That is how long in total researchers spent on peer reviews in 2020 – the equivalent of 15,000 years of voluntary work! Before each publication, an editor asks three researchers to review the work of their colleagues: a vital verification process. Some researchers feel the sector does not value peer-reviewing enough, but others see it as key to scientific production, since the time spent checking the work of others is ‘rewarded’ by review of their own work. Even better: some researchers are spending part of their free time combatting fraud. Some of these ‘detectives’ even develop ad-hoc tools, like France’s Problematic Paper Screener. The PubPeer site plays a special role in the work of committed researchers. Set up in 2012 by American neuroscientist Brandon Stell – a researcher from the French National Centre for Scientific Research (CNRS) – and jointly run by Boris Barbour – a CNRS neurobiologist – PubPeer describes itself as a scientific forum for commentary on published papers. It enables moral issues and problems of professional ethics and scientific integrity to be raised anonymously and that sometimes leads to the correction or retraction of papers. So sentinels from every field are recruited! Then there is the Retraction Watch database set up in 2010 by two journalists specialising in health care: Ivan Oransky and Adam Marcus. It plays a leading role in the listing of retracted papers. With more and more information supplied by players in the field of research, the total list of retractions almost doubled from 2018 on to reach 32,000 by the end of 2021.

Looking at publishers

Publishers are in the front line of the war on scientific fraud.

They must be doubly vigilant and plan new ways of dealing with the issue. Publishers play a central role in the fight against fraud through peer-reviewing done by scientists working in the same field. Leading publications such as Nature or Science even employ staff to organise peer review. Others have active researchers working as editors. But in practice, there are sometimes omissions in the assessment of manuscripts: on the one hand, peer reviewers may delegate the task to their juniors (often postgraduate students); on the other, editorial committees can be less than meticulous. As for the ‘haute-couture’ journals – to borrow the term coined by computer specialist Guillaume Cabanac – they are not infallible. At the height of the pandemic, there were a number of controversies involving The New England Journal of Medicine or The Lancet. Mindful of their scientifically rigorous image, those journals publish the details of retracted papers in their columns. Publishers are making more and more use of tools to counter or identify fraud (PubPeer, Retraction Watch, Problematic Paper Screener, etc.) and can even include them in their validation procedures. Finally, ‘predator’ publications that are very low-priced because they charge their authors for publication often print papers as received, fraud and/or errors included. The list drawn up by the American librarian Jeffrey Beall was discontinued in 2016 under pressure from such unscrupulous publishers. It had identified more than 900 and there are certainly over a thousand today, a figure to be compared to the 25,000 scientific publications listed on the Scopus site (the latter are all peer-reviewed).

New tools to combat fraud

Algorithms can be used to generate papers automatically … while others can identify those papers in the scientific press.

Have you heard of Ike Antkare (pronounced I can’t care)? It is the name of an imaginary researcher created by Grenoble computer expert Cyril Labbé in 2011 to show exactly how commonplace fake papers were. In just a few months, the avatar had become one of the most widely-quoted scientists in the world – more than Albert Einstein or Alan Turing. The secret? Many spurious scientific papers produced using an automatic paper generator (SCIgen) developed by MIT* students in 2005, which were then systematically listed by the publication search engine Google Scholar. The automatically-generated texts were readable and grammatically perfect, but meaningless, as shown by this title: ‘Research on coastal atmospheric change and Latin dance performance based on target detection network.’ Yet they were published – and often sold – by unconscientious scientific publishers. To identify that kind of paper, Cyril Labbé then designed SCIgen detection, a programme that can dissect scientific literature. Working with French computer specialist Guillaume Cabanac and Russian mathematician Alexander Magazinov, he also developed software named the Problematic Paper Screener to detect dubious texts. It enabled the identification and reporting of fraudulent papers generated by algorithms. The screener listed 2,482 papers as ‘problematic’ in 2021. Given the progress made over the last few years in automatic language processing and, more generally, artificial intelligence, the phenomenon may well continue to grow.

* Massachusetts Institute of Technology

Open science, an antidote for fraud?

Free access to scientific publication could make it easier to expose fraud and mistakes.

Many scientists are very much in favour of ‘open science’, which involves providing researchers and citizens with free access to publications. Generally, open science is backed by the ministry responsible for research, which promises free access to the results of studies funded by the taxpayer, on publishers’ sites or via open public archives such as HAL. Since 2018, the proportion of open-access scientific publications has risen from 41% to 56%. The plan is to increase that to 100% by 2030. In March 2022, with the same aim of transparency and reproducibility in research – a guarantee of reliability – the ministry published a guide to the sharing of scientific paper source data for researchers. Editors too are increasingly practising open peer review, where the reviewers’ comments and the scientists’ answers are published along with the paper. Sometimes the initiative is very successful: the authors of Nature Communications, which has been offering this option since 2016, have 60% open peer review. The health crisis can be seen as a full-scale test on this subject: open access to scientific research for the general public has gathered pace, both on pre-publication sites and those of paying publishers keen to pass on information rapidly. So the assessment of researchers and establishments, reduced quantity and greater quality, and the economy of open access journals all help to lay solid foundations for open science, an entire model of scientific culture that must continue to evolve.