Three diets, three papers, one retraction, and lots of concerns

Note: this post is not an allegation of misconduct. I do not have any strong feelings about low- or high-carb diets.

This post tells the tale of three papers. Paper #1 was retracted, republished as paper #2, and republished a second time as paper #3. Let’s take a look at what happened. Based on an original Twitter thread on Twitter and on ThreadReaderApp.

Paper #1

In September 2018, the journal Clinical Cardiology published a paper by US scientists called “Long-term health effects of the three major diets under self-management with advice, yields high adherence and equal weight loss, but very different long-term cardiovascular health effects as measured by myocardial perfusion imaging and specific markers of inflammatory coronary artery disease.” [Paper #1].

The paper with the very long title was quickly retracted about a month later, leaving nothing but a retraction notice. A manuscript-version of the paper can still be found on ResearchGate.

The paper describes a cohort of 120 obese people with an average BMI of over 40, which is classified as “Extreme Obesity” according to this NIH table. The 120 people were randomly assigned to 6 diets (3 main groups: vegetarian, low to moderate fat, or low-carbohydrate, and each of these groups with or without vitamin supplementation). Each diet was about 1500-1600 kcal/day, and each participant had to do a workout three times a week. Throughout a period of a full year, a range of clinical parameters such as weight, inflammatory markers, blood cholesterol values, and cardiovascular imaging were collected.

The results of this study showed that although people in all 6 groups lost weight, the low-carb diet resulted in a significant increase in cardiovascular risks, and the highest amount of people gaining weight after they stopped the diet.

Retraction of paper #1

Within days of publication, George Henderson wrote a critical post about the Three Diet paper on his blog HopefulGeranium, which reads “Three of the authors work at Fleming’s medical imaging company in California, one is a deceased psychologist from Iowa, another is a pediatric nutritionist from New York and one is a Kellogg’s employee from Illinois. How this group was able to run a 12-month diet trial in 120 subjects is something of a mystery.”

The paper was retracted within a month.

So what led to the retraction? The notice is quite vague, stating “The article has been withdrawn due to concerns with data integrity and an undisclosed conflict of interest by the lead author“.

The swift retraction of this paper was featured at RetractionWatch in an article called “A convicted felon writes a paper on hotly debated diets. What could go wrong?” (an alternative link can be found here). The post is remarkable because the first author left several long comments in which he talks about many different topics, but fails to address any concerns about the paper in question.

Both posts at HopefulGeranium and RetractionWatch reported that the first author has admitted to data fraud before, that he has been convicted of medical billing fraud (a felony in the US), and that he has been indefinitely stripped of his medical license in the state of Iowa and debarred by the FDA “from providing services in any capacity to a person that has an approved or pending drug product application” (updated Dec 2020). More public records can be found here, here, and here. In addition, both articles point out that the first author has tried to discredit Robert Atkins (who promoted the low-carb “Atkins diet”) by deceivably obtaining and disclosing Atkins’ death reports to a vegan activist group, suggesting that the first author of the retracted paper is not a big fan of the low-carb diet approach.

Re-publication as Paper #2

After the paper was retracted, it was quickly republished in a non-PubMed indexed journal labeled by others as “predatory”, the Biomedical Journal of Scientific & Technical Research under the exact same title [Paper #2].

Re-publishing another time as Paper #3

After republishing the paper, the authors were not satisfied yet. As pointed out by user Cheshire on Twitter yesterday, the study was published a third time in another journal. This time it was called “The Weight, Inflammation, Diet and Heart (WIDtH) Study Looking at Potential Primary Prevention of Coronary Artery Disease“, published in Acta Scientific Pharmaceutical Sciences, another non-PubMed indexed journal [Paper #3].

Comparing the three papers

The three papers all appear to contain the same measurements, suggesting they all describe the exact same study.

However, comparing the three papers, there are some interesting changes in the author and affiliations lists.

Paper #1, doi: 10.1002/clc.23047, lists the following authors: Richard M. Fleming, Matthew R. Fleming, Gordon M. Harrington, Keith-Thomas Ayoob, David W. Grotto, Andrew McKusick. Affiliations (all within the USA) are: FHHI-Omnific Imaging-Camelot, University of Northern Iowa, Albert Einstein College of Medicine, and Kellogg Company (formerly Block Medical).

Paper #2, doi: 10.26717/BJSTR.2018.10.002006, lists the following authors: Richard M Fleming, Matthew R Fleming, Gordon M Harrington, Keith Thomas Ayoob, and David W Grotto. Note that Andrew McKusick is not listed anymore, although the data and affiliations in this paper appear otherwise to be identical to Paper 1.

Paper #3, no DOI, lists the following authors: Richard M Fleming, Matthew R Fleming, Gordon M Harrington and Tapan K Chaudhuri. Affiliations have changed a bit and now include: FHHI-Omnific Imaging-Camelot, University of Northern Iowa, and Eastern Virginia Medical School. Note that author David Grotto and Kellogg Company are no longer listed, but that Tapan K Chaudhuri and Eastern Virginia Medical School have been added.

Several concerns arise when carefully going over affiliations, patient cohorts, and data. Here are the observations and concerns as raised by George Henderson, RetractionWatch, several PubPeer posts on Paper #1 and Paper #2, and me.

Missing conflict of interest statement

Several authors have a non-academic affiliation, and the papers do not appear to contain a Conflict of Interest (CoI) statement. This is an important omission, since the paper’s finding suggest that a low-carb diet is not successful (perceived CoI for Kellogg’s) and that cardiovascular health should be monitored (perceived CoI for Fleming RM because of a patent involving medical imaging).

• Richard M Fleming RM, Matthew R Fleming, and Andrew McKusick are affiliated with the private clinic “FHHI-Omnific Imaging-Camelot El Segundo”.
• David W Grotto is affiliated with Kellogg Company, and in addition is the founder of a nutrition firm giving dietary advice.

This might seem fairly irrelevant at first. However, one could imagine that Kellogg’s might use the data from this “peer-reviewed” paper to claim that low-carb diets are bad and that you should buy more high-carb food items, like cereal breakfasts (which is what Kellogg’s is selling). It is really hard for most non-scientists to distinguish between scientific papers that have been published in journals that take peer review seriously or those that have been published after less scrutiny.

Location of trial

With several authors from geographically different locations, it is unclear at which institution these 120 patients were recruited or where the study was conducted. This makes it harder to verify if the participants were recruited under proper IRB approval and under which institution consent was obtained. Given their specialities, it is unclear if the three academic authors could have had HIPAA-compliant access recruit or have HIPAA-reliant access to the medical files of a group of 120 adults with severe obesity.

• Gordon M. Harrington, deceased, is listed at the University of Northern Iowa, where he was a professor of Psychology
• Keith-Thomas Ayoob is listed at the Albert Einstein College of Medicine, where he is a pediatric nutritionist
• Tapan K Chaudhuri appears to be a retired nuclear medicine specialist who previously worked at Eastern Virginia Medical School

Randomization

The 120 participants were randomly assigned into the 6 diet groups by casting a die. Yet, each group contained a perfect number of 20 participants, and each group was reported to almost identical age, weight, BMI, and other clinical parameters at the start of the study. This seems hard to achieve by randomly assigning patients casting a die. What are the chances of casting a die 120 times and getting exactly 20 times each of the values “1”, “2”, “3”, “4”, “5”, and “6”? The authors obviously could have used a better randomizer (let them draw numbers out of a hat), but their description does not appear to be complete.

Adherence

All 120 participants stayed in the study for a full year and more. The participants all had a very high BMI (average >40), had to follow a caloric restrictive diet (1500 calories), and exercise three times a week. The perfect adherence – with no loss of participants even at the 70 week follow up data point – raises questions. Often, many participants drop out of these types of study because they move, become pregnant, become ill, etc.

Discrepancies in averages

There are discrepancies in the averages reported in the papers. Here are some baseline values that do not appear to average correctly:

• Average weight is reported as 272 lbs, but the calculated average is 275. 
• Average BMI is reported as 42.4, but the calculated average is 42.7. 
• Average LDL is reported as 185, but the calculated average is 189.

Demographics similarities and discrepancies

There are some unexpected similarities in the demographics listed in Paper #3. In the average ages + standard deviations listed for each of the 6 groups, there are at least 4 very similar looking pairs of numbers (42.55 and 43.55, 6.82 and 6.84, 9.27 and 9.27, and 45.10 and 43.10; shown in bright green). Also, the number of men vs women changed between Paper #1 /2, and Paper #3 (shown in red).

Questions for the authors

It would be great if the authors could answer some of the issues raised here, on PubPeer, or in the letter of concern I sent today to the Editors of the journals and Research Integrity Officers at the affiliated institutes. All concerns about the validity of the papers could be resolved if the authors could provide us a detailed list of the raw, original data, HIPAA approval for use of clinical data by a not-treating physician, and copies of the IRB approval and (anonymized) patient consent forms.