Duplications in Spectrum Plots

One of my recent investigations led me to expand my set of figure types to look at. For our 2016 mBio study, in which I scanned >20,000 papers for image duplication, I focused on real photos of Western blots, agarose gels, tissue sections, etc.

Some examples of photos from biomedical papers. Top left: Western blot. Top right: agarose gel. Bottom left: petridish with bacteria. Bottom right: immunostaining of tissue sections. None of these photos have image duplications. Source: Wikimedia Commons.

My scans also included flow cytometry images, since these consists of thousands of individual dots that will land at a slightly different and unique place, even if the experiment gets repeated. So if two flow cytometry images, or parts of those images, are identical, that is unexpected, and I might flag these as image duplications.

Flow cytometry image, consisting of 1000s of dots in a unique pattern. Source: Wikimedia Commons.

In my searches I had not really looked at other figures in biomedical papers, such as line graphs. It is hard for me to compare line or bar graphs since they all look very similar. And if they are smoothened, it is very hard to tell if two graphs within the same paper or between two papers are the same. In addition, line graphs are easy to manipulate behind your computer without the reader being able to tell.

Some examples of line graphs. Source: Wikimedia Commons.

But recently I noticed that some line graphs might be easier to tell apart – and easier to tell if they are unexpectedly similar. As I was looking at a set of papers in the field of nanoparticles, I noticed that these papers often contained certain types of line graphs with a lot of detail. These plots look a bit like earthquake measurements, with a lot of tremors, and every plot looked unique. That makes them easier to compare to each other, and to tell them apart or not.

One type of graph that had that amount of detail that you can tell them apart are XRD plots. XRD stands for “X-ray diffraction”, and it is a technique to analyze the structure of a crystal and its arrangement of atoms. The details of this method are obviously way outside of my scope of knowledge, but Barbara L Dutrow from Louisiana State University and Christine M. Clark from Eastern Michigan University wrote an explanation here.

In short, the crystal is bombarded with X-rays in different angles in a rotating machine, and the intensity of the reflected rays gets continuously measured. At certain angles that are specific for certain crystal structures, the rays get reflected. The results of these measurements are displayed as a spectrum that plots the angle of the X-rays (X-axis) vs. the intensity of the reflected signal (Y-axis). The position of the peaks in this spectrum will tell the user the structure and dimensions of a crystal (I hope I got this right hahaha).

Examples of XRD spectra showing plots that contain peaks specific for certain atoms, as well as background “noise”. Source: ResearchGate.

Another type of spectra that research papers on nanoparticles often contain is XPS plots. XPS stands for “X-ray photoelectron spectroscopy”. Here, the chemical composition of the surface of a material can be analyzed. XPS plots show peaks that are characteristic for certain elements, and it will tell the user which elements are present in the surface layer of an unknown material.

XPS plots are somewhat similar to XRD plots in that they show characteristic peaks separated by “noise” measurements.

Obviously, if two materials are very similar in structure and composition, the peaks in their XRD or XPS plots will fall at the same position, and two plots might look very similar at first glance. However, the noise in between the peaks should look different.

Here is a graph taken from a PLOS ONE article with four XRD plots, each from a nanoparticle baked at a different temperature, to illustrate this. The structure and composition of these nanoparticles are expected to be very similar.

XRD plot of nanoparticles baked at different temperatures. Source: https://doi.org/10.1371/journal.pone.0154704.g001 (edited by me).

I have marked the peaks on the top graph in blue. All four spectra look very similar to each other if you just look at the peaks. That is expected because they have the same composition.

But in between the peaks there are regions where no reflection was measured. Here you will see what is called “noise”, just the tremble of the background measurement. I have marked some of those regions in red. The noise parts of the 4 graphs all are different. So, even though the four graphs look similar to each other at first glance, each graph is unique in the noise parts.

However, I am not an expert in crystallography, XRD, or XPS. Maybe I am wrong here. So my question is: would one expect the noise parts of these types of plots to look exactly the same, even if the same crystal was measured twice? If you are an expert, please let me know in the comments!

For now, based on the plot above, and many others that I found online (see here, here, and here) I will assume that the noise parts of these plots should never look the same.

Although most XRD and XPS plots in the biochemical literature appear to be unique, some others are remarkably similar. Here are some PubPeer entries questioning the unexpected identity of such plots.

In some cases there is even apparent repetition within the same plot. Here is an example of some unexpected noise stutter, as marked by a pseudonymous PubPeer user.

Stuttering plot, highlighted by user Hoya Camphorifolia on Pubpeer. Source: https://pubpeer.com/publications/B060C4FC51F7918BBA893B2B3780E7#2

Smut Clyde (pseudonym) has found many of those noise stutter examples, where plots appear to be composed of repeating parts. For example, read Smut’s article called “Nanodandruff and synthetic spectroscopy” on Leonid’s blog For Better Science.

Some more examples of unexpected repeats within spectra plots can be found on PubPeer:

Let me know what you think about these unexpected repeats in the comments below. Do you think these repeats can occur by chance?

Advertisements

Concerns about a paper on HPV vaccination and pregnancy rates

A short post based on a Twitter thread I wrote today about a paper I had seen a while ago. You can also read the ThreadReaderApp post, which many people find easier to read.

A couple of weesk ago, I saw a social media post claiming that the HPV vaccine would cause infertility in women, based on results in a peer-reviewed paper. It sounded hard to believe, because many other studies have found that the HPV vaccine is safe to use. There was no time that day to look up the details and I let it go.

Yesterday, I talked to a friend about HPV vaccination, and I remembered – and found – the HPV vaccine/infertility paper again. Here is the paper, published in a Taylor & Francis journal:

HPV types, screening, and vaccination

Here is some background about the HPV vaccine. “HPV” stands for human papillomavirus, which is a group of viruses that can be transferred from one person to another during sex. HPV infections are very common among sexually active persons, usually do not cause any symptoms, and disappear by themselves. However, in some cases, the body does not clear up the infection, and these long-lasting infections can cause disease. Different HPV strains are associated with different types of symptoms. Certain HPV types can cause genital warts, while other types (types 16 and 18 in particular) are associated with cervical, anal, and several other types of cancer. The HPV vaccine is specifically designed to prevent infections with these cancer-associated virus strains. It is basically a vaccine that can prevent certain types of cancer, and thus can save lives.

Most women over 21 with access to healthcare will be familiar with the “Pap” test, in which some tissue is taken from the cervix and examined for the presence of pre- or cancerous cells. This exam is usually performed every 3 years, and can detect early stages of cervical cancer. For women over 30, new US guidelines recommend a Pap test plus HPV detection every 5 years.

The HPV vaccine is relatively new (released around 2006) and it is recommended for girls and boys age 11-12, before teenagers become sexually active and can acquire – or spread – the infection. You can read more about the current CDC recommendation regarding HPV vaccination here.

The HPV vaccine is safe and does not negatively affect pregnancy rates

The HPV vaccine has proven to be very safe (see for example here and here) so it was strange to see a paper claiming it would reduce pregnancy rates. That would be horrible!

The claim made in the DeLong paper also appeared to contradict results from a large 2017 study on 3500 women trying to become pregnant. This study found that HPV vaccination status had no negative effect on pregnancy probability. In fact, the researchers found that prior HPV vaccination in women with a history of sexually transmitted infections actually increased their chances of becoming pregnant.

Hat tip for finding this study: Alison Gemmill through Twitter.

A first look at the DeLong paper

Here is a screenshot of the abstract. Reading it, you might assume that the sole author analyzed data from 8 US million women, and you might think “wow, that is a big study”. But digging a bit deeper in the abstract reveals that the study looked at survey results from only 700 women, so much smaller than the abstract suggests.

For her study, DeLong used responses to the National Health and Nutrition Examination Survey (NHANES). She first shows this graph of declining pregnancy rates in women between 25-29 years old. This seems plausible. Women have babies at a later average age than a decade ago.

The sample set of the 700 women surveyed between 2007 and 2014 included 118 women who got the HPV shot and 582 who did not. This also makes sense because the vaccine was only introduced around 2006. The vaccine is typically given to young teenage girls so if you interviewed women aged 25-29 between 2007-2014, most of them did not get the vaccine because it was not being offered to them.

Here is a look at the demographics between vaccinated and un-vaccinated women (Table 2). There are some interesting statistical differences between these 2 groups, marked here by me in red.

Higher % college degrees in the HPV-vaccinated group

There is one very important difference between the 2 groups that were compared in this study. The women who had an HPV shot had a significantly higher chance of having a college degree. There was also a trend (albeit not significant) that they had a higher family income. 

There are several plausible explanations for the higher rates of HPV vaccination among women with a college degree. Higher educated women might know more about the benefits of the vaccine, or might have better access to healthcare, than women without a college degree or of lower-income families. We could even jokingly argue that getting the HPV vaccine might increase one’s collage application chances.

As a side note, the surveyed women appeared to be only 2.25 feet (67 cm) tall (highlighted in blue) 😂 But @ThatsRegretTab1 quickly pointed out that this looked like a duplication of the data in the “age” row.

Correlation between education levels and age of giving birth

This difference in % college degrees and family income between the 2 groups could be a HUGE confounding factor in pregnancy rates in young women. Here is why.

Take a look at this graph, published in the New York Times about a year ago. It shows that women with college degrees, on average, have their first baby at a later age than women without a college degree.

Source: https://www.nytimes.com/interactive/2018/08/04/upshot/up-birth-age-gap.html

Women with a college degree or higher are on average 30.3 years old when they have their first baby. So if you interview women who are 25-29 old and ask if they ever had been pregnant, that means you are leaving out women who did not have their first baby YET.

A much better study design would have been to include women up to 35 or 40 years old, so you are also including the women who had their babies at a slightly later age.

The way the author chose the age group to be limited to young women, together with the significant difference in % college degrees between vaccinated vs. unvaccinated women suggests this is a very biased, non-scientific study.

Other critical reviews of the DeLong paper

Here are some other critical blog posts about this study that bring up other flaws in DeLong’s study.

Points raised in these posts are the following:

  • DeLong is a well-known antivaxer, someone who claims that vaccines cause autism.
  • Although she is an associate professor of economics and finance, she has no background in science, epidemiology, or medicine.
  • The study failed to report the rates of contraception in either vaccination group. Contraception is obviously another important confounding factor in the changes of becoming pregnant, but the paper does not report the answers to questions about contraception, even though they were included in the survey data that was used.
  • Although DeLong claims that the vaccine is negatively affecting birth rates, she did not find the dose-response relationship that one would expect if the vaccine was toxic.

Conclusion

  • The study’s claim that HPV vaccination inhibits the chances of becoming pregnant are incorrect, because the author only included young women, and ignored the fact that many women have their first baby only after turning 30. The author ignored the important confounding factor of having a college degree, which was significantly higher in the vaccinated group. By not including women 30 years or older, she biased the results of her survey to ignore the fact that HPV vaccination status, college degree, and age of first pregnancy are intricately connected.
  • You can read my concerns about this paper in this PubPeer post.
  • I would love to see a similar analysis done on a larger group and that included women up to 40 years old, and the answers to contraceptives use.

Weekly digest, July 23

Some interesting articles I came across in the past couple of weeks.

How a data detective exposed suspicious medical trials – David Adam – Nature
Anaesthetist John Carlisle has spotted problems in hundreds of research papers — and spurred a leading medical journal to change its practice.

‘Bad science’: Australian studies found to be unreliable, compromised – Liam Mannix – The Age
Hundreds of scientific research papers published by Australian scientists have been found to be unreliable or compromised, fuelling calls for a national science watchdog. For the first time, a team of science writers behind Retraction Watch has put together a database of compromised scientific research in Australia.

Continue reading “Weekly digest, July 23”

How to report misconduct to a journal

This post is based on a Twitter thread from April 2019, with some additional information. I have added some of my own reports and some UnSplash stock photos for illustrations.

I got several questions from other scientists who were interested learning how to report suspected misconduct or other irregularities in scientific papers. In this post, I will discuss how to do that.

Continue reading “How to report misconduct to a journal”

Fabrication: The Diederik Stapel case

One of the most discussed case of fabricated data comes from the Netherlands. Diederik Stapel was a professor of Social Psychology at Tilburg University and the University of Groningen. Some of his research was featured on news sites, such as his Science paper that found that people discriminate more in messy environments or another (unpublished) study that thinking about meat made people less social.

Continue reading “Fabrication: The Diederik Stapel case”

PubPeer – a website to comment on scientific papers

If you are interested in scientific integrity, you will probably know PubPeer, a website where you can leave anonymous or signed comments on scientific papers.

PubPeer was launched in late 2012 by neuroscientist Brendon Stell and brothers Richard and George Smith, with Boris Barbour and Gabor Brasnjo acting as advisers. At the start, the founders and advisers were anonymous, but they revealed their identity in 2015.

Continue reading “PubPeer – a website to comment on scientific papers”

Human Photosynthesis

Unfortunately, several angry Twitter users pointed out it was insensitive of me and irrelevant to mention the country where the Human Photosynthesis Study Center scientists are located. I am confused about this, but I do not want to be insensitive. It appears it is OK to mention most countries but not certain others. I will just try to continue to be an equal-opportunity science integrity detective.

Yesterday, Twitter user @Arroboso pointed out research on “Human Photosynthesis” through this tweet.

Of course I was curious. Last time I checked, humans are not capable of photosynthesis. Instead, I learned that humans are heterotrophs, organisms that rely on eating other organism to get their energy from.

Continue reading “Human Photosynthesis”