If you are interested in scientific integrity, you will probably know PubPeer, a website where you can leave anonymous or signed comments on scientific papers.
PubPeer was launched in late 2012 by neuroscientist Brendon Stell and brothers Richard and George Smith, with Boris Barbour and Gabor Brasnjo acting as advisers. At the start, the founders and advisers were anonymous, but they revealed their identity in 2015.
Post-Publication Peer Review
The PubPeer website features post-publication peer review (PPPR), a place to leave scientific praise or criticism after a paper has been published.
Scientific papers traditionally are reviewed before publication in a process called peer-review. During peer review, manuscripts are sent to a couple of other scientists, who were not involved with the study, but who have the expertise to read and improve the paper. Peer reviewers can comment on weaknesses of the manuscript, such as flawed methods, far-fetched conclusions, missing references, or missed hypotheses, and suggest improvements. After their evaluation, the peer reviewers will determine if the manuscript could be accepted for publication if the suggested edits are included. The authors then have a chance to respond and improve the paper, and hopefully get it accepted for publication.
Pre-publication peer review has long been viewed as the golden standard in evaluating and improving the quality of scientific papers. However, many scientists (including myself) think that a paper can also be commented on after publication. Scientific knowledge keeps on growing, and scientific papers are science’s stepping stones, but scientists should be allowed to comment on papers, even after they have been published.
Before PubPeer was launched, there were only limited options to comment on published papers. One could write a letter, comment, or reply to the journal in which the original paper had been published. However, this process was cumbersome and would take months, so most scientists would rather spend their time on something else. It is also hard to check if a paper had been commented on.
Alternative PPPR websites
A year after PubPeer was launched, in the fall of 2013, a similar service was introduced by the National Institute of Health (NIH) / National Center for Biotechnology Information (NCBI), PubMed Commons. This platform was integrated with NCBI’s PubMed, the main access site for biomedical papers, and quickly gathered a number of devoted commenters. However, it never became very popular, and it was discontinued in March 2018. Only 6,000 of the 28 million articles indexed in PubMed had been commented on by that time. Comments on PubMed Commons could not be left anonymously, which might have been the main reason that the service was not used a lot.
In addition, some journals such as those published by the Public Library of Science (PLOS) or BioMed Central (BMC), have their own way to comment on papers. As with the now discontinued PubMed Commons tool, users can only comment using their real name. These features are not used a lot.
Publons, the site where scientists can keep track of their peer reviews and editorial contributions, also has the possibility to comment on papers after their publication, but users need to register first.
Commenting on a scientific paper, in particular when the comments are not positive, is risky business for academics, which depend on letters of recommendation, faculty track evaluations, and collaborations for their career. Therefore, most academics are reluctant to leave comments under their full name.
PubPeer offers anonymity
PubPeer uses a different model than PubMed Commons or other sites. It allows users to choose between leaving signed comments (under their real name) or anonymous – which quickly became the most popular choice.
There are many opinions about the possibility to comment on scientific papers anonymously. Anonymity allows people to reveal scientific misconduct without repercussions for their career. Traditional, pre-publication peer review has been anonymous for that exact same reason. On the other hand, people have argued that it would lead to hateful or false comments by competitors. But, as with pre-publication peer review, most scientists are very capable of formulating their critiques in an objective manner. Of course, we all have experienced the sour comments by reviewer #3, but anonymous scientific peer-review usually remains civil and polite.
When PubPeer was first launched, there were two options to leave comments. All users had to register, but could chose to post under their full name, or as anonymous “peers”. The first comment on a post would always be “Peer 1”, and additional comments were listed as “Peer 2”, “Peer 3”, etc.
In 2013, PubPeer introduced a third option to leave comments, anonymously and without registration. Such comments would be shown as “unregistered”, and quickly became the most popular option. Most “unreg” comments revealed serious concerns about papers, such as image duplication and manipulation.
In 2016, PubPeer introduced an additional layer of security for people who left unregistered comments. It stopped keeping track of IP addresses. This was the result of a lawsuit filed by an author of many papers flagged by PubPeer users for possible image problems. This scientist claimed to have lost a job offer because of the negative PubPeer comments, and sued the site in an attempt to get the IP addresses of the unregistered commenters. Although PubPeer won this legal battle, they realized that it was better to not log IP addresses in the first place, so there would be nothing to hand over to legal authorities if they were subpoenaed again.
In this new approach, commenters can register without IP tracking, by assigning them a user name – a plant, animal, or microorganism name from the tree of life. Users also get a “secret key”, a password that they can use to log in. Commenters can thus create multiple accounts, and switch between them, or switch between anonymous and full-name comment mode easily.
In order to keep the discussions constructive and civil, PubPeer has several guidelines for posting comments. Statements need to stick to facts and be publicly verifiable. For example, one could say that two images look very similar to each other, but not say that the authors committed fraud. Comments from new accounts are moderated and occasionally removed if they violate those rules.
Since its start in 2012, PubPeer has become a very useful tool for the public posting of shortcomings or concerns in scientific papers. It has led to many corrections and retractions of problematic papers that otherwise would not have been noticed. One could compare this to the effect of social media postings on misconduct by airlines or police officers – where video footage taken by bystanders has led to public outcry about people being treated unfairly, and a change of protocols or behavior (although there is still much that needs to change).
Similarly, for scientific literature, Paul S. Brookes has shown that internet coverage of problematic papers leads to an increased chance of the paper being corrected or retracted.
Indeed, PubPeer has played an important role in the critical assessment and subsequent retraction of published papers, such as the Obokata paper on STAP cells, which was commented on immediately after publication. And the scientist mentioned above, who lost the lawsuit against PubPeer to get commenters’ IP addresses, currently has 77 papers flagged on PubPeer, and over 20 retractions.
How to search on PubPeer
Searching on PubPeer is easy. You can type in the name of the DOI, PMID or author to search for papers. You can also search for particular commenters by typing in: Users: <name>.
Update: The search engine for users appears to be a bit picky. These searches work :
- Users: bik
- Users: elisabeth
- Users: “Elisabeth M Bik”
Some power commenters to look out for are:
- users: “Hoya Camphorifolia”
- users: “Actinopolyspora Biskrensis”
- users: “Indigofera Tanganyikensis”
There are also very helpful PubPeer extensions for your favorite browser (Chrome, Firefox), that alert you for comments on papers, for example during PubMed searches. It is not perfect, but it is very helpful. Here is a screenshot of how that might look like in PubMed:
How to comment on PubPeer
It can be scary to comment on a scientific paper using PubPeer, especially the first couple of times. I would recommend to not start posting under your full name, unless you are confident that your comment would not harm your career, or if you have something positive to say. Positive comments are always appreciated, of course!
To create an anonymous (or a signed) account, click on the “create account” button on the top right to register. Anonymous accounts will get a name from the tree of life (e.g., “Homo sapiens”) and “a secret key”- a 16 digit password that you will need to store in a safe place.
PubPeer’s Frequently Asked Questions section offers many helpful guidelines, or you can just browse the most recent comments to get an idea of what people might discuss.
- PubPeer, the online journal club
- How to Publish a Scientific Comment in 1 2 3 Easy Steps [Satire] – Rick Trebino – Semantic Scholar
- PubMed Commons closes its doors to comments – Elie Dolgin – Nature
- Post publication peer-review: Everything changes, and everything stays the same – Bonnie Swoger – Scientific American – 2014
- 5 Steps to Writing a Winning Post-Publication Peer Review – Publons.com
- PubPeer wins appeal of court ruling to unmask commenters – Alison McCook – RetractionWatch – 2016
- Meet PubPeer 2.0: New version of post-publication peer review site launches today – Alison McCook – RetractionWatch – 2017
15 thoughts on “PubPeer – a website to comment on scientific papers”
Dr. Bik: As you know, I am a non-scientist who has anonymously commented on PubPeer and Twitter about research papers. It has been a learning curve in many respects, and I am still not certain that I am a particularly welcome participant since I can in no way be considered a “peer.” I think on several occasions I have helped correct the scientific record and uncovered suspected misconduct, but I have also consumed lots of time of scientific professionals (including Dr. Bik’s) as they respond to and/or correct my comments. Is there a consensus about whether non-scientists should participate on a forum like PubPeer?
LikeLiked by 1 person
Dear Regret, I personally think that people without a scientific background have similar “rights” to comment on problematic images as scientist do, as long as they have been trained on spotting what is right or wrong. You did the right thing by asking scientific professionals for advice in the beginning, and still occasionally when you are not sure, and you will only post about cases you know are right. That is great and that makes you a very valuable contributor on PubPeer. We need more people like you.
LikeLiked by 1 person
This guest post comes from a non-scientist, not even a former scientist or a citizen scientist or someone who always wanted to be a scientist. If you are not a scientist, you are not qualified to judge scientific works. No matter how good you are at catching some irregularities in the pictures or whatever. You are not qualified, cause you don t understand the underlying scientific concepts behind those pictures. Science should be only judged by scientists. You can read the scientific papers and think for yourself that these pictures maybe are manipulated or not but you are not qualified to accuse scientists for fraud and ruin their lives, no matter you are right or wrong. It s a serious business. I emphasize it s a real business and not only scientists lives depend on publishing, also their relatives lives depend on this business. You should understand that if somebody cheats in a paper, there must be a real reason for that. Nobody is insane or sadistic to publish nonsense. In my experience, in 99% of times, experiments don t work at all and as a result it will not lead to anything publishable. It means 99% of scientists should change their careers cause the science is not for them. But please understand that a lot of people can t do anything except this ridiculous business that you call it science. You guys are funny, cause it seems you don t live in the real world. You see the science as some sort of sacred religion that all the bad cheaters should be burnt in the hell. But, this view has nothing to do with this real world. For you guys, it s really easy to say, OK, if you can t do a good science, leave it and find an another job or maybe find a janitor work. But, keep in mind, you are talking about people and they are human like you and they do everything to have a good life, and personally I believe if this shit hole that you call it science is the only way to have a better life, cheating on papers is just being generous to this corrupt society and they are really respectable if they don t steal things or don t murder people. Fortunately, I truly believe that 99% of scientists don t give a shit to your thoughts at all, and that s a good sign I believe, that your geeky thoughts are abandoned by the scientific society as general. I introduced PubPeer to a scientific community, which they classified themselves as a community that have a really high standards in research integrity. They were like what the fuck are you talking about?! 99% of them said they don t give a shit to PubPeer or whatever shit hole you use for post-publication review.
I’ve noticed recently that I must be logged in before I can search for my own username or I have the same results as you. After logging in just now, I was able to successfully search for Elisabeth M. Bik also.
“You can also search for particular commenters by typing in: “Users: name“.”
This way to find a person’s comments doesn’t seem to be working.
I tested with your name, and also an anonymous one.
users:”Elisabeth M. Bik”
users:”Elisabeth M Bik”
Users: “Elisabeth M Bik”
“Users: Elisabeth M Bik”
“Users:Elisabeth M Bik”
Users:Elisabeth M Bik
Users: Elisabeth M Bik
These searches works for me:
* users: Elisabeth
* users: bik
* users: “Elisabeth M Bik”
But it is puzzling that users: Elisabeth M Bik does not work.
Replying to Regret, ah, yes, strange, it’s a login thing. You have to be logged in for searches to work. I wonder if pubpeer is willing to change this. It doesn’t seem like a good function.
I didn´t know about this website! I will look into it for information about ammoniumperchlorat and its possible replacements ([url=https://www.ammoniumperchlorat.com/blog/was-ist-ammoniumperchlorat.html]was ist ammoniumperchlorat)[/url}
Seriously, I do not see anything great the famous scientists invented or discovered. It was just use of common sense; Totally applications. It is just that the inventions were new and never seen before. They just had to reason.
I totally disagree with the theories of centrifugal and centripetal forces. Can anyone prove that they exist without using what they crammed in class?
merci pour les infourmation
yeah me2 too