Below, you'll find two different articles about how to read a scientific paper. The second one is written by a science journalist and was added to this guide in 2024. We hope you find both articles useful. They overlap and bring useful techniques to light.
Reprinted by permission of the author, Jennifer Raff, Assistant Professor, Department of Anthropology, University of Kansas, https://about.me/jenniferraff :: original URL: https://violentmetaphors.com/2013/08/25/how-to-read-and-understand-a-scientific-paper-2/
Last week’s post (The truth about vaccinations: Your physician knows more than the University of Google) sparked a very lively discussion, with comments from several people trying to persuade me (and the other readers) that their paper disproved everything that I’d been saying. While I encourage you to go read the comments and contribute your own, here I want to focus on the much larger issue that this debate raised: what constitutes scientific authority?
It’s not just a fun academic problem. Getting the science wrong has very real consequences. For example, when a community doesn’t vaccinate children because they’re afraid of “toxins” and think that prayer (or diet, exercise, and “clean living”) is enough to prevent infection, outbreaks happen.
“Be skeptical. But when you get proof, accept proof.” –Michael Specter
What constitutes enough proof? Obviously everyone has a different answer to that question. But to form a truly educated opinion on a scientific subject, you need to become familiar with current research in that field. And to do that, you have to read the “primary research literature” (often just called “the literature”). You might have tried to read scientific papers before and been frustrated by the dense, stilted writing and the unfamiliar jargon. I remember feeling this way! Reading and understanding research papers is a skill which every single doctor and scientist has had to learn during graduate school. You can learn it too, but like any skill it takes patience and practice.
I want to help people become more scientifically literate, so I wrote this guide for how a layperson can approach reading and understanding a scientific research paper. It’s appropriate for someone who has no background whatsoever in science or medicine, and based on the assumption that he or she is doing this for the purpose of getting abasic understanding of a paper and deciding whether or not it’s a reputable study.
The type of scientific paper I’m discussing here is referred to as a primary research article. It’s a peer-reviewed report of new research on a specific question (or questions). Another useful type of publication is a review article. Review articles are also peer-reviewed, and don’t present new information, but summarize multiple primary research articles, to give a sense of the consensus, debates, and unanswered questions within a field. (I’m not going to say much more about them here, but be cautious about which review articles you read. Remember that they are only a snapshot of the research at the time they are published. A review article on, say, genome-wide association studies from 2001 is not going to be very informative in 2013. So much research has been done in the intervening years that the field has changed considerably).
Before you begin: some general advice
Reading a scientific paper is a completely different process than reading an article about science in a blog or newspaper. Not only do you read the sections in a different order than they’re presented, but you also have to take notes, read it multiple times, and probably go look up other papers for some of the details. Reading a single paper may take you a very long time at first. Be patient with yourself. The process will go much faster as you gain experience.
Most primary research papers will be divided into the following sections: Abstract, Introduction, Methods, Results, and Conclusions/Interpretations/Discussion. The order will depend on which journal it’s published in. Some journals have additional files (called Supplementary Online Information) which contain important details of the research, but are published online instead of in the article itself (make sure you don’t skip these files).
Before you begin reading, take note of the authors and their institutional affiliations. Some institutions (e.g. University of Texas) are well-respected; others (e.g. the Discovery Institute) may appear to be legitimate research institutions but are actually agenda-driven. Tip: google “Discovery Institute” to see why you don’t want to use it as a scientific authority on evolutionary theory.
Also take note of the journal in which it’s published. Reputable (biomedical) journals will be indexed by Pubmed. [EDIT: Several people have reminded me that non-biomedical journals won’t be on Pubmed, and they’re absolutely correct! (thanks for catching that, I apologize for being sloppy here). Check out Web of Science for a more complete index of science journals. And please feel free to share other resources in the comments!] Beware of questionable journals.
As you read, write down every single word that you don’t understand. You’re going to have to look them all up (yes, every one. I know it’s a total pain. But you won’t understand the paper if you don’t understand the vocabulary. Scientific words have extremely precise meanings).
Step-by-step instructions for reading a primary research article
1. Begin by reading the introduction, not the abstract.
The abstract is that dense first paragraph at the very beginning of a paper. In fact, that’s often the only part of a paper that many non-scientists read when they’re trying to build a scientific argument. (This is a terrible practice—don’t do it.). When I’m choosing papers to read, I decide what’s relevant to my interests based on a combination of the title and abstract. But when I’ve got a collection of papers assembled for deep reading, I always read the abstract last. I do this because abstracts contain a succinct summary of the entire paper, and I’m concerned about inadvertently becoming biased by the authors’ interpretation of the results.
2. Identify the BIG QUESTION.
Not “What is this paper about”, but “What problem is this entire field trying to solve?”
This helps you focus on why this research is being done. Look closely for evidence of agenda-motivated research.
3. Summarize the background in five sentences or less.
Here are some questions to guide you:
What work has been done before in this field to answer the BIG QUESTION? What are the limitations of that work? What, according to the authors, needs to be done next?
The five sentences part is a little arbitrary, but it forces you to be concise and really think about the context of this research. You need to be able to explain why this research has been done in order to understand it.
4. Identify the SPECIFIC QUESTION(S)
What exactly are the authors trying to answer with their research? There may be multiple questions, or just one. Write them down. If it’s the kind of research that tests one or more null hypotheses, identify it/them.
Not sure what a null hypothesis is? Go read this, then go back to my last post and read one of the papers that I linked to (like this one) and try to identify the null hypotheses in it. Keep in mind that not every paper will test a null hypothesis.
5. Identify the approach
What are the authors going to do to answer the SPECIFIC QUESTION(S)?
6. Now read the methods section. Draw a diagram for each experiment, showing exactly what the authors did.
I mean literally draw it. Include as much detail as you need to fully understand the work. As an example, here is what I drew to sort out the methods for a paper I read today (Battaglia et al. 2013: “The first peopling of South America: New evidence from Y-chromosome haplogroup Q”). This is much less detail than you’d probably need, because it’s a paper in my specialty and I use these methods all the time. But if you were reading this, and didn’t happen to know what “process data with reduced-median method using Network” means, you’d need to look that up.
You don’t need to understand the methods in enough detail to replicate the experiment—that’s something reviewers have to do—but you’re not ready to move on to the results until you can explain the basics of the methods to someone else.
7. Read the results section. Write one or more paragraphs to summarize the results for each experiment, each figure, and each table. Don’t yet try to decide what the results mean, just write down what they are.
You’ll find that, particularly in good papers, the majority of the results are summarized in the figures and tables. Pay careful attention to them! You may also need to go to the Supplementary Online Information file to find some of the results.
It is at this point where difficulties can arise if statistical tests are employed in the paper and you don’t have enough of a background to understand them. I can’t teach you stats in this post, but here, here, and here are some basic resources to help you. I STRONGLY advise you to become familiar with them.
THINGS TO PAY ATTENTION TO IN THE RESULTS SECTION:
-Any time the words “significant” or “non-significant” are used. These have precise statistical meanings. Read more about this here.
-If there are graphs, do they have error bars on them? For certain types of studies, a lack of confidence intervals is a major red flag.
-The sample size. Has the study been conducted on 10, or 10,000 people? (For some research purposes, a sample size of 10 is sufficient, but for most studies larger is better).
8. Do the results answer the SPECIFIC QUESTION(S)? What do you think they mean?
Don’t move on until you have thought about this. It’s okay to change your mind in light of the authors’ interpretation—in fact you probably will if you’re still a beginner at this kind of analysis—but it’s a really good habit to start forming your own interpretations before you read those of others.
9. Read the conclusion/discussion/Interpretation section.
What do the authors think the results mean? Do you agree with them? Can you come up with any alternative way of interpreting them? Do the authors identify any weaknesses in their own study? Do you see any that the authors missed? (Don’t assume they’re infallible!) What do they propose to do as a next step? Do you agree with that?
10. Now, go back to the beginning and read the abstract.
Does it match what the authors said in the paper? Does it fit with your interpretation of the paper?
11. FINAL STEP: (Don’t neglect doing this) What do other researchers say about this paper?
Who are the (acknowledged or self-proclaimed) experts in this particular field? Do they have criticisms of the study that you haven’t thought of, or do they generally support it?
Here’s a place where I do recommend you use google! But do it last, so you are better prepared to think critically about what other people say.
(12. This step may be optional for you, depending on why you’re reading a particular paper. But for me, it’s critical! I go through the “Literature cited” section to see what other papers the authors cited. This allows me to better identify the important papers in a particular field, see if the authors cited my own papers (KIDDING!….mostly), and find sources of useful ideas or techniques.)
Now brace for more conflict– next week we’re going to use this method to go through a paper on a controversial subject! Which one would you like to do? Shall we critique one of the papers I posted last week?
UPDATE: If you would like to see an example, you can find one here
———————————————————————————————————
I gratefully acknowledge Professors José Bonner and Bill Saxton for teaching me how to critically read and analyze scientific papers using this method. I’m honored to have the chance to pass along what they taught me.
It’s one of the first, and likely most intimidating, assignments for a fledgling science reporter. “Here,” your editor says. “Write up this paper that’s coming out in Science this week.” And suddenly you’re staring at an impenetrable PDF—pages of scientific jargon that you’re supposed to understand, interview the author and outside commenters about, and describe in ordinary English to ordinary readers.
Fear not! The Open Notebook is here with a primer on how to read a scientific paper. These tips and tricks will work whether you’re covering developmental biology or deep-space exploration. The key is to familiarize yourself with the framework in which scientists describe their discoveries, and to not let yourself get bogged down in detail as you’re trying to understand the overarching point of it all. As a specific example, we’ve marked up a Science paper in the accompanying image.
But first, let’s break down what a typical scientific paper contains. Most include these basic sections, usually in this order:
The author list is as it sounds, a roster of the scientists involved in the discovery. But hidden within the names are clues that will help you navigate the politics of reporting the story. The first name in the list is often (but not always) the person who did the most work, perhaps the graduate student or postdoc who is the lead on the project. This person is usually (but not always) designated as the “corresponding author” by an asterisk by their name, or by their email address being given on the first or last page of the paper. If the corresponding author is not the first name in the author list, then take extra care to Google the various authors and figure out how they relate to one another. (In many fields, such as biology and psychology, the last author in the list is typically the senior author or lab head. In others, such as experimental physics where the author list can number in the dozens or hundreds, authors are usually listed alphabetically.) The senior author might be able to provide some broad perspective as to why and how the study was undertaken. But the first or corresponding author is much more likely to be the person who actually did the work, and therefore your better request for an interview.
The abstract is a summary of the paper’s conclusions. Always read this first, several times over. Usually the significance of the paper will be laid out here, albeit in technical terms. A good abstract will summarize what research was undertaken, what the scientists found, and why it’s important. (Compare the abstract of this recent Nature paper, on the discovery of a prehistoric human hybrid, to the first three paragraphs of Sarah Kaplan’s Washington Post story reporting the discovery. Kaplan clearly captures the essence of the new findings as described in the abstract.) Relevant numbers such as the statistical significance of the finding are often highlighted here as well. Abstracts are prone to typographical errors, so be sure to double-check numbers against the body of the paper as well as your interview with the author.
The body of the paper lays out the bulk of the scientific findings. Pay special attention to the first couple of paragraphs, which often serve as an introduction, describing previous research in the field and why the new work is important. This is an excellent place to hunt for references to other papers that can serve as your guidepost for outside commenters (more on that later). Next will come the details of how the research was done; sometimes much of this is broken out into a later methods section (see below). Then come the results, which may be lengthy. Look for phrases such as “we concluded” to clue you in to their most important points. If statistics are involved, see Rachel Zamzow’s primer on how to spot shady statistics.
The final section (sometimes labeled as discussion) often summarizes the new findings, puts them in context, and describes the likely next steps to be taken. If your reading has been dragging through the results section, now is the time to refocus. “That sort of information will help a writer answer the nearly inevitable “so what?” question for their readers as well as their editors,” says Sid Perkins, a freelance science writer in Crossville, Tennessee, who writes for outlets including Science and Science News for Students.
The figures are the data, graphics, or other visual representations of the discovery. Read these and their captions carefully, as they often contain the bulk of the new findings. If you don’t understand the figures, ask the scientist to walk you through them during your interview. Don’t be afraid to say things like, “I don’t understand what the x-axis means.”
The references are your portal into a world of additional inscrutable PDFs. You need to plow through at least a couple of the citations, because they are your initial guide in figuring out who you need to call for outside comment. The references are referenced (usually by number) within the body of the text, so you can pinpoint the ones that will be most helpful. For instance, if the text talks about how previous studies have found the opposite of this new one, go look up the cited references, because those authors would be excellent outside commenters. If you do not have access to the journals described in the references, you can at least look at the paper abstract, which is always outside the paywall, to get a sense of what those earlier studies concluded. (For further caveats on references, see below.)
The acknowledgments are meant for transparency, to show the contributions of the various authors and where they got their funding from. Things to look for in here are whether they thank other scientists for “discussions” or “review” of the work; sometimes peer reviewers are explicitly acknowledged as such, in which case you can call those people right away for outside comment. Occasionally there are humorous tidbits that you can pick up on for a story, such as when authors thank the field-camp guards who kept them safe from predatory polar bears. The funding section is usually pro forma, but it is worth scanning for mention of unusual sources of income, such as from a science-loving philanthropist. If the authors declare competing financial interests (such as a patent filing) you will need to report those out and make sure you understand what financial conflicts of interest may be clouding their objectivity.
The methods often appear in a ridiculously small typeface after the body of the paper. These lay out how the actual experiments were done. Scour these for any details that will bring your story to life. For instance, they might describe how the climate models were so complicated that they took more than a year to run on one of the world’s most powerful supercomputers.
Supplementary information comes with some but not all papers. In most cases it is extra material that the journal did not want to devote space to describing in the paper itself. Always check it out, because there may be hidden gems. In a 2015 study of global lake warming, the only way to find out which specific lakes were warming—and talk about the nearest ones for readers—was to wade through the supplementary information. In another recent example, Harvard researchers left it to the supplementary information to explain that they cranked up a leaf-blower to see how lizards fared during hurricanes, a fact that the Associated Press’s Seth Borenstein turned into his lede.
So now you’re armed with the basics of what makes up a science paper. How should you tackle reading for your next assignment? The task will be more manageable if you break it into a series of jobs.
Your first dive into a paper should be aimed at gathering the most important information for your story—that is, what the research found and why anyone should care. For that, consider following the approach of Mark Peplow, a freelance science journalist in Cambridge, England, who writes for publications including Nature and Chemical & Engineering News.
If it’s a field he’s relatively familiar with, such as chemistry or materials science, Peplow takes a first pass through the paper, underlining with a red pen all the facts that are likely to make it into his initial draft. “That means I can produce a skeleton first draft of the story by simply writing a series of sentences containing what I’ve underlined, and then go into editing mode to jigsaw them into the right order,” he says. (In my annotated example, I’ve done this for the abstract using a purple pen.)
As Peplow reads, he looks for numbers to help make the story sing (“… so porous that a chunk of material the size of a sugar cube contains the surface area of 17 tennis courts”—see orange highlighter in the annotated paper) and methodological details that might prompt a fun interview question (“How scary was it to be pouring that very hazardous liquid into another one?”). He also keeps an eye out for anything indicating an emerging trend or other examples of the same phenomenon, which can be useful for context within the story or as a forward-looking kicker (see how he pulls this off in this Chemical & Engineering News story).
But what if the paper is in a field you’re not experienced with, and you don’t understand the terminology? Peplow has a plan for that too. “I read the abstract, bathe in my lack of understanding, and mentally throw the abstract away,” he says.
Then he goes through the paper, underlining fragments he understands and putting wiggly lines next to paragraphs that he thinks sound important, but doesn’t actually know what they mean. Jargon words get circled, and equations ignored. He forges onward, paying attention to phrases such as “our findings,” “revealed,” “established,” or “our measurements show”—signs that these are the new and important bits. “Once I’ve reached the end of the paper, and I’m sure I don’t understand it, I remind myself it’s not my fault,” Peplow says.
At that point, Peplow starts looking up definitions for the jargon words, either with Google or Wikipedia or in a stack of science reference books he picked up for free when a local library closed. He jots definitions of the words on the paper. To understand concepts, he sometimes searches EurekAlert! for past press releases that explain core concepts, or Googles a string of keywords and adds “review” to hunt for a more comprehensible description.
By this point, Peplow can circle back to the paragraphs marked with wiggly lines and start to understand them better. What he doesn’t yet comprehend, he marks down as an interview question for the researcher.
Before picking up the phone for that interview, it’s worth making a second pass through the paper to see what else you need to help you in your reporting. Check, usually near the end of the paper, to see whether the scientists discuss what the next steps should be—either for their own team or for other groups following up to confirm or expand on the new results, says Perkins. That can provide a ready-made kicker for your story.
Susan Milius, a reporter who covers the life sciences for Science News, often makes a beeline straight for the references to try to start identifying outside commenters for a piece. She will find those PDFs and then look within the references’ references to build a broad understanding of the field. One caveat, though: Be sure to research how these possible commenters are connected to the author of the current study. Once, Milius phoned an outside commenter who had published on the topic in question some years earlier—but that scientist turned out to be the spouse of the new paper’s author. She had a different last name than her husband.
It’s also worth remembering that the authors may well be biased in which references they include in the paper. Self-citations, in which authors try to boost their citation count by adding their previous publications to the reference list, are common. And sometimes authors deliberately omit papers by competing groups, a fact that is not always caught during the peer-review process. So don’t rely on the references within the PDF to be comprehensive; try a Google Scholar search using keywords from the paper to unearth whether there are competing groups out there.
Other clues may lie in how long the manuscript took to make it through the peer-review process. For many journals these dates come at the very end of the paper, marked something like “submitted” and “accepted.” Different journals have different timescales for publishing, but it is always worth looking to see whether the manuscript languished an extraordinary amount of time (like many months) in the review process. If so, ask the author why things took so long. (A fairly innocuous way to do this is to say something like, “I noticed it took a while for this paper to be accepted. Can you tell me how that process went?” Then be prepared for the authors to go on a rant about peer review.)
Finally, see if there are additional sources of information you can sweep into your reporting. Check to see if the author’s institution is issuing a press release about the work; if this isn’t already posted on EurekAlert!, ask the author during the interview if they are preparing additional press materials and, if so, how you can get hold of those. This is also a good time to ask for any art, such as photos or videos to illustrate your story. You will of course have already looked at all their figures in detail, so you’ll be well placed to request the art that is most relevant to what you and your editor are looking for.
With these tools at your side, you should be well suited to tackle your next scientific paper.
Alexandra Witze is a science journalist in Boulder, Colorado, and a member of The Open Notebook’s board of directors. Her news story on the Martian subglacial lake (marked up above) appeared in Nature. Follow her on