How to Read a Paper

Reference: Keshav, S. "How to read a paper." ACM SIGCOMM Computer Communication Review 37.3 (2007): 83-84.

0. Abstract

Researchers spend a great deal of time reading research papers. However, this skill is rarely taught, leading to much wasted effort. This article outlines a practical and efficient three-pass method for reading research papers. I also describe how to use this method to do a literature survey.

1. Introduction

2. The Three-Phase Approach

  • The first pass gives you a general idea about the paper
  • The second pass lets you grasp the paper’s content, but not its details
  • The third pass helps you understand the paper in depth

2.1 The First Pass

This pass should take about five to ten minutes and consists of the following steps:

  1. Carefully read the title, abstract and introduction
  2. Read the section and sub-section headings, but ignore everything else
  3. Glance at the mathematical content (if any) to determine the underlying theoretical foundations
  4. Read the conclusions
  5. Glance over the references, mentally ticking off the ones you’ve already read

At the end of the first pass, you should be able to answer the five Cs:

  • Category: What type of paper is this? A measurement paper? An analysis of an existing system? A description of a research prototype?
  • Context: Which other papers is it related to? Which theoretical bases were used to analyze the problem?
  • Correctness: Do the assumptions appear to be valid?
  • Contributions: What are the paper’s main contributions?
  • Clarity: Is the paper well written?

  • Using this information, you may choose not to read further.
  • The first pass is adequate for papers that aren’t in your research area, but may someday prove relevant.

2.2 The Second Pass

In the second pass, read the paper with greater care, but ignore details such as proofs. Note down terms you didn’t understand, or questions you may want to ask the author.

  1. Look carefully at the figures, diagrams and other illustrations in the paper. Pay special attention to graphs. Are the axes properly labeled? Are results shown with error bars, so that conclusions are statistically significant? Common mistakes like these will separate rushed, shoddy work from the truly excellent.
  2. Remember to mark relevant unread references for further reading (this is a good way to learn more about the background of the paper).
  • The second pass should take up to an hour for an experienced reader.
  • After this pass, you should be able to grasp the content of the paper. You should be able to summarize the main thrust of the paper, with supporting evidence, to someone else.
  • This level of detail is appropriate for a paper in which you are interested, but does not lie in your research speciality.

2.3 The Third Pass

  • The key to the third pass is to attempt to virtually re-implement the paper: making the same assumptions as the authors, re-create the work.
  • You should identify and challenge every assumption in every statement. You should think about how you yourself would present a particular idea.
  • This comparison of the actual with the virtual lends a sharp insight into the proof and presentation techniques in the paper and you can very likely add this to your repertoire of tools.
  • You should also jot down ideas for future work.

  • This pass can take many hours for beginners and more than an hour or two even for an experienced reader.
  • At the end of this pass, you should be able to reconstruct the entire structure of the paper from memory, as well as be able to identify its strong and weak points.
  • You should be able to pinpoint implicit assumptions, missing citations to relevant work, and potential issues with experimental or analytical techniques.

3. Doing a Literature Review

  1. Use an academic search engine such as Google Scholar or CiteSeer and some well-chosen keywords to find three to five recent highly-cited papers in the area. Do one pass on each paper to get a sense of the work, then read their related work sections. You will find a thumbnail summary of the recent work, and perhaps, if you are lucky, a pointer to a recent survey paper. If you can find such a survey, you are done.
  2. Find shared citations and repeated author names in the bibliography. Download the key papers and set them aside. Then go to the websites of the key researchers and see where they’ve published recently. That will help you identify the top conferences in that field.
  3. Go to the website for these top conferences and look through their recent proceedings. A quick scan will usually identify recent high-quality related work.
Written on February 11, 2018