A famous word-of-mouth study measured rank, not sales

The writer's problem

The finding that online reviews move book sales is cited constantly in publishing writing, usually as a settled fact traced to one study, Chevalier and Mayzlin's The Effect of Word of Mouth on Sales: Online Book Reviews. The paper is cited far more often than its methodology is actually read, and the methodology changes what the finding can support. The NBER working paper page, later published in the Journal of Marketing Research in August 2006, states what the authors actually measured.

What the documents show

The NBER abstract states the authors examine the effect of consumer reviews on relative sales of books on Amazon.com and BarnesandNoble.com, finding reviews were overwhelmingly positive at both sites, that an improvement in a book's reviews increases relative sales at that site, and that 1-star reviews carry more impact than 5-star reviews. The full working paper states the data collection more specifically: the authors searched both sites for a sample of 6,429 titles, narrowed for some analyses to smaller subsamples, over a two-day window, with Amazon data collected 6 May 2003 and BN.com data the following day. The paper states plainly the authors lack access to either site's proprietary sales data; instead they use each site's public sales rank as a proxy, comparing a book's rank across sites to control for shared factors. This is a meaningful qualification: the widely repeated claim that reviews increase sales is, in the paper's own terms, about relative rank movement across two retailers in 2003, not units sold, revenue, or any period outside the sample.

The editorial choice

An editor citing this study to support a broader claim about reviews and sales today should name the proxy, rank rather than units, the sample period, 2003, and the retailers studied, rather than repeating that a study found reviews increase sales as an undated, unconditional fact. The authors themselves flag the rank-to-sales conversion as an approximation, citing a separate source's estimated formula for translating Amazon rank into sales volume, adjusted for the time elapsed since that source's own sample.

What stays with the author

The paper does not measure today's retail environment, recommendation algorithms, or any platform other than the two studied in 2003, so extending its finding to modern review ecosystems, including AI-written or AI-solicited reviews, goes beyond what the document supports. An author weighing reviews driving sales as a justification for review-seeking strategy should treat this paper as evidence for a mechanism observed once, in one dataset, not a current, generalizable rate of return.

• Does the citation name the actual measurement used, sales rank, rather than implying verified unit sales?
• What period and platforms does the underlying data actually cover, and does that match the claim being made today?
• Is a working paper's plain-language abstract being treated as equivalent to reading its data and methodology sections?

Chevalier and Mayzlin's own paper supports a specific, dated finding about relative sales-rank movement tied to reviews on two early-2000s bookselling sites; the methodology section, more often skipped than the abstract, is what actually tells a reader how far that finding can be stretched.