This volume presents a series of chapters written by leading behavioral and experimental economists discussing their methods, objectives, and substantive conclusions gleaned from the data collection methods they employ. The volume, as well as the series of which it is a part, is described by the editors as “a deliberate effort at raising methodological awareness”.

I was very happy to have the opportunity to review this volume, for a number of reasons. First, as a scientist, I believe it is always useful to consider not only what we are doing but how (and why) we are doing it. We're all so busy doing science, we often don't have time to reflect on it. Just as introspection is helpful in our daily lives (and is often accomplished via exogenous means, like New Years' Resolutions), introspection about our goals, our methods and our scientific objectives is important and valuable, and may need an exogeneous shock to trigger it. This volume provides such a shock for its readers and, in doing so, offers a great service to the field and our scientific endeavors.

Second, as an economist, I have seen both the power and the limitations of the economic method in explaining and predicting behavior. This volume provided insights into other methods that we can use (and are using), and a careful discussion of their costs and benefits. The “multi-methods” approach; studying the same question using multiple methodologies, is especially useful to triangulate on a domain, unexplained regularity, or important effect.

Finally, as an experimental and behavioral economist, I have experienced the challenges of introducing new methods to examine existing questions. Reading about how others have resolved this struggle is of interest to me personally as well. In sum, methodological discussions in economics are rare and fruitful, and I value the contribution this volume makes to that endeavor.

However, I had some concerns about the volume as well. It is structured as a series of chapters responding to the first chapter written by Gul and Pesendorfer. This first chapter outlines their philosophy about economic methodology. Subsequent chapters challenge one or another of their claims. Thus the book reads like a pile-up on Gul and Pesendorfer. Although I personally agree with the responses more than the original chapter, I would have preferred to see a more balanced treatment of the methodological issues raised. Allowing Gul and Pesendorfer to respond at the end to the challenges raised by the subsequent chapters, or including chapters by authors who agree with Gul and Pesendorfer rather than exclusively critics would have made the book more like a dialogue.

My second concern is that the readers are not introduced to this debate at its inception, but instead in the middle. The Gul and Pesendorfer chapter is itself a response to Camerer, Loewenstein and Prelec (2005) to which they refer repeatedly but which is not included in the volume. For researchers with an interest in methodology and who have been following this debate, the volume provides a summary and progression. But for readers who are new to this area, the book assumes too much. I would have preferred for this article to be reprinted in the volume (perhaps in an appendix), to make the volume self-contained.

Finally, I think that the volume (and the discussion more generally) would have benefitted from the inclusion of researchers from other disciplines. Physicists, astronomers and (even) psychologists have all struggled with the same questions this book explores: the validity of different sources of data to answer their research questions. Understanding how these other fields have resolved this debate would, I believe, help in our own evaluation and resolution. I believe that the exclusive focus on economists represents a missed opportunity.

At the end of the volume I was left with a few overall impressions. The topic of new methodologies is both critical and under-analyzed. The individual chapters are well-written and well-reasoned, and certainly add value to that discussion. However, the volume would have been improved by providing a more balanced discussion, providing at least a summary of the backstory to the debate, and including insights from other fields as to how they have handled the methodological challenges that we as a field are now facing.

That said, I greatly enjoyed the time I spent with the book. I finished reading it with not only this review, but with multiple pages of good ideas, quotes to insert into research papers, and a deeper understanding of why we do what we do. My concerns about the book are far outweighed by these productive outputs.