A Harvard graduate student is playing dice with your future

In his latest letter, Angus Deaton stresses the advantages of assessment by publication-related metrics, but warns of the dangers to diversity when the highest ratings come from a highly concentrated group of journals.

When I talk to British or European economists, one of the complaints that I hear most often is about the role that the top journals are playing in our profession. Whether in Cambridge, Edinburgh, or Paris, promotion is now conditioned on publishing in top journals, many of which are seen as essentially inaccessible to authors outside the United States. Strangely, these complaints come, not only from anxious and frustrated young scholars, but also from their department chairs and directors, whose freedom of action is limited by the demands of external evaluation.

The advantage of public metrics
While I do think we have a problem — on which more below — I am also old enough to remember the days before metrics and assessments, when professors of no great academic distinction had great power over their departments, and over the profession more generally. I recall an elderly Italian professor in the early 70s, well oiled with brunello, spluttering with rage over a rejection from a new-style journal (I think the European Economic Review) whose editor had dared to cite the comments of an unknown referee. I am not sure the world of those days is entirely gone, and it may even now survive in (not so remote) parts of continental Europe, but the metrics and the top journals have helped diminish its power.

I should also say that the journals have always been kind to me. When I was a very young research assistant in Cambridge, I wrote up some results, and a friend suggested I send the paper to a journal. I had no idea that that was even possible, and didn’t know enough to be surprised when Econometrica accepted it. Though I was surprised when the paper took four years to appear; the society was then tottering on the verge of bankruptcy, and in the end, could only print on paper that had apparently failed East Germany’s quality controls even for sanitary purposes. More seriously, Econometrica, and the Econometric Society’s summer meetings provided a route to professional recognition for me and for others who were just starting out. The professional societies provided an open ladder of advancement. It would be ironic indeed if the top journals today were to become a barrier to young unknown Europeans who do not have the advantages of graduating from the handful of top schools.

So what is there to worry about? 
The top five journals today are usually taken to be the American Economic Review, the Quarterly Journal of Economics, the Journal of Political Economy, Econometrica, and the Review of Economic Studies, though if you are a finance economist or a theorist, there might be some substitutions. Notably, these are not necessarily the journals with the highest impact factors, which are also sometimes used in evaluations. The QJE does well on impact factors too, but the AER falls a long way (essentially because it ‘wastes’ pages on papers from its annual conference and on shorter papers and comments — an essential professional function that is not counted in the metrics), and is replaced by its sister journals, the Journal of Economic Literature, and the Journal of Economic Perspectives, where citation counts are not a reliable mark of original research. As noted by David Card and Stefano DellaVigna in a recent paper in the JEL, the top five journals publish only around 300 papers a year, fewer than twenty years ago, and even if the number of co-authors is rising, the fraction of active authors who can publish in these journals has fallen over time. So the bar is very high and rising all the time. For those outside of North America, it seems impossibly so. Only one of the journals is under European control, and two out of five are not under the control of a professional society. All of the AEA journals have term-limits on editors and, while editors are encouraged to pursue their likes and dislikes, they would not do so for long if they routinely excluded other approaches. This can be bitterly contested territory, but professional associations are capable of dealing with it. The editors of the two ‘other’ journals have published many important and game-changing papers over the years, and papers in the QJE attract the highest citation counts of the five — but they face few constraints on pursuing their personal agendas. That is a very good thing for the diversity of all economics journals, but it is unclear why they — or their graduate students, who referee many of the papers (you are expecting Elvis, but you get the Elvis impersonator) — should play quite such a large part in shaping the profession, not only in North America, but in the rest of the world.

The threat to diversity
One of the most striking changes in American economics has been the increasing prevalence of foreign-born economists. Two-dozen countries are so represented in my department (including non-traditional economics powerhouses such as Korea, Algeria, Mexico, and Belarus), and the third of the faculty who are American-born are on average a good deal older. Many of the public intellectuals of American economics are also foreign-born: from Bengal to Jamaica, think of Amartya Sen, Danny Kahneman, Daron Acemoglu, Luigi Zingales, Raghuram Rajan, Abhijit Banerjee, Esther Duflo, Simon Johnson, Enrico Moretti, and Peter Blair Henry, who are not only fine academics, but are explaining their ideas to a broad audience interested in popular social science. These people came to the US with a very different backgrounds and presumptions than did the ex-farm boys from the western United States who used to be so prominent in economics. I remember being astonished in the Cambridge of the 1960s by George Stigler’s (1959) claim that ‘the professional study of economics makes one politically conservative,’ and wondering if the last word could possibly be a typo? American economics is immensely richer for the inflow of these immigrants, but it is hard not to wonder what might have happened had they stayed put. Economics, like a species, needs diversity as the material for change in times of crisis, and diversity is perhaps limited when so many go through almost identical training programs in a small number of universities. Exporting the standards of those schools and the top five journals to British and European programs, while protecting against the excesses of the old world, risks a uniformity that would diminish the local approaches that might contain the future of economics. Heterodox economics is endangered as it is — George Stigler, in the same article, wrote that a believer in the labor theory of value could not get a desirable job, not because of his radicalism, but because the hiring committee could not bring themselves to believe that such a person could be both intelligent and honest. A modern American hiring committee might think that there was something to be learned from studying the labor theory of value, and would be unlikely to endorse such a monolithic view of economics, but perhaps a committee of external evaluators in France, Germany, or Britain, armed with its metrics, impact factors, and citation counts, might unwittingly do so.