In a new study, accepted for presentation at the RES Annual Conference on gender and equality in economics, Erin Hengel and Eunyoung Moon1 show that articles written by male economists are cited less than articles published by women in the same journals. The authors also find that men’s citations rise when they co-author with women, and that women’s citations fall while they co-author with men, conditional on acceptance.
Women are under-represented in top economics journals. Although they make up 20-30 percent of academic economists, women are only 11 percent of all authors published in top-five journals since 1990, 12 percent since 2000 and 14 percent since 2010 (see Lundberg and Stearns (2019) and Figure 1). Over that same period, there has been very little growth in the number of exclusively female-authored papers; almost no growth in the number of majority female-authored papers; and no meaningful change in the number of mixed-gendered papers with a senior female author. The only tepid growth that has occurred, is largely — if not entirely — due to an increase in the number of published articles by senior men co-authoring with a minority of junior women.2
Several recent papers have investigated why. Hengel (2019) found female-authored papers in top-four economics journals are held to higher writing standards in academic peer review. As a result, their manuscripts are subject to greater scrutiny, spend longer under review and women, in turn, respond by conforming to those standards. Using submissions data from a set of four semi-overlapping journals, Card et al. (2020) found that female-authored papers are higher quality conditional on referee recommendations.
In a new study, we join this research by investigating gender differences in quality conditional on acceptance by a top-five journal. We find that (i) articles authored by men are cited less than articles by women published in the same journals; (ii) men’s citations rise when they co-author with women; and (iii) women’s citations fall while they co-author with men. Under strong — but we believe reasonable — assumptions, these results imply top economics journals hold female-authored papers to higher standards, and, consequently, do not publish the highest quality research. They also suggest that authors of both sexes will be less willing to collaborate with women, all else equal.
The female advantage in citations
Results from our principle analysis are displayed in Figure 2. The grey markers in the upper panel represent the average female advantage in citations, conditional on controls.4 Column (1) adjusts for year of publication, journal, co-author count, and authors’ experience at the time an article was written-all factors found in previous research to influence citations, conditional on quality.5 Column (2) adds a control for author reputation at the time citations were collected; it accounts for the impact a stronger reputation later on can have on citations of an author’s earlier work, all else equal. Columns (3)-(5) restrict the data to articles published after 1990. Columns (4) and (5) control for sub-field. Columns (6)-(7) are estimated on the sample of articles published after 2000 and control for journal and year, only.
Every model in Figure 2 suggests female-authored papers are cited more than male-authored papers. Figure 3 reinforces this conclusion. It plots the distribution of citations for solo-authored papers; consistent with results in Figure 2, women’s citations are, on average, higher than men’s citations.
Everyone’s citations rise when they collaborate with women
Figure 4 analyses how citations change as the same economist co-authors with members of the opposite sex. Light grey figures (first panel) represent the returns to male economists from co-authoring with women; dark grey figures represent the returns to female economists from co-authoring with men. As in Figure 2, models differ by controls and sample restrictions.
Everyone’s citations rise when they collaborate with women, conditional on acceptance to a top-ranked journal (Figure 4, upper panel). Men’s citations increase 10-12 log points when they co-author with an equal share of women (triangle). Women’s citations fall -12 to -19 log points when they co-author with an equal share of men (square).
Male and female comparisons are relative to papers they either solo author or co-author entirely with other men and women, respectively.
Estimates in Figure 4 are potentially biased by contributions from unobserved co-authors. For example, suppose, hypothetically, that Anna Schwartz, Milton Friedman and Kenneth Arrow collaborated on one paper while Arrow collaborated with Gérard Debreu on another. If Friedman’s contribution means Arrow’s article with Schwartz and Friedman is cited more than his article with Débreu, then the ratio of female authors on his papers will spuriously correlate with citations unless Friedman’s presence is accounted for.
To adjust for this, we additionally restrict our sample to senior male economists with at least two top-five papers co-authored with a single junior author of each sex. This creates a treatment group — i.e., senior male authors co-authoring with exactly one junior woman — that very closely resembles the untreated group — i.e., those very same seniors co-authoring with exactly one junior man.
Consistent with results in Figure 4, we find a senior male author’s work is more highly cited when it is co-authored with a woman: citations increase 60-70 log points when senior male economists co-author with junior women as opposed to junior men.
What does this mean?
Assuming citations are not biased in favor of women conditional on quality (as well as included controls), our results suggest female-authored papers published in top economics journals are higher quality than male-authored papers.6
Higher quality female-authored papers could be consistent with gender-neutral acceptance standards if women’s papers are accepted more often or the variance in their quality is greater (see Hengel and Moon 2020, Theorem 3.1).7 Neither appears to be the case. As shown in Figure 3, variance in quality is lower in female-authored papers, conditional on acceptance. Recent evidence from a set of journals that partially overlaps with our own suggests men's and women’s manuscripts are accepted at roughly equivalent rates (Card et al. 2020).
Figures 4 and 5 also rule out consistent same- or opposite-sex complementarities meaningfully contributing to gender differences in quality, conditional on acceptance. The impact of co-authoring with members of the opposite sex would be positive for men and women if they produce higher quality work by collaborating with one another; conversely, both effects should be negative if everybody works better with members of their own sex. Our evidence is not consistent with either story. Instead, we find that the impact of co-authoring with a man is negative for women while the impact of co-authoring with a woman is positive for men. This suggests that women disproportionately contribute to the quality of co-authored work — again, conditional on acceptance — and is consistent with women facing tougher standards in peer review.
Finally, our results indicate that female economists may not be able to correct for higher standards all on their own. Although the fall in women’s citations when they co-author with men suggests adding male co-authors can alleviate higher standards, men also experience a rise in citations when they co-author with women, suggesting co-authorship alone probably cannot eliminate them. And when combined with evidence of the ‘Matilda effect’ in tenure decisions — as Sarsons et al. (2019) shows, tenure committees discount women’s contributions to mixed-sex co-authored papers — co-authoring with men may have other negative consequences, as well.
Policy conclusions?
As Becker (1957) has shown, discrimination hurts its victims and its perpetrators. In the presence of complete and perfect markets, harm inflicted on the former is fully absorbed into higher costs for the latter. As a result, bias has no effect on equilibrium outcomes — e.g., women can simply publish their higher quality papers in currently lower-tiered (but non-discriminatory) journals, confident that their actions will lead to an immediate adjustment in journal rankings.
When competition isn’t perfect, however, discrimination interacts with one or more market frictions to prevent those who discriminate from fully internalizing its costs. Consequently, its victims will have to partially bear them. For example, imperfect information about journal rankings may mean tenure and promotion committees’ expectations are slow to adjust to the lower quality of journals that reject too many women. As a result, women (and the men they co-author with) are tenured and promoted at lower rates than they otherwise would be if markets were complete. To the extent that grant committees similarly rely on applicants’ past publication histories to choose between projects, women will also have a harder time funding future work.
Moreover, discrimination undoubtedly distorts authors’ decisions in ways that can further misallocate available resources. Indeed, our own evidence implies male and female economists are better off collaborating with men — all else equal — suggesting authors of both sexes may forgo higher quality co-authoring opportunities with women in order to partner with men.
In economics, we generally approach incomplete markets by proposing policies that try to fix individual imperfections. When the space of market failures is large and each is impossible to eradicate completely, however, policies based on outcomes are sensible alternatives (Lundberg 1991). Not only are they objective and non-punitive, they may also create positive externalities that might not have been achievable using markets alone (see, e.g., Niederle, Segal, and Vesterlund 2013; Besley et al. 2017). For example, if journals clearly signal a determination to publish more female authors, they will likely decrease the relative price of co-authoring with women, encourage better co-author matches and increase the aggregate quality of economic research.
Nevertheless, an adequate understanding of the context in which discrimination occurs is absolutely crucial to responsible policy-making. More research is certainly needed. We hope journals are challenged to address the tougher standards they likely impose on women, willing to support the access and research needed to better understand them and open to whatever policy options most effectively check them.
Notes
1. University of Liverpool
2. Data from Hengel and Moon (2020).
3. Printing restrictions limit us to tw-colours. Full colour displays are available at https://erinhengel.github.io/Gender-Quality/quality-summary.html
4. To proxy for quality, we follow previous research and use the inverse hyperbolic sine (asinh) of citations as our dependent variable (Card and DellaVigna 2020; Card et al. 2020). To define article gender, we use its proportion of female authors (for a justification of this indicator, see Hengel 2019). Results and conclusions are similar if we use alternative transformations or raw citation counts and/or define article gender by various other means — e.g., solo-female-authored or exclusively female-authored (see Hengel and Moon 2020).
5. See Hengel and Moon (2020) for select citations and Tahamtan, Afshar, and Ahamdzadeh (2016) for an extensive review of this literature.
6. Prior research suggests this assumption holds: homophily skews citations toward male authors (Ferber 1986; Dion, Sumner, and Mitchell 2018); economists believe female-authored papers are cited less, holding quality constant (Card et al. 2020); men are more likely than women to cite their own work (King et al. 2017); subsequent papers cite earlier female-authored papers 4-15 percent less often than they cite male-authored papers after accounting for a similar set of control variables — and the gender imbalance is largely driven by the referencing patterns of men (Dworkin et al. 2020; for related evidence see also Koffi 2019).
7. To prove this statement, we assume the quality of male- and female-authored submissions are normally (although not necessarily identically) distributed.
References
Becker G, 1957. The Economics of Discrimination Chicago, Chicago U P.
Besley T, Folke O, Persson T, and Rickne J. 2017. ‘Gender Quotas and the Crisis of the Mediocre Man: Theory and Evidence from Sweden’. American Economic Review 107 (8), 2204-42.
Blank R M, 1991. ‘The Effects of Double-Blind Versus Single-Blind Reviewing: Experimental Evidence from the American Economic Review’. American Economic Review 81 (5), 1041-67.
Card D and DellaVigna S, 2020. ‘What Do Editors Maximize? Evidence from Four Economics Journals’. Review of Economics and Statistics 102 (1), 195-217.
Card D, DellaVigna S, Funk P, and Iriberri N. 2020. ‘Are Referees and Editors in Economics Gender Neutral?’ Quarterly Journal of Economics 135 (1), 269-327.
Ceci S J, Ginther D K, Kahn S and Williams W M. 2014. ‘Women in Academic Science: A Changing Landscape’. Psychological Science in the Public Interest 15 (3), 75-141.
Dion M L, Sumner J L and McLaughlin Mitchell S. 2018. ‘Gendered Citation Patterns Across Political Science and Social Science Methodology Fields’. Political Analysis 26 (3), 312-27.
Dworkin J D, Linn K A, Teich E G, Zurn P, Shinohara R T and Bassett D S. 2020. ‘The Extent and Drivers of Gender Imbalance in Neuroscience Reference Lists’. http://arxiv.org/abs/2001.01002v1.
Ferber M A. 1986. ‘Citations: Are They an Objective Measure of Scholarly Merit?’ Signs 11 (2): 381-89.
Gilbert J R, Williams E S, and Lundberg G D . 1994. ‘Is There Gender Bias in Jama’s Peer Review Process’. JAMA 272 (2), 139-42.
Hengel E. 2019. ‘Publishing While Female. Are Women Held to Higher Standards? Evidence from Peer Review.’ Mimeo.
Hengel E and Moon E. 2020. ‘Gender and Quality at Top Economics Journals’. Mimeo.
King M M, Bergstrom C T, Correll S J, Jacquet J and West J D. 2017. ‘Men Set Their Own Cites High: Gender and Self-Citation Across Fields and over Time.’ Socius 3, 1-23.
Koffi M. 2019. ‘Innovative Ideas and Gender Inequality’. Mimeo.
Lundberg S J. 1991. ‘The Enforcement of Equal Opportunity Laws Under Imperfect Information: Affirmative Action and Alternatives’. Quarterly Journal of Economics 106 (1), 309-26.
Lundberg S J and Stearns J. 2019. ‘Women in Economics: Stalled Progress’. Journal of Economic Perspectives 33 (1) 3-22.
Niederle M, Segal C, and Vesterlund L. 2013. ‘How Costly Is Diversity? Affirmative Action in Light of Gender Differences in Competitiveness’. Management Science 59 (1), 1-16.
Sarsons H, Gërxhani K, Reuben E, and Schram A. 2019. ‘Gender Differences in Recognition for Group Work’. Journal of Political Economy, forthcoming.
Tahamtan I, Askar S A, and Ahamdzadeh K. 2016. ‘Factors Affecting Number of Citations: A Comprehensive Review of the Literature’. Scientometrics 107 (3), 1195-1225.
