Seung-Whan Choi. New Explorations into International Relations: Democracy, Foreign Investment, Terrorism, and Conflict. Studies in Security and International Affairs Series. Athens: University of Georgia Press, 2016. 344 pp. $84.95 (cloth), ISBN 978-0-8203-4907-7; $32.95 (paper), ISBN 978-0-8203-4908-4.
Reviewed by Erin K. Jenne (Central European University)
Published on H-Diplo (November, 2016)
Commissioned by Seth Offenbach
New Explorations into International Relations: Democracy, Foreign Investment, Terrorism, and Conflict by Seung-Whan Choi takes the reader on a step-by-step journey through the process of statistical replication. In a series of stand-alone empirical chapters, Choi evaluates influential statistical studies in the field of international relations by locating flaws in the original research design, rectifying the flaws, and rerunning the analyses. In doing so, he takes aim at a range of well-known and widely cited statistical studies that have shaped scholarly understandings of international relations—from the relationship between ethnic diversity and civil conflict to the effect of democracy versus capitalism on international peace to the importance of the rule of law in attracting foreign direct investment (FDI). For each study, Choi identifies problems specific to that study that are big enough to threaten its validity, including missing data, model misspecification, selection bias, measurement validity, and endogeneity. He then reruns each analysis after correcting the problems in the original design.
The overarching aim of the book is to make a case for replication as an invaluable tool in the social scientist’s research arsenal. Choi states in the introduction, “It seems, though, that contemporary academic culture views being critical as stepping on the toes of scholars rather than as an essential part of scientific inquiry. Many political scientists are apprehensive about getting their hands dirty; this attitude impedes the scientific advancement of the discipline” (p. xxv). Choi demonstrates that he has no such qualms in proceeding chapters, which resemble “demos” for how replication can and should be be done across a range of well-known empirical studies on civil conflict, military interventions, state leadership, terrorism, international development, and democracy. The result is an array of findings that either overturn, or suggest important modifications to, important theories in the field. The added value of Choi’s book over the already-published articles on which the book is based is that the author narrates the process by which replication was conducted on each study. Proof of the value of his approach is that Choi arrives at different, sometimes opposite, conclusions than were in the original study, suggesting correspondingly different policy advice.
The result is a wonderful exposition of the process by which researchers can use statistical methods to evaluate, both theoretically and technically, the validity of major policy-relevant studies in the field. The book is roughly divided into two parts—chapters 1 through 8 is devoted to replications. These stand-alone chapters are dedicated to replicating famous studies based on statistical work.
Choi sets the stage for replication in each chapter by first contextualizing the original study in the corresponding academic debate in the field. He then walks the reader through the problems with the original study, the chosen method of addressing these problems, and the methods used to rerun the analysis along with results. He concludes shortly in each chapter by noting the ways in which the replicated results contradict those of the original study and speculates about what this might mean from the point of view of domestic or foreign policy. Any one of these chapters would serve as an excellent example of how to conduct replication using statistical techniques. The chapters also offer helpful tips for what to avoid when conducting one’s own data analysis, including model misspecification or failing to correct for endogeneity, missing data, or selection bias. The results speak to the potential payoff of replicating influential international relations theories for the purpose of advancing the state of the field. At the same time, the book makes clear the level of commitment and specialized knowledge required to adopt replication as one’s core research strategy.
Three chapters stand out in the first half of the book. The first is Choi’s replication of James D. Fearon and David D. Laitin’s famous statistical finding that a county’s conflict potential is driven not by ethnic diversity but by opportunities to wage war (measured in part by rough terrain and the presence of lootable resources, such as oil and gems). In this chapter, Choi argues convincingly that the authors have not corrected for the possibility of mutual causation. Using simultaneous equation modeling to address this problem, he reruns the analysis to show that ethnically diverse societies are in fact more likely to experience conflict, contrary to the original findings.
In another notable chapter, Choi takes on the well-known “selectorate” model developed by James D. Morrow, Bruce Bueno de Mesquita, Randolph M. Siverson, and Alastair Smith, who analyze data to show that “W”—the “winning coalition,” the size of the government’s effective constituency—determines the degree of public goods provision, with large selectorates associated with more expansive provision of public goods. Choi argues that the model in the original study is misspecified, and that when the proper controls are included, W is no longer associated with the provision of public goods. In rerunning the analysis, Choi finds instead that the level of civil liberties in the country is a more significant and reliable predictor of public goods provision.
In yet another chapter, Choi challenges Eric Gartzke, who used statistical evidence to demonstrate that it is shared capitalist economic models that explain peace between democratic states. Choi re-analyzes his data—after correcting it for sample selection bias and model misspecification—and concludes instead that it is shared democratic values and not shared capitalist models that reduce the likelihood of inter-state conflict.
In the second half of the book, Choi turns to the “next steps” following replication. He demonstrates how researchers might use replicated studies to generate and test new theories or propositions relevant to international relations debates. Drawing on replicated studies, chapters 9 through 13 outline novel research designs aimed to test new ideas in international relations. Here, Choi mixes studies that draw directly on replications from the first part of the book (such as the selectorate theory of public goods provision) with studies merely inspired by these replications (including a fascinating analysis of the impact of US military interventions on the country’s level of foreign and domestic terrorism). The results of his analyses are provocative, showing, for example, that when leaders with prior military leadership use indiscriminate violence against civilians, it is likely to radicalize the opposition, increasing the likelihood of civil war; that the higher the quality of democratic governance, the less likely it is that the country will be able to counter terrorism; that more democratic state-dyads are less likely to use military conscription; and that developing democratic countries undergoing economic crises are likely to receive reduced FDI due to investor fears of politically motivated fiscal irresponsibility. The link between the first and second half of the book might have been made more explicit, but the studies alone are well worth the read.
Choi might have devoted more space—beyond the introduction and conclusion—to the logic and craft of replication itself, namely, why and how to do it. Given that this is a text about data replication, what is missing is a more transparent discussion of how replication works in practice, covering both the payoffs and the pitfalls. Broader questions could have been posed, such as how one ensures that replication is conducted effectively and ethically, how one knows when a replicated study has the potential to advance the field rather than merely muddy the waters, and how to reconcile sometimes disconsonant empirical findings. There could also have been a more directed discussion about how one chooses the studies to replicate and how to spot problems in research designs. Finally, the author could have reflected on what may be the most common mistakes in statistical analysis, including the most effective procedures for addressing these problems. Could a replication protocol of sorts be developed?
While he is a strong advocate of the use of data replication, Choi acknowledges that there are clear professional costs to this practice. He quotes John Ishiyama, “Given the challenges associated with publishing replication attempts, researchers now have little incentive to conduct such studies.” Demonstrating the fragility of the results on which many important international relations theories are based, Choi’s book gives the reader a renewed appreciation for the vital role of replication in “normal science” as an achievable goal rather than a rarefied, albeit unrealistic, Popperian ideal. We can only hope that more researchers devote their time to replicating foundational studies in international relations, for the sake of advancing the field and generating well-informed policy advice.
. James D. Fearon and David D. Laitin, “Ethnicity, Insurgency, and Civil War,” American Political Science Review 97, no. 1 (2003): 75-90.
. James D. Morrow, Bruce Bueno de Mesquita, Randolph M. Siverson, and Alastair Smith, “Retesting Selectorate Theory,” American Political Science Review 102, no. 3 (2008): 393-400.
. Erik Gartzke, “The Capitalist Peace,” American Journal of Political Science 51, no. 1 (2007): 166-191.
. John Ishimaya, “Replication, Research Transparency, and Journal Publication,” PS: Political Science & Politics 47, no. 1 (2014): 78-83, quotation on 82-83.
If there is additional discussion of this review, you may access it through the network, at: https://networks.h-net.org/h-diplo.
Erin K. Jenne. Review of Choi, Seung-Whan, New Explorations into International Relations: Democracy, Foreign Investment, Terrorism, and Conflict.
H-Diplo, H-Net Reviews.
|This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License.|