Trends in Qualitative Research
I recently spent two weeks at the Institute for Qualitative and Multi-Method Research (IQMR) at the University of Syracuse. It is a great program for learning about and gaining the skills needed for a wide range of qualitative methods used in Political Science research. Too often qualitative methods are thought to be no more than reading and basic analysis. IQMR has been at the forefront of systemizing and promoting the different types of qualitative research as distinct methodologies. Of particular interest, were the sessions introducing new techniques geared towards enhancing the robustness and reproducibility of qualitative research. These tools emphasize making the data and the analytical framework more transparent. There is an exciting amount of innovation ongoing in the design and practice of qualitative methods, and I wanted to highlight a couple of them that I believe will have a substantive impact on the future of Political Science research.
A Bayesian Analytical Framework
Interestingly, Bayes’s theorem is the topic du jour with both qualitative and quantitative methodologists as a means to improve inferences made with respect to collected evidence. For example, Nate Silver, best known for his election forecasting models, places Bayesian logic as the core theme in his book — The Signal and the Noise. According to Silver, “Bayes’s theorem is nominally a mathematical formula. But it is really much more than that. It implies that we must think differently about our ideas— and how to test them. We must become more comfortable with probability and uncertainty. We must think more carefully about the assumptions and beliefs that we bring to a problem” (Silver 2012, Kindle Location 294–296).
Framed in this manner, Bayes’s theorem is a process for evaluating new information using the following three steps.
- What is the probability that the new piece of evidence would exist if the hypothesis is true?
- What is the probability that the new piece of evidence would exist if the hypothesis is false?
- What is the probability that the hypothesis is true given the prior research on the subject?
These questions are represented by the formula:
In which, “the updated probability of a hypothesis being true in light of new evidence
is equal to the prior probability attached to H (or
times the likelihood of the evidence in light of hypothesis H and our background knowledge
, divided by the prior likelihood of E . Thus, the more unlikely a piece of evidence is in light of alternatives to explanation H, the more that evidence increases our confidence that H is true if the evidence proves consistent with H” (Bennett 2008: 709).
Bennett illustrates how this process can improve inference by looking at three famous case studies of historical explanations. The application of Bayesian logic can also address one of the main critiques about small-N research, that is, how much can be inferred about general phenomena from a single case-study or even the examination of a small number of cases. Applying the steps listed above, scholars can determine the degree to which the new case updates confidence in a given theory.
Another new initiative in qualitative research is Active Citation. The aim is to establish a standard that “any critical and contested substantive empirical point in a scholarly case study should be backed by a precise and annotated citation to one or more presumptively primary sources” (Moravcsik 2010: 31). Furthermore, these citations “must contain a hypertext link to a reproduction or transcript of some part of the source” (ibid). From the discussion at IQMR, I understand that the citation itself would be around 100 to 150 words from the source to comply with copyright requirements.
I am excited about these new opportunities for creating more robust and reproducible qualitative research. It often gets forgotten that it only became common for quantitative researchers to make their data and code available approximately ten years ago. Similar moves in qualitative work have the potential to change how we do research by creating incentives for collaboration and through shared standards for evaluation of evidence and citation practices.
Finally, if you are interested in learning more about these or other qualitative methods, you should check out the IQMR website.
- Silver, Nate. 2012. The Signal and the Noise : Why Most Predictions Fail– but Some Don’t. New York: Penguin Press. ↩
- Bennett, Andrew. 2008. Process tracing: A bayesian perspective. In The Oxford Handbook of Political Methodology. Ed. Janet M. Box-Steffensmeier, Henry E. Brady, and David Collier. The Oxford handbooks of political science. Oxford, England ; New York: Oxford University Press. ↩
- Moravcsik, Andrew. 2010. Active citation: A precondition for replicable qualitative research. PS: Political Science & Politics 43 (01): 29–35. ↩