Using automated text classification to explore uncertainty in NICE appraisals for drugs for rare diseases Abstract
with Lea Weidmann, John Cairns, and Orlagh Carroll
Accepted, International Journal of Technology Assessment in Health Care, 2023
This study examined the application, feasibility and validity of supervised learning models for text classification in appraisals for rare disease treatments (RDTs) in relation to uncertainty, and analyzed differences between appraisals based on the classification results. We analyzed appraisals for RDTs (n = 94) published by the National Institute for Health and Care Excellence (NICE) between 01/2011 and 05/2023. We used Naïve Bayes, Lasso, and Support Vector Machines (SVM) models in a binary text classification task (classifying paragraphs as either referencing uncertainty in the evidence base or not). To illustrate the results of the classification task, we tested hypotheses about differences between RDT appraisals in relation to the appraisal guidance (technology appraisal (TA) or highly specialized technology (HST) appraisal guidance), advanced therapy medicinal product (ATMP) status, disease area, and age group. The best performing (Lasso) model achieved 83.6 percent classification accuracy (sensitivity = 74.4 percent, specificity = 92.6 percent). Paragraphs classified as referencing uncertainty were significantly more likely to arise in HST appraisals compared to TA appraisals (AOR=1.44, 95% CI 1.09, 1.90, p=0.004). There was no significant association between paragraphs classified as referencing uncertainty and appraisals for ATMPs, non-oncology RDTs, and RDTs indicated for children only or adults and children. These results were robust to the threshold value used for classifying paragraphs but were sensitive to the choice of classification model. Using supervised learning models for text classification in NICE appraisals for RDTs is feasible but the results of downstream analyses may be sensitive to the choice of classification model.
with Raluca Pahontu and Timothy Hicks
Accepted, British Journal of Political Science, 2023
The onset of the COVID-19 pandemic constituted a large shock to the risk of acquiring a disease that represents a meaningful threat to health. We investigate whether individuals subject to larger increases in objective health risk – operationalised by occupation-based measures of proximity to other people – became more supportive of increased government healthcare spending during the crisis. Using panel data which tracks UK individuals before and after the outbreak of the pandemic, we implement a fixed-effect design which was pre-registered before the key treatment variable was available to us. While individuals in high-risk occupations were more worried about their personal risk of infection, and had higher COVID death rates, there is no evidence that increased health risks during COVID-19 shifted attitudes on government spending on healthcare, nor broader attitudes relating to redistribution. Our findings are consistent with recent research demonstrating the limited effects of the pandemic on political attitudes.
with Lotte Hargrave
British Journal of Political Science, 2022
Research on political style suggests that where women make arguments that are more emotional, empathetic, and positive, men use language that is more analytical, aggressive, and complex. However, existing work does not consider how gendered patterns of style vary over time. Focusing on the UK, we argue that pressures for female politicians to conform to stereotypically ‘feminine’ styles have diminished in recent years. To test this argument, we describe novel quantitative text analysis approaches for measuring a diverse set of styles at scale in political speech data. Analysing UK parliamentary debates between 1997 and 2019, we show that female MPs’ debating styles have changed substantially over time, as women in parliament have increasingly adopted stylistic traits that are typically associated with ‘masculine’ stereotypes of communication. Our findings imply that prominent gender-based stereotypes of politicians’ behaviour are significantly worse descriptors of empirical reality now than they were in the past.
with Benjamin E Lauderdale
American Journal of Political Science, 2022
with Lucy Barnes and Benjamin E Lauderdale
American Journal of Political Science, 2021
Legislative Studies Quarterly, 2020
Parliamentary Debate in the UK House of Commons Abstract
with Roberta Damiani
Chapter in The Politics of Legislative Debate (eds. Hanna Bäck, Marc Debus, and Jorge M. Fernandes, Forthcoming, 2020)
with Benjamin E Lauderdale, Delia Bailey and Douglas Rivers
International Forecasting Journal, 2020
British Journal of Political Science, 2019
with Benjamin E Lauderdale
Journal of Politics, 2017
with Andrew C. Eggers, Dominik Hangartner & Simon Hix
British Journal of Political Science, 2016
with Benjamin E Lauderdale
Revise and Resubmit, American Political Science Review, 2023
Moral Foundations Theory (MFT) aims to explain the origins of and variation in human moral reasoning. Applications in political science have revealed differences in the degree to which liberals and conservatives explicitly endorse five core moral foundations of care, fairness, authority, loyalty and sanctity. However, differences in self-reported assessments of the moral relevance of each foundation do not imply that citizens with different political orientations respond to concrete scenarios based on different moral intuitions. We introduce a new approach for measuring the implicit importance of the 5 moral foundations by asking survey respondents from the UK and the US to compare pairs of vignettes which describe violations relevant to each foundation. We analyse responses to these comparisons using a hierarchical Bradley-Terry model which allows us to evaluate the relative importance of each foundation to individuals with different political perspectives. Our results suggest that, despite prominent claims to the contrary, voters on the left and the right of politics share broadly similar moral intuitions.
Revise and Resubmit, British Journal of Political Science, 2023
What are the effects of reason-giving on political attitudes? Both political philosophers and political scientists have speculated that defending proposals with reasons may change voters’ preferences. However, while prominent models of attitude formation predict that the explicit justification of one’s political views may result in attitudes that are more ideologically consistent, less polarized, and more stable, empirical work has not assessed the connection between reason-giving and attitudes. Implementing a survey experiment in which some respondents provide reasons before stating their opinions on six issues in UK politics, I find that reason-giving has very limited effects on the ideological constraint, temporal stability, or polarization of the public’s political attitudes. These findings have important implications for our understanding of deliberative conceptions of democracy – in which reason-giving is a central component – as well as for our understanding of the quality of voters’ political opinions.
with Benjamin E Lauderdale
Revise and Resubmit, American Journal of Political Science, 2023
What drives ideological division about political problems? When prioritising which problems are most in need of redress, voters might disagree about the severity of individual outcomes that constitute such problems; the prevalence of those problems; or whether such problems are amenable to solution by government action. We field a large survey experiment in the UK and US and develop a new measurement approach which allows us to evaluate how ideological disagreements change when respondents consider the individual badness, social severity, and priority for government action of a set of 41 political problems. We find that large ideological divergences are observed in beliefs about social severity and priority for government action, not individual problem badness, and only in the US. An important implication of these results is that polarization over problem-prioritization is more likely to be an emergent property of US politics than of the psychology of US citizens.
with Timothy Hicks, Alan M. Jacobs, J. Scott Matthews, and Tom O’Grady
Revise and Resubmit, Journal of Politics, 2023
In response to the COVID-19 pandemic governments implemented large-scale economic and social policies that, outside of war time, are unprecedented in scale and scope. They highlighted the state’s capacity to guarantee economic and health security, and they reached beyond demographic groups that are more typically beneficiaries of state support. Because of this, we hypothesise that exposure to the pandemic and these policy responses caused ideological change, including attitudes to the role of government in the economy, redistribution, and the deservingness of beneficiaries of state support. We test this expectation using data from the long-running (2014–present) British Election Study Internet Panel, together with a unique panel survey fielded to existing BES respondents in April and September, 2020. Our panel makes it possible to track individuals on a rich set of variables both before and during the pandemic. We find virtually no evidence that the pandemic, or exposure to pandemic-induced shocks, affected ideological beliefs about the role of government, or economic and social policy attitudes. In a follow-up survey experiment on British respondents we test one possible reason for this lack of change –- a lack of elite cues -– but find that exposure to elite cues linking the pandemic to a greater government role in providing welfare, national insurance and public spending has no impact on ideological beliefs either. We conclude that the pandemic was not, and could not have been, a cause of mass ideological change.