ECPR

Install the app

Install this application on your home screen for quick and easy access when you’re on the go.

Just tap Share then “Add to Home Screen”

Improving V-Dem Coding Transparency with Large Language Models (LLMs) - Co-authored with Nils B. Weidmann

Quantitative Research Methods
Comparative Politics
Authoritarian Regimes and Democratic Transitions
Seraphine Maerz
University of Melbourne
Seraphine Maerz
University of Melbourne

Abstract

Panel: Using LLMs for text analysis V-Dem's democracy measures rest on expert coders who assign ordinal scores without recording the factual basis for their judgements. We propose to use large language models (LLMs) to generate verifiable, referenced background information for each country-year and indicator, which coders can then verify and rank-order — improving transparency without inflating coder workload. We outline a pilot study on V-Dem v16 that selects influential, event-driven expert-coded (Type-C) indicators, generates background information and provisional codes with a web-enabled OpenAI model via a reproducible `quallmer` pipeline, and validates the results against V-Dem's own human coding. The pilot prepares the ground for an experiment embedded in the v17 coding process.