基于上下文学习的大规模语言模型民意数据插补方法 / In-Context Learning for the Imputation of Public Opinion Data with Large Language Models
1️⃣ 一句话总结
本文提出利用大语言模型的上下文学习能力来填补民意调查中的缺失数据,该方法在多种缺失场景下均优于传统统计插补方法(如MICE PMM),尤其在非随机缺失情况下表现更佳,且能生成更窄的置信区间并接近理想的95%覆盖率。
Large language models have been widely evaluated as simulators of individual survey responses. In practice, however, fully unobserved responses are rare; the dominant problem is partial non-response. Imputation aims to restore the overall structure of a survey dataset by filling in these missing values. It has its own well-defined evaluation criteria and differs fundamentally from prediction. We propose to impute missing survey data through in-context learning (ICL). We systematically evaluate ICL design choices across different missingness mechanisms (MCAR, MAR, MNAR) on 150 opinion variables spanning 15 waves of the American Trends Panel. Compared to well-established statistical methods for data imputation like MICE PMM, our ICL approach consistently reduces absolute error across all missingness mechanisms, with the largest gains under non-random missingness (MNAR). Notably, the best-performing specification (gpt-oss-120b with 100 in-context examples) achieves near-nominal aggregate coverage (approaching the 95% level) with confidence intervals two to five times narrower than MICE PMM. We publish a Python package with an sklearn-like API to enable easy deployment of our method using local and proprietary LLMs.
基于上下文学习的大规模语言模型民意数据插补方法 / In-Context Learning for the Imputation of Public Opinion Data with Large Language Models
本文提出利用大语言模型的上下文学习能力来填补民意调查中的缺失数据,该方法在多种缺失场景下均优于传统统计插补方法(如MICE PMM),尤其在非随机缺失情况下表现更佳,且能生成更窄的置信区间并接近理想的95%覆盖率。
源自 arXiv: 2606.09351