含大语言模型评分的结构方程模型中的共同方法变异：基于模型的校正与预测驱动校正的比较

英文标题/原题

Shared Method Variance in Structural Equation Models With Large Language Model Scores: Model-Based Versus Prediction-Powered Correction

作者

刘康; 杨晓娟

通讯作者

刘康

研究方向

测量方法；结构方程模型；大语言模型标注

关键词

large language models; LLM annotation; method variance; structural equation modeling; prediction-powered inference

投稿/发表信息

发表期刊/目标

Communication Methods and Measures

投稿/成果状态

投稿中

投稿日期

2026-10-23

发表日期

-

原地址/投稿系统地址

https://rp.tandfonline.com/

期刊与检索情况

检索情况

SSCI、Scopus（未核实）

摘要/说明

Large language models (LLMs) are increasingly used to annotate text in computational content analysis, and their scores are entered as indicators in structural equation models after agreement with human codes has been checked construct by construct. When one model–prompt combination scores both constructs of a structural relation, its systematic tendencies enter both, and this shared method variance biases standardized paths by an amount that does not shrink with sample size. In five Monte Carlo studies, relative bias reached 77%–88%, confidence intervals excluded the true value, and averaging repeated scores, adding indicators, and ordinal estimation did not help. Independent human codes for a random subsample of texts can correct the bias in two ways. An anchored multitrait–multimethod model was the most efficient correction but was biased when the LLM's errors did not follow its measurement model, whereas prediction-powered inference was robust but less precise. A bootstrap comparison of the two corrections held its nominal size and detected strong misspecification. An application to online discussion posts from course forums, scored by open-weight LLMs from two model families, illustrates the diagnostics and shows how strongly anchored estimates can depend on specification. We close with recommendations for design and reporting.

基金信息

无

论文备注

项目目录：E:\小作坊
_论文
_LLM评分误差模拟（P003）。CMM 投稿包：05_投稿上传文件\投稿包_CMM\（manuscript、manuscript_anonymous、online_supplement_anonymous、cover_letter，中英对照/）。数据与代码：OSF wcjyh（已公开）。投稿历程：BRM 稿号 BR-Org-26-943，2026-08-10 投、2026-08-25 退回补材料（原题 When LLM Scores Enter Structural Equation Models…）；SEM 投稿号 261828491（稿号 HSEM-2026-0294），2026-10-03 投、2026-10-08 初审拒稿；2026-10-23 改投 CMM。