A new study led by Olivier Toubia of Columbia Business School finds that AI-powered digital twins, designed to mimic individual behavior, are not yet reliable replacements for human subjects in social science research. While the twins performed better than random chance, they exhibited significant distortions in their responses, averaging a 25% error rate across 19 experiments.
The research team, which recruited over 2,000 individuals from across the United States, gathered extensive data on each participant – including age, ethnicity, income, education, religious practices, political preferences, personality traits, spending habits, mathematical abilities, and vocabulary skills. This data was then used to create digital twins by feeding the information into a large language model and prompting it to respond as if it were the individual. The resulting dataset, created with the intention of being open-source, has already been downloaded approximately 25,000 times.
Evaluations included assessing how the twins and real individuals would react to donations made to both Republican and Democratic party candidates, and their views on algorithmic hiring. While the digital twins demonstrated some ability to predict responses, they frequently skewed towards demographic stereotypes and produced more homogenous answers than actual people. For example, when asked to rate self-control on a scale, the twins showed more variation than a language model using only demographic information, but still produced inaccurate results.
According to Toubia, the disappointing performance is attributable to key distortions in the twins’ responses. “There’s some promise,” Toubia says, “But [the twins’ performance] was overall a bit disappointing.” The study suggests that while AI surrogates hold potential, they currently create a “funhouse mirror” effect, distorting the views of the individuals they are meant to represent.
Originale Berichterstattung lesen
💬 Kommentare
📜 Kommentar-Richtlinie