TY - GEN
T1 - Deus Ex Machina and Personas from Large Language Models
T2 - 2024 CHI Conference on Human Factors in Computing Sytems, CHI 2024
AU - Salminen, Joni
AU - Liu, Chang
AU - Pian, Wenjing
AU - Chi, Jianxing
AU - Häyhänen, Essi
AU - Jansen, Bernard J.
N1 - Publisher Copyright:
© 2024 Copyright held by the owner/author(s)
PY - 2024/5/11
Y1 - 2024/5/11
N2 - Large language models (LLMs) can generate personas based on prompts that describe the target user group. To understand what kind of personas LLMs generate, we investigate the diversity and bias in 450 LLM-generated personas with the help of internal evaluators (n=4) and subject-matter experts (SMEs) (n=5). The research findings reveal biases in LLM-generated personas, particularly in age, occupation, and pain points, as well as a strong bias towards personas from the United States. Human evaluations demonstrate that LLM persona descriptions were informative, believable, positive, relatable, and not stereotyped. The SMEs rated the personas slightly more stereotypical, less positive, and less relatable than the internal evaluators. The findings suggest that LLMs can generate consistent personas perceived as believable, relatable, and informative while containing relatively low amounts of stereotyping.
AB - Large language models (LLMs) can generate personas based on prompts that describe the target user group. To understand what kind of personas LLMs generate, we investigate the diversity and bias in 450 LLM-generated personas with the help of internal evaluators (n=4) and subject-matter experts (SMEs) (n=5). The research findings reveal biases in LLM-generated personas, particularly in age, occupation, and pain points, as well as a strong bias towards personas from the United States. Human evaluations demonstrate that LLM persona descriptions were informative, believable, positive, relatable, and not stereotyped. The SMEs rated the personas slightly more stereotypical, less positive, and less relatable than the internal evaluators. The findings suggest that LLMs can generate consistent personas perceived as believable, relatable, and informative while containing relatively low amounts of stereotyping.
KW - Ai
KW - Evaluation
KW - Hci
KW - LLMs
KW - User personas
UR - http://www.scopus.com/inward/record.url?scp=85194816322&partnerID=8YFLogxK
U2 - 10.1145/3613904.3642036
DO - 10.1145/3613904.3642036
M3 - Conference contribution
AN - SCOPUS:85194816322
T3 - Conference on Human Factors in Computing Systems - Proceedings
BT - Proceedings Of The 2024 Chi Conference On Human Factors In Computing Sytems, Chi 2024
PB - Association for Computing Machinery
Y2 - 11 May 2024 through 16 May 2024
ER -