Are Large Language Models Ready to Replace Human Survey Respondents? Evidence from a Comprehensive Transportation Survey

30/07/2026

Are Large Language Models Ready to Replace Human Survey Respondents? Evidence from a Comprehensive Transportation Survey

  • facebook
  • linkedin
  • twitter
  • whatsapp
1


Abstract:
Large Language Models (LLMs) are increasingly being explored as a potential means of generating travel survey data, particularly as traditional household activity-travel surveys face growing challenges related to declining response rates, increasing costs, respondent burden, and sampling limitations. While recent studies suggest that state-of-the-art LLMs can generate human-like responses in narrowly defined travel choice settings, such as mode choice among existing transportation alternatives, little is known about their ability to produce coherent, human-like responses across an entire transportation survey spanning multiple behavioral domains and including emerging transportation technologies. To address this gap, we use Gemini 2.5 Pro to generate a synthetic survey response for each respondent in the Transportation Heartbeat of America survey based solely on the respondent's observed individual-, household-, and geographic characteristics. We then evaluate the resulting synthetic dataset along three complementary dimensions: (a) its ability to reproduce aggregate trends in travel behaviors, attitudes, and preferences; (b) its ability to maintain internal response consistency while remaining appropriately sensitive to question wording; and (c) its ability to reproduce meaningful behavioral differences across population subgroups. The results show that LLM-generated responses reasonably reproduce aggregate patterns for relatively simple outcomes, such as driver status and vehicle ownership. However, for more behaviorally complex outcomes, the synthetic data exhibit compressed variation, excessive concentration in dominant response categories, unrealistic attitudinal consistency, and limited ability to reproduce the behavioral heterogeneity observed across population subgroups. Overall, the findings suggest that, while current LLMs show promise as tools for complementing traditional survey methods, they should not yet be viewed as substitutes for human respondents in comprehensive transportation surveys.

About the Speaker:
Prof. Chandra Bhat is the Joe J. King Endowed Chair Professor in Engineering at The University of Texas at Austin, where he teaches transportation systems analysis and transportation planning. He is internationally recognized for pioneering the development and application of statistical and econometric methods to analyze human choice behavior in transportation, urban systems, and public policy. He has received numerous honors for his research and professional contributions, including the 2024 W.N. Carey, Jr. Distinguished Service Award from the Transportation Research Board (TRB), the 2022 Theodore M. Matson Memorial Award from the Institute of Transportation Engineers (ITE), the 2017 Council of University Transportation Centers (CUTC) Lifetime Achievement Award, and the 2013 Alexander von Humboldt Research Award. Dr. Bhat has consistently been ranked among the world's top three researchers in transportation and logistics. He is the immediate past Editor-in-Chief of Transportation Research Part B: Methodological and currently serves as Director of the USDOT-funded National Center for Understanding Future Travel Behavior and Demand.

Click here to register

IIMA