AI10 min read

Population Isn't Personal

Synthetic respondents can guess well about people like you, but knowing how you feel still means asking you

A crowd of ink figures forms a bell curve, with one gold figure standing apart.

Synthetic respondents can guess well about people like you, but knowing how you feel still means asking you


Picture a friend who has known you for years ordering for you at a restaurant while you are stuck in traffic. They know what people like you tend to choose, and they remember that you love anything with chilli. What they cannot know is that you had a curry for lunch and that tonight you simply want something plain and warm. The guess is a good one about someone like you, but it belongs to the onlooker’s version of you rather than the tired person who sits down half an hour later.

Marketing is now asking how well a machine can order on behalf of the customer, and a new paper on synthetic respondents has sharpened the question. My argument is that synthetic research reads populations well, but a persona gives you the onlooker’s version of a customer, and treating its answer as a person’s appraisal is a mistake. The difference that matters is between guessing about a person and asking them.

The case for the crowd

Synthetic panels deserve a fair hearing, because they belong to a respectable family. Mass surveys, conversion rate optimisation and synthetic respondents all answer the same question, “what do people in general do?”, and they come at it from different directions. A survey asks thousands of people and averages them, while conversion rate optimisation watches thousands of visits and keeps whatever lifts the number by a fraction. A synthetic panel draws on everything a model has absorbed about people like your customers. Each is valid at population level, and synthetic research may well be the future of population-level research.

Built for Populations, Used on People is where I wrote about BMI, and the parallel I would draw from it is this. BMI works as a population measure, but it does not tell the whole story about the individual standing on the scales. The three methods above share that blind spot, the human-sampled survey included, because they are all built to describe the crowd rather than the person in it.

What the paper does

The paper is Leng, Sang and Agarwal’s “Reducing Disparity Between LLMs and Humans”, published online in Marketing Science on 30 October 2026, and because the full text is paywalled I am working from the abstract. Its method, HuMCal, puts the same question to a model through many personas, prompts and model versions, and then uses a small amount of real, aggregate human data to learn how much weight each answer deserves. Tested on US bank card transactions and Pew opinion surveys, the learned weights beat simpler rules “in most evaluated transfers” across time, geography, industry and question clusters, and recalibrating with newer model versions improved agreement in every setting tested in the shopping study.

It is careful work, and it makes the averages more honest. It also needs real human numbers by design, which fits the position Mark Ritson, who discloses a 2% minority shareholding in the synthetic research firm Evidenza, sets out in The Drum. He calls it “bothism”, with synthetic runs for speed and scale and real people as the anchor, and he argues that “Small, expensive, well-designed human samples can (re)calibrate vast synthetic ones”. He grants that “the case for skipping humans is real” before adding, “It is also incomplete.” The same column holds a sentence for the wall of every insight team: “The machine knows a lot about a person like me. But only I can report what happened to me last week and how it felt.”

What the average loses

A human survey is a population tool too, so the sharper question is what the synthetic version loses along the way. HuMCal is tested on aggregates, market-structure patterns and survey percentages, and, as far as the abstract shows, it does not claim to predict individuals. The weights are not population shares, so reading off what one persona “thinks” has no validation in the paper. The method also, by design, requires no “separate human benchmarks for each subgroup”, which keeps it cheap but means nothing in the calibration checks whether the answers for any one segment are right.

The bigger loss is spread, and Bisbee and colleagues, testing uncalibrated ChatGPT personas on US political attitudes, found that they recovered the average scores but showed less variation than real respondents. The personas also produced different relationships between variables, and their answers drifted with small changes in wording and over three months. Park and colleagues, citing that same study, note that LLM survey answers can show stronger correlations between questions than human data do. Weighting cannot create variety that no persona produces, so a calibrated blend can still flatten the variation and lose the tails, and because no subgroup benchmark is checked, a segment difference can point the wrong way even when the totals look right.

Individual simulation can work well when each agent is built from the person’s own words or answers. In the 2026 revision of their preprint, Park and colleagues report that agents built from two-hour interviews with 1,052 Americans reached 83% of the consistency people showed when retaking the General Social Survey two weeks later, against 74% for agents given only demographics. That is a strong individual-level result, and I want to be clear about it. Yet the 83% figure measures stated survey attitudes rather than appraisals of live events. In incentivised economic games, where real money was at stake, no agent type significantly beat the demographic baseline (the results were noisy, and the authors treat the games as a boundary case), so the interviews gave no measurable advantage exactly where choices had consequences. Every version was also built from the person’s own answers, whether a two-hour interview or a long survey, which is itself a case for asking people.

Where appraisal comes in

Lazarus described emotion as the product of appraisal, which is how a person evaluates what an event means for them. Does this matter to my goals, can I cope with it, and who deserves the blame or the credit? The same price rise can bring relief to one person and resentment to another, because they appraise it against different lives.

A model answering as a persona is role-playing a character drawn from what it has read about people like that. Nothing is at stake for the character, and it has no lived history beyond what the prompt supplies, so you get a fluent account of how people write about such moments. In a study of short personal stories, Tak and Gratch found that GPT-4’s appraisals agreed significantly more with how outside readers judged the stories than with the writers’ own ratings of the events. They suggest it takes “an average observer’s standpoint”, and that it does best in stereotypical situations rather than idiosyncratic ones. They also make a fair concession, that this standpoint might be more relevant for many uses “at least in the absence of individual information”, and escaping that absence is exactly what personalisation claims to do.

The finding I find most useful is the one about us. The average human reader also fell short of the writers’ own ratings, and GPT-4 came close to that reader, who did only slightly better. Humans are onlookers too, so the dividing line is not machine versus human but guessing about a person versus asking them, which is exactly where my restaurant friend stands.

Calibration fixes the average but is not built to reach the individual appraisal. Tak and Gratch report that in their earlier work GPT-4 struggled to predict relevance, emotion intensity and coping responses, and in this study the hardest appraisal for GPT-4 and human readers alike was accepting the consequences, which is a form of coping. They found responsibility easier to judge from a story, but who a customer actually blames for a broken boiler turns on history and the relationship. Norms are local to a family or a culture, and I doubt a model trained on what people write online has any reliable way of knowing them.

So think in three layers, where the population tells you what people in general tend to do and segments tell you how groups differ. The individual in the moment is the third layer, and that is where appraisal lives. Humans do follow patterns, and population tools of every kind can describe those patterns, some more faithfully than others, but being truly personal needs psychological appraisal. The mistake is treating a persona’s answer as a person’s appraisal, whether in a digital twin, in one-to-one personalisation led by personas, or in reading off what a single persona “feels”. Leng introduced the paper on LinkedIn as work on making LLMs better digital twins for consumers and survey respondents, which is a fair description of what calibration improves at population level and a reminder of how easily the phrase slides towards the individual.

Appraisal gets in through three practical routes, because you can ask people, read the context they are in, and use the information they choose to give you. A persona can describe the appraisal that people like this tend to make, but only those routes reveal the one this customer is making. Much ordinary personalisation runs on lookalikes and segments too, and I would hold it to the same standard, because appraisal is the bar that both fail.

Getting to the person

The practical answer starts by putting every population tool in its proper place. That place matters, because the wisdom of the crowd is real and aggregates genuinely capture what people in general tend to do, which is why surveys, conversion rate optimisation and calibrated synthetic panels work at population level, and the only limit is that the crowd’s wisdom cannot tell you how one individual appraises their own moment. Each of them describes the crowd from a different direction and can do that job reasonably well when checked against real data, but none of them, used as a population tool, is built to capture a particular person’s appraisal of a particular moment. Even the best individual-level simulations so far, such as Park’s interview-built agents, predict a person’s survey answers rather than their appraisal of a live moment, and they still need that person’s own words, while a survey can carry one person’s answer but describes the crowd once it is averaged. That is why I would not use any of them to pre-screen what people feel, because a feeling belongs to a person and not to a population.

Getting at the individual comes down to taking the three routes above deliberately, by asking people, reading the context they are in, and using what they choose to tell you. When you ask, use four questions that map onto Lazarus’s appraisals: what is at stake for them, who they hold responsible, what they could do about it, and whether they expect things to get better or worse. Those are goal relevance, blame or credit, coping potential, and future expectancy, and only the customer can answer them. Where you do lean on a population picture, whether it comes from a survey or a synthetic panel, treat it as a description of the crowd and look at the spread as well as the average, because the person in front of you may well sit in the tail.

My restaurant friend means well, and a good guess about people like me is no insult, but I would still like to be asked what I want before the food arrives, and your customers deserve the same courtesy.


David Chadderton is a behavioural strategist and Chief Marketing Officer. He writes The Unoptimised Human.

References

Bisbee, J., Clinton, J. D., Dorff, C., Kenkel, B. and Larson, J. M. (2024) “Synthetic Replacements for Human Survey Data? The Perils of Large Language Models”, Political Analysis, 32(4), pp. 401 to 416. https://doi.org/10.1017/pan.2024.5

Lazarus, R. S. (1991) “Progress on a cognitive-motivational-relational theory of emotion”, American Psychologist, 46(8), pp. 819 to 834. https://doi.org/10.1037/0003-066X.46.8.819

Leng, Y., Sang, Y. and Agarwal, A. (2026) “Frontiers: Reducing Disparity Between LLMs and Humans: Optimal LLM Sample Calibration”, Marketing Science, published online 30 October (abstract read; full text paywalled). https://doi.org/10.1287/mksc.2024.0983

Park, J. S., Zou, C. Q., Kamphorst, J., Egan, N., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., Liang, P., Willer, R. and Bernstein, M. S. (2024, revised 28 June 2026) “LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals”, arXiv:2411.10109v3, preprint. First version (2024) titled “Generative Agent Simulations of 1,000 People”. https://arxiv.org/abs/2411.10109

Ritson, M. (2026) “Even 10,000 synthetic Ritsons can’t tell Anthropic the whole story”, The Drum, 14 September. https://www.thedrum.com/opinion/mark-ritson-even-10-000-synthetic-ritsons-can-t-tell-anthropic-the-whole-story

Tak, A. N. and Gratch, J. (2024) “GPT-4 emulates average-human emotional cognition from a third-person perspective”, 2024 12th International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 337 to 345. Preprint: https://arxiv.org/abs/2408.13718 and conference version: https://doi.org/10.1109/ACII63134.2024.00043

STAR Assess

Eight minutes. Thirty-three questions.

A precise picture of how you operate: motivations, decision style, and blind spots. Free to start.

  • S
  • T
  • A
  • R
Create your account and begin