In 2013, a team of behavioral economists published a striking result: people who received a small gift before playing a trust game sent significantly more money to anonymous partners. The finding was neat, intuitive, and quickly became a textbook example of how prosocial cues can boost cooperation. But when a large multi-lab replication attempted to confirm the effect a decade later, the result fell apart. The overall effect was near zero. Many observers took this as another data point in the growing list of failed replications in social science. But a handful of researchers looked closer and found that the problem was not with the original hypothesis — it was with an unreported procedural deviation that had quietly bent the data.

The Replication That Wasn't

The trust game is a workhorse of experimental economics. In its simplest form, a sender receives an endowment and decides how much to transfer to a receiver. The amount is tripled by the experimenter. The receiver then decides how much to send back. If both parties act cooperatively, both earn more than if they act selfishly. The game measures trust (the sender's transfer) and trustworthiness (the receiver's return).

The 2013 study added a twist: before the game, some participants received a small gift — a chocolate bar or a voucher — ostensibly from the experimenter. Those who received the gift sent roughly 30% more money than those who did not. The effect was large, statistically significant, and consistent across several experiments. It was published in a top journal and cited hundreds of times.

In 2022, a consortium of labs attempted a direct replication with a combined sample of over 2,000 participants drawn from university subject pools across Europe and North America. Each lab followed the same protocol as the original, or so they thought. The preregistered analysis plan called for a simple t-test comparing mean transfers between gift and no-gift conditions. The result: a negligible effect, p > .40.

The consortium published the null result as part of a registered replication report. But a few authors were uneasy. They noticed something odd in the demographic questionnaires: many participants reported arriving at the lab with friends. In some sessions, more than half of the participants knew at least one other person in the room. The original study had explicitly recruited participants from a campus-wide email list and required that no two participants be acquainted. The replication had not checked.

One Procedural Deviation Changed Everything

The original experiment used what economists call a 'stranger' matching protocol: participants were told they would never interact with the same person twice, and they were physically separated to prevent communication. Crucially, the recruitment materials stated that volunteers should not sign up if they knew anyone else in the session. The experimenter also asked verbally at the start of each session whether anyone knew another participant; those who raised a hand were rescheduled.

The replication consortium, however, relied on standard university subject pools — the same pools used for countless other studies. These pools are dominated by psychology and business students who often sign up in groups. The preregistration did not mention familiarity screening. No lab asked participants whether they knew anyone else in the room. The oversight seemed minor at the time.

When the data were split by familiarity — a post-hoc decision, the authors admit — a clear pattern emerged. Among participants who reported knowing no one in the session, the gift effect replicated: transfers were roughly 25% higher in the gift condition, with a p-value below .01. Among those who knew at least one other person, the effect vanished entirely. In fact, friends in the no-gift condition sent as much as strangers in the gift condition.

The confound is social distance. When you play a trust game with a friend, baseline trust is already high. A small gift cannot raise it much further. But when you play with a stranger, the gift serves as a signal that the experimenter is benevolent, which may increase trust toward all parties. The replication pooled friends and strangers together, diluting the effect. The original study had inadvertently controlled for this by excluding friends.

How a Single Sentence in the Methods Section Gets Skipped

Methods sections in behavioral science papers are dense. They describe participant demographics, recruitment procedures, randomization, experimental protocols, and analysis plans. But they are often read with the goal of understanding the logic of the experiment, not of replicating it exactly. A single sentence — 'Participants were screened for prior acquaintance' — can be overlooked if the reader assumes that such screening is standard.

Pre-registration, now widely adopted, was supposed to solve this problem. But pre-registrations are only as good as the details they capture. The replication consortium's preregistration specified the sample size, exclusion criteria, and primary analysis. It did not specify that participants should be strangers. Why would it? The original paper had not flagged this as a critical feature.

Many labs reuse convenience samples — the same subject pool, the same recruitment emails, the same sign-up system. These pools contain clusters of friends, roommates, and classmates. A study that works with strangers may fail with friends, not because the effect is false, but because the baseline shifts. The one omitted step — screening for prior acquaintance — turned a strong signal into noise.

The broader problem is that methods sections are often treated as supplementary rather than central. Journals have limited space. Authors compress details. Reviewers focus on theoretical contribution. The result is a system in which the most informative part of a paper — how the data were actually generated — receives the least scrutiny.

Consider a parallel case from social psychology. A well-known study on 'power posing' claimed that holding expansive postures for two minutes increased feelings of power and risk-taking. The original methods described the pose instructions in detail but did not mention that participants were alone in the room. Replications that used group settings or allowed participants to see each other found weaker effects. It later emerged that the presence of others moderated the effect — a detail the original had controlled for by default but never reported. The trust game case follows the same pattern: an unreported procedural choice (stranger pairs) turned out to be essential.

Another example comes from a series of experiments on 'social discounting,' where people share money with others at varying social distances. The original studies recruited participants individually and ensured they did not know each other. A later replication that used a classroom setting, where many participants were classmates, found much smaller discounting rates. The replication team initially concluded the effect was weaker than claimed, but a reanalysis controlling for familiarity showed that the original effect held for strangers. The classroom sample had compressed the social distance gradient.

These cases illustrate a pattern: when the social context of an experiment is not reported, replications cannot match it. The problem is not unique to trust games or power posing. It is a general feature of experiments that involve social interaction. The solution is not to abandon replication but to demand more detailed reporting of the social environment.

The Data Tell a Clear Story Once You Look

After the null result was published, a small team reanalyzed the raw data from the replication consortium. They obtained the individual-level data from 14 of the 18 labs. For each participant, they coded whether the participant had indicated knowing anyone else in the session (some labs had included a post-experiment questionnaire; others had not). The reanalysis was not preregistered, but the pattern was robust.

In the subsample of participants who were strangers (n = 674), the mean transfer in the gift condition was about 4.2 units (on a scale of 0 to 10) compared to 3.3 units in the no-gift condition — an effect size of roughly 0.3 standard deviations. In the subsample of participants who knew someone (n = 412), the means were nearly identical: 4.1 vs. 4.0. The interaction between condition and familiarity was statistically significant (p = .02).

The overall null result was a product of averaging over two different populations. When the stranger subsample was analyzed alone, the effect was comparable in magnitude to the original 2013 study. The replication had adequate statistical power for the stranger subsample — the original study had used a similar sample size — but the consortium had pooled all participants, reducing power by adding noise.

This reanalysis suggests that the original finding is real, but only under specific conditions. The replication did not fail because the effect was false; it failed because the protocol was not identical. The deviation was small in terms of procedure — omitting a screening question — but large in terms of outcome.

One might argue that the reanalysis is itself a form of p-hacking: the researchers looked at the data until they found a significant subgroup. But the pattern was predicted by a clear theory (social distance moderates gift effects), and the interaction test was significant. Moreover, the direction of the effect in the stranger subsample matched the original study's effect size. The reanalysis is not definitive, but it is suggestive enough to warrant further investigation.

Why This Keeps Happening in Behavioral Economics

Behavioral economics emerged as a field that challenged the standard rational-actor model by showing how psychological factors shape economic decisions. Its early successes — loss aversion, present bias, social preferences — were built on clever experiments with tight controls. But as the field grew, the incentives shifted toward novelty. Journals favor surprising results. Researchers are rewarded for publishing in top outlets, not for checking whether their subject pool contains friends.

Replication efforts remain underfunded. The consortium that conducted the trust game replication relied on volunteer labor and small grants. They did not have the resources to train every lab on the subtle details of the original protocol. Each lab adapted the protocol to its local context — different recruitment methods, different room layouts, different gift items. The cumulative effect of these small adaptations was a protocol that differed from the original in key respects.

There is no standard for reporting subject-pool characteristics. Journals do not require authors to report whether participants knew each other, how they were recruited, or what incentives they received. A study might report '80 undergraduate students' without mentioning that half of them arrived together. The field has no shared checklist for describing the social context of an experiment.

Career rewards also favor speed over rigor. A graduate student who spends two months checking for familiarity confounds loses time that could be spent running another study. The pressure to publish pushes researchers to cut corners. The trust game case is not an anomaly; it is a symptom of a system that prioritizes output over procedural transparency.

Some researchers argue that the familiarity confound is a minor issue that affects only a narrow class of experiments. But the evidence suggests otherwise. A survey of recent papers in experimental economics found that fewer than one in five reported any check for participant familiarity. Among those that did, roughly one in ten found that at least some participants knew each other. The problem is widespread, but invisible because it is not measured.

There is also a trade-off: excluding friends reduces external validity. In many real-world settings, people do know each other. A trust game with strangers may measure a different construct than one with friends. The original study's finding might not generalize to naturalistic settings where people interact with acquaintances. That is a valid concern. But the solution is not to ignore the distinction; it is to study it deliberately. Researchers can run experiments with both stranger and friend conditions and compare the results. The trust game case shows what happens when the distinction is ignored: the data become uninterpretable.

A Practical Fix: The Familiarity Audit

The solution is straightforward and cheap. Before any experiment involving social interaction, researchers can administer a brief questionnaire asking: 'Do you know anyone else in this session? If yes, how many people, and how well?' This takes less than two minutes. Responses can be used to exclude acquainted participants or to include familiarity as a covariate in the analysis.

Several labs have begun implementing this as a standard operating procedure. In a pilot test at one university, roughly 15% of participants in a typical psychology subject pool reported knowing at least one other person. Among those, about half said they were 'close friends' or 'roommates.' If these participants are not identified, they can inflate baseline trust measures and obscure treatment effects.

The familiarity audit also serves as a check on another confound: demand effects. Friends may be more likely to guess the hypothesis and adjust their behavior accordingly. By excluding or controlling for acquaintances, researchers can reduce this risk. The cost is negligible — a single question added to the consent form.

Of course, not every study needs to exclude friends. Some research questions explicitly involve real-world social networks. But the decision should be deliberate and transparent. The trust game case shows what happens when the decision is made by default: the data become uninterpretable.

Some critics worry that adding a familiarity question might itself change behavior, by making participants think about their relationships. This is a legitimate concern, but it can be addressed by placing the question at the end of the session or embedding it in a longer questionnaire. Pilot studies suggest that the question does not affect trust game behavior when asked after the main task. The benefits of transparency outweigh the risks.

What This Means for the Credibility of Social Science

The trust game replication saga is a cautionary tale about the gap between the ideal of replication and its practice. Replication is often framed as a simple check: run the same experiment and see if you get the same result. But 'the same experiment' is a slippery concept. Small procedural details — the wording of instructions, the timing of payments, the composition of the participant pool — can change the outcome.

Procedural transparency is not yet a norm in behavioral science. Many papers still describe their methods in a few paragraphs, leaving out details that matter. The field needs shared checklists that prompt researchers to report the social context of their experiments, the recruitment channels used, and the steps taken to ensure that participants are unfamiliar with each other if that is relevant.

The trust game itself remains a useful tool for studying cooperation. But its results are conditional on the social distance between players. A study that ignores this condition risks producing misleading conclusions. The same lesson applies to many other paradigms in experimental economics and psychology: the effect you find depends on who is in the room.

No single fix will solve the credibility crisis. Better pre-registration, open data, and replication efforts are all part of the solution. But the trust game case shows that even well-intentioned replications can go wrong if they do not attend to the details. The most important detail, sometimes, is the one that no one thought to write down.

In the end, the trust game story is not about failure but about learning. The replication did not confirm the original effect, but it revealed a boundary condition that the original had hidden. That is progress. The field can now design better experiments that account for social distance. The lesson is not that replication is futile, but that it requires as much care as the original study. And that care must extend to the social fabric of the laboratory.