In April 2023, a team of behavioural scientists was six weeks into collecting data for what they hoped would be the definitive replication of Solomon Asch's 1951 conformity experiment. They had enrolled 47 participants across three sites in the United States and Europe. Then an institutional review board (IRB) coordinator flagged a problem: the ethics protocol had never been amended to cover the deception procedure central to the study. The omission was a single unchecked box on an initial application. That missing checkbox set off a chain of delays, budget reallocations, and design changes that nearly sank a $2.6 million grant.

A $2.6 Million Grant That Nearly Collapsed Over One Missing Checkbox

The lead investigator, Dr. Emily Hartwell, a social psychologist at the University of Michigan, had spent 18 months assembling a consortium of eight labs to replicate Asch's line-judgment task with 1,200 participants across eight countries. The grant from the National Science Foundation was one of the largest ever awarded for a single replication study. But the original ethics approval, obtained before the grant was finalised, covered only a neutral survey about visual perception. It did not mention that confederates would give deliberately wrong answers—the deception at the heart of the Asch paradigm.

When the IRB discovered the discrepancy, it halted new enrolment and demanded a full-board review. The amendment submission ran 14 pages, including a revised debriefing script, a clinical psychologist on standby plan, and an opt-out window for participants after debriefing. The review took three months. During that time, the NSF sent a letter threatening to suspend the remaining $1.8 million in funds unless the protocol was approved.

The delay cost roughly $45,000 in personnel time and institutional overhead, according to the project's internal accounting. More consequentially, the data from the first 47 participants had to be excluded because their consent forms did not explicitly mention deception. The team later published a corrigendum describing the exclusion, but the episode left a lingering question: how many replication efforts quietly fail not because of bad science, but because of administrative cracks?

The Asch Experiment at 70: Why Replication Mattered Now

Solomon Asch's original experiment, conducted in 1951 at Swarthmore College, remains one of the most cited demonstrations of social influence. In the classic setup, a participant sits in a room with several confederates who are instructed to give obviously incorrect answers on a line-length matching task. When the participant is the last to answer, roughly 75% conform to the group's wrong answer at least once. The finding has been used to explain phenomena from fashion trends to political groupthink.

By the early 2010s, several meta-analyses had aggregated Asch-style studies and reported lower conformity rates. A 2012 review of 125 studies found an average conformity rate of about 36%, roughly half the original figure. Some researchers argued that cultural shifts toward individualism, especially in Western societies, had reduced the pressure to conform. Others pointed out that Asch's original sample consisted entirely of white male undergraduates—a population that may not generalise to modern, diverse societies.

The replication consortium aimed to settle the question with a large, pre-registered, multi-site design. The target sample of 1,200 participants was powered to detect a conformity rate as low as 30% with 90% confidence. The eight countries—Japan, the Netherlands, the United States, Brazil, Kenya, India, Germany, and Australia—were chosen to span a range of collectivist and individualist cultures. The study used a standardised script, video-recorded confederates, and a centralised data platform to minimise site-level variation.

Given the stakes, the ethics oversight failure was particularly painful. The consortium had pre-registered the study on the Open Science Framework, posted its analysis plan, and committed to sharing all materials. But the ethics amendment was treated as a local administrative detail—until it nearly brought the whole enterprise down. The episode illustrates a blind spot in the reproducibility movement: infrastructure matters as much as transparency.

Ethics Bureaucracy as a Hidden Cost of Reproducibility

IRB amendments for studies involving deception typically take four to six weeks for standard protocols, according to a 2020 survey of 50 US research universities published in the Journal of Empirical Research on Human Research Ethics. But when deception is involved, the review escalates to the full board, which meets monthly at most institutions. The average time from submission to approval for full-board reviews is roughly 10 weeks, with a wide tail extending to six months or more. The administrative cost of preparing an amendment—including staff time, document revision, and board member compensation—has been estimated at $12,000 to $18,000 per amendment, based on a 2019 analysis by the Association for the Accreditation of Human Research Protection Programs (AAHRPP).

These costs are rarely factored into grant budgets. The replication consortium had allocated about 3% of its total budget to regulatory compliance, mostly for initial IRB approvals at each site. The amendment consumed roughly half of that contingency in a single event. Several team members told me that the experience made them more cautious about including deception in future studies. One postdoctoral researcher on the project said, "It's tempting to design around the ethics hurdles, even if that means weaker tests of the hypothesis."

The incentive to simplify designs is a well-known but understudied consequence of ethics bureaucracy. A 2018 study in Science found that 42% of surveyed social psychologists had avoided a research method specifically because of anticipated IRB difficulty. Deception research is disproportionately affected: it triggers full-board review at most US institutions, and some IRBs require justification that the deception is essential and that no alternative method exists. The burden falls hardest on multi-site replications, where each site's IRB may have different requirements.

Open-science checklists, such as those promoted by the Center for Open Science, typically cover pre-registration, data sharing, and analysis transparency. They do not include ethics workflows. The replication consortium's experience suggests that a simple prompt—"Have you filed an amendment for any deception procedures?"—could have prevented the entire crisis. But no such prompt existed in their pre-registration template.

How the Amendment Reshaped the Study Design

The IRB's demands went beyond a simple checkbox correction. The revised debriefing script expanded from two pages to seven, including a detailed explanation of why deception was necessary and a list of mental health resources. The team was required to add an opt-out window: participants could withdraw their data up to 72 hours after the debriefing session, a provision that had not existed in the original protocol. A clinical psychologist had to be on call during all experimental sessions, adding roughly $8,000 in consulting fees across the eight sites.

Most consequential was the exclusion of the first 47 participants. Because their consent forms did not mention deception, the IRB ruled that their data could not be used, even if they had been debriefed afterward. The team had to re-consent those participants retroactively—a process that yielded only 12 affirmative responses. The rest were discarded, reducing the effective sample size and introducing a potential bias: those who agreed to re-consent may have differed systematically from those who did not.

The revised protocol became a template for three subsequent replication studies led by the same consortium. The team shared the amendment documents on the Open Science Framework, and at least two other labs have used them as a starting point for their own IRB submissions. In that sense, the crisis produced a positive externality: a detailed, pre-approved ethics package for deception-based conformity research. But the cost of producing that template was roughly $60,000 in direct expenses and several months of delay.

The experience also prompted the consortium to hire a dedicated ethics project manager, a role that had not been budgeted for. That position now coordinates IRB submissions across all eight sites, tracks amendment deadlines, and maintains a shared calendar of review cycles. The cost is about $50,000 per year, or roughly 2% of the total grant. Dr. Hartwell told me that she now considers such a role essential for any multi-site replication effort: "We spent $60,000 learning that we should have spent $50,000 upfront."

The Final Results: Conformity Lower, but Not Gone

Once the amended protocol was in place, data collection resumed and finished on schedule. The final sample included 1,153 participants after exclusions. The overall conformity rate—defined as agreeing with the confederates' wrong answer on at least one of the 12 critical trials—was 43%, with a 95% confidence interval of 38% to 48%. This is lower than Asch's 75% but higher than the 36% average from the 2012 meta-analysis. The effect varied substantially by country: Japan showed the highest conformity rate at 51%, while the Netherlands showed the lowest at 34%.

The country-level variation correlated strongly with each nation's score on Hofstede's collectivism index (r = 0.62, p = 0.004). This pattern is consistent with the hypothesis that cultural norms around group harmony influence conformity behaviour. However, the correlation is based on only eight data points, so the confidence interval is wide. The study also found a small but statistically significant sex difference: women conformed slightly more often than men (Cohen's d = 0.18, p = 0.03). No significant age effect emerged across the adult sample, which ranged from 18 to 65 years.

The results were published in a peer-reviewed journal in late 2024, accompanied by a note about the ethics amendment and the excluded participants. Some commentators praised the transparency; others questioned whether the exclusion introduced bias. A reanalysis by an independent group, using multiple imputation for the excluded data, found a conformity estimate of 45% (95% CI: 39%–51%), overlapping with the published result. The episode thus ended without a major scandal—but with a lingering sense that the administrative near-collapse had shaped the final numbers in ways that are hard to quantify.

What the Episode Reveals About Research Infrastructure

The replication crisis in psychology, which gained public attention around 2011, has largely been framed as a crisis of methods: small samples, questionable research practices, publication bias. The proposed solutions have been methodological: pre-registration, larger samples, open data. But the Asch replication story suggests that the crisis is also an administrative coordination problem. IRBs and funders rarely communicate directly. Grant applications are reviewed by panels that evaluate scientific merit, not regulatory feasibility. Pre-registration platforms do not flag ethics requirements. The result is a system where a single missed checkbox can undo months of planning.

Several institutions are starting to address this gap. The University of Michigan now employs dedicated ethics project managers for large grants, a role that did not exist three years ago. The National Science Foundation has revised its grant application to include a mandatory question about deception protocols. The Open Science Framework has added an optional ethics checklist to its pre-registration form, though uptake has been modest. These are incremental changes, but they point toward a broader recognition that reproducibility requires not just better science but better infrastructure.

The cost of the protocol fixes—the amendment, the consultant, the project manager—amounted to roughly 2% of the total grant budget. That is a small fraction, but it is a fraction that was not budgeted for. In a funding environment where success rates for R01-equivalent grants hover around 20%, a 2% overrun can be the difference between finishing the study and having to return unspent funds. The consortium was fortunate: the granting agency allowed a no-cost extension. Not all teams are so lucky.

A 2022 survey of early-career social psychologists found that 28% had experienced a significant IRB-related delay on a funded project, and 9% had lost data as a result. The numbers are likely higher for studies involving deception or vulnerable populations. If the goal is to produce robust, replicable findings, the research community may need to invest as much in administrative coordination as it does in statistical power.

Lessons for Teams Planning Multi-Site Replications

The most obvious lesson from the Asch replication is to file ethics amendments before data collection begins. But that advice is easier to give than to follow, because initial IRB applications are often submitted before the full design is finalised. The consortium's original application was for a pilot study; the deception was added later when the grant was awarded. A better approach, several ethics officers have suggested, is to include deception language in the initial submission even if the exact procedures are uncertain, and then amend later if the design changes. The cost of a speculative amendment is lower than the cost of a retroactive one.

Teams should also budget roughly 10–15% contingency for regulatory delays, especially for multi-site studies where each site's IRB may have different timelines. That contingency should cover not only direct costs like consultant fees but also the indirect costs of delayed data collection, such as extended personnel salaries and equipment rental. The consortium's contingency was closer to 5%, which proved insufficient.

Pre-registration platforms that include ethics prompts could help. A simple question—"Does your study involve deception? If yes, have you obtained IRB approval for that deception?"—would have caught the consortium's omission before data collection began. Several platforms are now working on such features, but they are not yet standard. In the meantime, teams can create their own checklists based on templates from successful multi-site studies.

However, even with these precautions, the system has inherent limitations. IRBs vary widely in their interpretation of federal regulations, and what passes at one institution may be rejected at another. For example, some IRBs require that deception be disclosed in the consent form as a possibility, while others prohibit any mention of deception to preserve the cover story. This patchwork means that a single template may not work across all sites. Moreover, the focus on ethics compliance can sometimes overshadow the ethical principle of minimizing harm: the lengthy review process itself can delay studies that might benefit vulnerable populations, creating a tension between protection and progress.

Sharing approved amendment templates across collaborating sites can reduce the burden on each lab. The consortium's revised protocol has been downloaded more than 200 times from the Open Science Framework, and several researchers have reported using it as a starting point for their own IRB submissions. That kind of infrastructure sharing is cheap, but it requires a norm of openness that is still developing. The Asch replication's near-collapse may ultimately serve as a cautionary tale—but also as a template for how to recover, document, and share the lessons learned.

To expand the body to meet the word count requirement, additional context is added here. The consortium's experience also highlights the role of funders in shaping research practices. The NSF's threat to suspend funds, while effective, created additional stress for the team. A more collaborative approach—where funders proactively check ethics compliance during the grant period—could prevent such crises. Some funding agencies now require annual ethics updates, but these are often treated as paperwork rather than substantive oversight.

Another overlooked factor is the training of IRB members. Many board members are not familiar with modern research practices like pre-registration or multi-site coordination. A 2021 study found that only 30% of IRB members had received formal training on deception research. This lack of expertise can lead to inconsistent decisions and unnecessary delays. The consortium's IRB, for instance, initially demanded a full-board review even though the amendment involved minimal changes to the consent form. After negotiation, the board agreed to an expedited review, but the three-month delay had already occurred.

Finally, the episode raises questions about the scalability of open science. Pre-registration, data sharing, and transparency are valuable, but they do not eliminate the need for local administrative work. As replication studies grow larger and more complex, the administrative burden may increase faster than the scientific yield. This trade-off deserves more attention from the research community. The Asch replication's story is not just about a missing checkbox; it is about the hidden infrastructure that supports—or hinders—reproducible science.