Groups of eight guess which of two urns was chosen, over twenty rounds. In signals, everyone sees every ball drawn. In the all treatment, they also see every bet the others have made. A Bayesian gains nothing from those bets, because the balls already say everything.

The headline is this gap. Accuracy in all runs about five points above signals, and the paper reads it as imitation repairing noisy counting.

This lesson audits that claim on four points. The size of the effect with its uncertainty. Where, and on whom, each treatment ran. Whether the gain can be attributed to imitation. And what the procedure itself could create.

The regression has forty four thousand rows, but treatment varies by session: eight of the all treatment, three of signals. By our count, the sample for this coefficient is eleven.

Table A one gives five point one points of accuracy, clustered standard error two point eight.

By our arithmetic, a ninety five percent interval runs from just below zero to ten and a half points, and widens with ten degrees of freedom. The paper prints one star, which in its other tables means the ten percent level, and the main text omits it.

The consensus effect, nine and a half points with the same standard error, is clear of zero. Agreement rises. Accuracy probably does.

Next, where the sessions ran. Data collection began in a physical lab at U C San Diego. No info, actions, and four sessions of the all treatment ran there.

Then the pandemic closed the lab. Its other four sessions, and every session of signals, ran online at Ohio State. One cell of the grid is empty. Signals never ran in the lab.

So the headline compares one hundred fifty two subjects from two sites with eighty two from one. As we read it, treatment is tangled with lab versus online, and with which university the students attend.

Only the online row holds the site fixed. That is four sessions against three.

Appendix F is the paper's answer. Its figure splits the all treatment by site.

The curves sit close, and the text reports no significant difference without printing a coefficient. Four sessions against four gives little power.

The larger problem is the missing test. The headline needs all minus signals with site held fixed. Only the online sessions allow that, and the appendix never reports it.

Read off the printed figures by the survey agent, the online gap is about five point four points. That is an estimate from pixels, with no standard error. It suggests the lab sessions do not create the gap, and nothing stronger.

Appendix M compares the two samples. In the all treatment, thirty one percent of subjects study science or engineering, against twenty one percent in signals.

The fix is one to one propensity matching. The text says the finding remains unchanged, and prints no number.

Digitized from the matched figure, the gap is about three point three points, with zero inside the band in most rounds.

The survey also found values that should not occur. Round one, a coin toss, is drawn at sixty three percent. Round two sits nine points above the Bayesian expectation. Signals ends near ninety six percent, against eighty three in Figure one. The check is unusable.

Appendix N looks for subjects who ignore the balls. It fits one sensitivity per subject, beta i, on the Bayesian log odds. Random means the credible interval for beta i includes zero.

In signals that is nineteen of eighty two subjects. In no info, twenty of eighty. About a quarter of each.

Subjects in the all treatment are never classified, so the random share cannot be compared across the two headline treatments.

Nobody drops these subjects and re-estimates the five points. Our reasoning on why it matters: a random subject scores fifty percent, so a modest difference in their share between arms moves the gap by a point or two.

This is the screen from the instructions. Each cell holds a rectangle for the bet and a circle for the ball. There is no running count of red against green.

By round twenty, the Bayesian statistic means counting one hundred fifty two circles.

Last round's bets are one row of seven rectangles. The paper calls imitation a low cognitive cost alternative to counting.

Our inference: part of the counting noise that imitation repairs comes from this interface. An on screen counter would test that. It limits how far the result travels. The comparison stays fair if signals subjects saw the same grid, which the paper never shows.

Subjects in the two all treatments described their strategy in free text. About forty percent of that text describes using balls with others' bets.

Asked whether bets were useful, high I Q subjects put forty four percent of their text on the Bayesian answer, no extra information. Low I Q subjects, seventeen percent.

Yet in the choices, high I Q subjects respond more to bets when the tally is close. The appendix credits those answers to a Bayesian subset. We read a mismatch, and text is never linked to choices.

On beliefs, one regression shows low I Q subjects predict accuracy worse. Subjects in the all treatment were never asked about signals.

One logit predicts a correct guess from the Bayesian log odds and the share of others who were right. Its error against Bayes is computed with those terms off, then on.

Signals cut the average error from point oh seven five to point oh four five, and actions to point oh three five, the paper's one third.

In rounds three to five, actions worsen the fit. The baseline nearly reaches zero at round eight, so the report distrusts its scale.

The model leaves a six point treatment effect that the paper says cannot be attributed to either source. By the survey's digitization, imitation closes a third of the raw shortfall.

Some checks come back clean. Appendix G splits the ten games in half. Measured from its figures, the gap is five point three points early, four point eight late.

Appendix H narrows the definition of a weak tally. The response to others' bets stays steep, and flattens only in the narrowest band.

Appendix L asks who gains. The gap is positive from the tenth percentile up, and largest near the twenty fifth, about eighty four percent against seventy. At the top both reach one.

So the direction is stable under each cut. All three reuse the same eleven sessions, and none speaks to site or to the standard error.

Here is the scorecard, in the survey's ratings. Handled: learning across games, the cutoffs, and who gains.

Partly handled: the site confound, where the clean contrast is omitted. And covariate imbalance, where the matched figure is anomalous.

Open: the ten percent significance level. A coefficient between four and five and a half points across the appendix regressions, where an I Q interaction shifts its meaning. The random subjects. Six unattributed points in Appendix K. Three signals sessions. And the missing tally, our own point.

The verdict is mixed. The direction survives every cut. The size is about five points, standard error near three. Consensus and the imitation mechanism are firmer than the accuracy headline.

Everything here is our proposal. First, cross treatment with site. Both treatments run in each mode, from one subject pool, sessions randomized to arms.

Second, sample size. The reported standard error implies session means varying by about four points. Eighty percent power against five points needs a standard error near one point eight, so about eleven sessions per arm, and about twenty five if the true gap is nearer three.

Third, add an arm with signals plus a running tally. If the gap closes, the interface was doing the work. Classify random subjects in every arm by a preset rule.

Read next: Table A one, Appendix F, the Appendix E screenshot, then Table K three.