When multiple artificial intelligence agents review one another's answers, they may be able to detect errors that a single model misses. Yet repeated agreement among several agents does not necessarily mean that the agreed-upon answer is correct. If the agents share the same flawed assumption or reasoning path, an incorrect answer may remain uncorrected and eventually become the group's consensus.
Minjun Cho and Seungwon Jeong investigated this problem by introducing a non-voting logic critic agent into multi-agent debate and then using tabular Q-Learning to determine when the critic should intervene. The key idea was not to let the critic decide the correct answer, but to have it identify only reproducible logical defects in existing solutions and to call it selectively when additional verification appeared useful.
Three AI Agents Debate, While a Fourth Does Not Vote
The study used multi-step reasoning tasks from BIG-Bench Hard (BBH). The solving group consisted of three agents using gpt-4.1-nano-2025-04-14, while the separate logic critic used gpt-4.1-mini-2025-04-14.
The three solver agents first generated their answers independently without seeing one another's responses. They then reviewed the other agents' answers and reasoning over three debate rounds, deciding whether to maintain or revise their own conclusions. After the third round, the final answer was determined by majority vote among the three solver agents.
The critic had a fundamentally different role. It did not act as a fourth voter and was not allowed to recommend a particular answer choice. Instead, it examined the reasoning for identifiable problems such as violations of task conditions, calculation errors, logical contradictions, or reproducible counterexamples.
To reduce the influence of majority size, the critic was not given the identities of the solver agents or the number of agents supporting each answer. Substantively duplicate hypotheses were also merged before evaluation. This design was intended to separate the frequency with which an answer was repeated from the logical validity of the reasoning behind it.
Criticism was also subject to a strict delivery rule. Feedback was passed to the solver agents only when the critic returned VERIFIED_DEFECT, reported confidence of at least 0.80, and provided a concrete witness—such as a calculation, contradiction, or counterexample—that the solvers could independently verify. The solver agents were then free to maintain or revise their answers after re-examining the original problem and the critic's evidence.
Under Pure Debate, 100 of 129 Initial Wrong Answers Remained Wrong
Rather than evaluating only final accuracy, the researchers classified the transition from the initial majority answer to the final majority answer into four categories:
correct → correct
correct → incorrect
incorrect → correct
incorrect → incorrect
In the 300-question evaluation, the initial majority answer was correct on 171 questions and incorrect on 129.
Under pure debate without a critic, 163 of the 171 initially correct answers remained correct, while eight changed to incorrect answers. Among the 129 initially incorrect answers, only 29 were corrected during the debate, while 100 remained incorrect after all three rounds.
Pure debate therefore achieved an overall accuracy of 64.0%, with an initial-error correction rate of just 22.5%.
The results indicate that repeated discussion among the solver agents alone was often insufficient to uncover shared flawed assumptions or reasoning errors.
Calling the Critic Every Round Raised Accuracy to 78%
The researchers next introduced fixed critic intervention, calling the critic in every round for every question. Across 300 questions and three rounds, this resulted in 900 critic calls.
Accuracy increased from 64.0% under pure debate to 78.0%, a gain of 14.0 percentage points. Macro F1 also increased from 0.621 to 0.731.
The transition analysis showed where the improvement came from.
The number of correct → incorrect transitions remained unchanged at eight in both pure debate and fixed critic intervention. In contrast, incorrect → correct transitions increased sharply from 29 to 71.
As a result, the initial-error correction rate rose from 22.5% to 55.0%, while the number of initial errors that remained incorrect fell from 100 to 58.
This suggests that the critic's main contribution was not preventing initially correct answers from being overturned. Instead, it primarily helped expose flaws in answers that the three solver agents had failed to correct through debate alone.
The Cost Problem: Does the Critic Need to Be Called Every Time?
Calling an additional model in every debate round improves opportunities for verification, but it also increases computational cost even when further criticism is unnecessary.
To address this issue, the researchers formulated critic invocation as a sequential decision problem and introduced a tabular Q-Learning-based selective intervention policy.
Q-Learning did not solve the reasoning problem or generate criticism itself. Its sole task was to choose between two actions at each round: CALL the critic or SKIP the critic.
The state used by the policy consisted of:
the current debate round,
the distribution of solver opinions,
the gap between the average self-reported confidence of competing answer groups,
whether the majority answer had changed from the previous round,
and the BBH task type. The reward combined final correctness with a penalty for critic calls. Therefore, if two action sequences produced the same correct final answer, the sequence using fewer critic calls received a higher reward.
Because there were three debate rounds, eight possible call patterns existed, ranging from 000, meaning that the critic was skipped in all three rounds, to 111, meaning that it was called every round.
The study did not run all eight patterns on every training question. Instead, action paths were distributed across the training data according to the initial opinion pattern and task type. Each assigned path was paired with its complementary path—for example, 010 with 101—so that both call and skip outcomes could be observed while limiting the total number of model calls.
Candidate policies were then evaluated using 5-fold cross-validation on separate validation data. The final policy was selected from policies whose validation accuracy was no more than 2 percentage points below that of fixed intervention, choosing the one with the fewest critic calls.
Before final evaluation, the Q-table, state discretization, critic-call cost, and action-selection rule were fixed. Evaluation answers were not used to update the policy further.
1 Percentage Point Lower Accuracy, but 46% Fewer Critic Calls
In the 300-question evaluation, Q-Learning-based selective intervention achieved 77.0% accuracy and a Macro F1-score of 0.703.
Its accuracy was 1.0 percentage point below the 78.0% achieved by fixed intervention. The difference in critic usage, however, was substantial.
Fixed intervention made all 900 possible critic calls. The Q-Learning policy called the critic only 486 times, using the critic in 54.0% of available intervention opportunities.
That represents 414 fewer calls, or a 46.0% reduction, compared with fixed intervention.
Despite the reduced critic use, the selective policy corrected 69 of the 129 initially incorrect majority answers, corresponding to an error-correction rate of 53.5%, compared with 55.0% under fixed intervention.
Because all three methods were applied to the same questions, the researchers used paired statistical comparisons.
Fixed critic intervention exceeded pure debate by 14.0 percentage points in accuracy. The paired-bootstrap 95% confidence interval for the difference was +9.33 to +19.00 percentage points, and the McNemar exact test produced a p-value of 1.57×10−8.
Q-Learning-based intervention exceeded pure debate by 13.0 percentage points. Its 95% confidence interval was +8.67 to +17.33 percentage points, with a McNemar p-value of 1.88×10−8.
The observed difference between Q-Learning and fixed intervention was -1.0 percentage point, with a 95% confidence interval of -3.33 to +1.33 percentage points and a McNemar p-value of 0.581.
The study therefore concluded that no statistically significant accuracy difference was detected between the two critic-based methods in the 300-question evaluation. This does not establish statistical equivalence between the methods; it means that a significant difference was not detected in that evaluation dataset.
Similar Accuracy in the 500-Question Evaluation, but Smaller Cost Savings
The researchers expanded the evaluation to 500 questions while retaining the same state representation and action-selection rules. The Q-values and policy were not readjusted after observing the expanded evaluation results.
Pure debate achieved 62.8% accuracy, fixed critic intervention achieved 75.2%, and Q-Learning-based intervention achieved 75.4%.
Their Macro F1-scores were 0.607, 0.710, and 0.691, respectively.
Across 500 questions, fixed intervention made all 1,500 possible critic calls. Q-Learning called the critic 1,362 times, reducing the number of calls by 9.2%. Total critic-token usage was approximately 10.8% lower than under fixed intervention.
The reduction was therefore considerably smaller than the 46.0% observed in the 300-question evaluation.
The transition analysis nevertheless showed a similar pattern.
All three conditions began with 216 questions whose initial majority answer was incorrect. Pure debate corrected 44 of them, for a correction rate of 20.4%. Fixed intervention corrected 107, corresponding to 49.5%, while Q-Learning-based intervention corrected 106, corresponding to 49.1%.
Correct → incorrect transitions were 14 under pure debate, 15 under fixed intervention, and 13 under Q-Learning intervention.
Again, the performance gains associated with critic intervention were therefore more closely associated with correcting initial errors than with reducing the loss of initially correct answers.
The paper does not report separate paired-bootstrap or McNemar test results for the 500-question evaluation. Accordingly, the observed accuracies of 75.2% and 75.4% should be treated as descriptive results rather than evidence that the two methods are statistically equivalent.
Conformity and Shared Model Errors Remain Difficult to Separate
The study provides evidence that an external, non-voting logic-verification mechanism can help correct errors that remain unresolved during ordinary multi-agent debate. It also shows that some critic calls can be skipped while preserving similar observed accuracy.
However, the researchers identify several limitations.
First, all three solver agents used the same GPT-4.1 model family. The persistence of incorrect consensus therefore cannot be attributed exclusively to social-style conformity between agents. It may also reflect shared error patterns among models with similar training and reasoning tendencies.
The experiments were also limited to multi-step reasoning tasks from BIG-Bench Hard. The results cannot automatically be generalized to open-ended generation, factual verification, planning, programming, or real-world decision-making.
The Q-Learning policy relies on a compressed representation of debate state, including opinion distributions and self-reported confidence. The authors note that a language model's reported confidence does not necessarily correspond to its true probability of being correct, and discretizing a complex debate history into a small number of categorical variables may discard useful information.
The difference in call reductions—46.0% in the 300-question evaluation versus 9.2% in the 500-question evaluation—also indicates that the efficiency of selective intervention may depend substantially on the distribution of debate states in the evaluation set.
Finally, the study did not compare Q-Learning against baselines that use the same critic budget but select calls randomly or according to simple rules, such as intervening only in 2:1 disagreements or when confidence gaps fall below a threshold. The current experiments therefore do not establish that Q-Learning identifies intervention opportunities more effectively than such simpler selective strategies.
The authors suggest that future research should compare heterogeneous model families, introduce random and rule-based intervention baselines, and incorporate token usage, latency, and actual API cost directly into the reward function.
Overall, the findings suggest that improving multi-agent reasoning may require more than simply adding more agents or extending the number of debate rounds. An independent mechanism for examining the logic behind competing answers—and selectively allocating that verification resource according to debate state—may provide one path toward balancing reasoning performance with computational efficiency.
Study: A Study on Mitigating Majority Conformity in Multi-Agent Debate through a Critic Agent and Q-Learning-Based Selective Intervention
Authors: Minjun Cho and Seungwon Jeong