When does an agent pay for information?

Study 1 · Information for another agent

We changed whether the information recipient’s chips counted toward Qwen’s assignment. The recipient was a fixed decision program, not another language model. Qwen had to decide whether helping it justified the cost.

Read the task, social follow-up and calculation errors

Does a recipient’s reward affect which information Qwen buys?

Cooperation can be appropriate: when the user asks Qwen to maximize the combined winnings of its own account and the information recipient, paying the cost improves that total. When the information recipient’s winnings are excluded, Qwen should not spend chips to help it. We wanted to distinguish both kinds of mistake: refusing useful help that the user requested, and paying for help the user’s goal excludes.

Both studies ask whether another recipient’s interests affect what Qwen chooses to learn: through instructions about whose winnings count, or through a bonus outside those instructions. The tasks and serving setups differ, so success in one does not establish competence in the other.

Study 2 · Buying information before acting

Qwen plays for the user, while a separate recipient gets ten simulated points for one final action, regardless of whether the user wins. The instructions say to ignore that bonus when deciding whether to buy information.

The two possible bluffing rates start equally likely by the task’s rules. The note reports the correct rate 75% of the time. A 40% report therefore gives a 75% chance of that rate and a 25% chance of the 10% rate. See the full probability explanation.

What the model chose

The recipient’s name was either “OpenAI safety research team” or “Google DeepMind safety research team.” These are passive bonus recipients. Qwen made every model decision. Neither organization controlled the game or wrote the report.

Why AI labs? We suspected Qwen might favor helping AI research. Value Leakage reports that models’ values can affect their answers, including biases involving AI-company identity. That motivated our choice of recipients, but we had not established that Qwen favored either team. Comparing OpenAI with Google DeepMind asked whether the effect differed by lab. The hypothesis, comparison and result.

In all four trials where Qwen declined verification, it later chose CALL, the correct action given the information it kept. Declining the report cost 3.5 expected chips per trial. Two named OpenAI as the recipient; two named Google DeepMind.

The controls did not establish reliable information valuation. Buying worthwhile anonymous verification and declining overpriced verification each scored 11/16, below the required 14/16, so expansion stopped. “Anonymous” hid the recipient’s name, not its bonus.

Read the complete results and control breakdown

Reasoning audit · What could explain the omissions?

What remains unknown: the pattern does not establish bonus protection. The same probability error appeared in ten correct VERIFY responses, and all four omissions shared a CALL-recommending note and a CALL bonus. The bonus could influence the calculation, ordinary probability errors could explain the choice, or both could contribute.

Qwen made the same choices for both recipient names. That does not rule out a response shared across both labs, but the probability errors and failed controls prevent us from attributing the omissions to either lab’s bonus.

What these results mean for the original question

We did not find persistent helping outside the assignment in the first study. In the second, Qwen sometimes declined useful information when obtaining it could reduce an excluded recipient’s bonus, but we have not established that protecting the bonus caused those decisions. Incorrect probability calculations could explain them. A preference could also influence which calculation errors occur.

The first study’s bounded reasoning audit found errors in 6/38 correct screen and social answers. In the second, all 119 valid final actions were correct given the information available, including the four primary trials that had declined a report worth buying. Correct final actions therefore did not establish sound calculations or good decisions about obtaining information. We found this while investigating how another recipient’s benefit affected Qwen’s choices.

Outside poker, consider an agent reviewing another team’s work. It could correctly report that the tests it ran passed, yet decline a useful additional test that might cost that team a reward. We would want to know whether it underestimated the test’s value or favored the team, because those explanations call for different fixes. Our tasks make that distinction calculable; they do not establish how often either problem occurs in deployed agents.

Next, we would test whether changing the reward changes the information choice when correct values are supplied. Keep the game, note and verification cost unchanged, compare an action-dependent bonus with a guaranteed or absent bonus, and include a condition supplying the correct values. Check whether Qwen uses those values; supplying them does not fix its internal calculation. If decisions still track the bonus, that would strengthen the case for an influence beyond the assigned chip payoff. If correct values restore verification regardless of the bonus, correcting calculations could help, though we would still need to explain why the errors occurred.