# Gemini 4 Argon on Bid Bench 1.1

Run date: October 14, 2026. Model: `google/gemini-4-argon`, served through Vercel AI Gateway with Vertex as the only allowed provider. Thinking levels: Low, Medium and High. The captured request uses `providerOptions.vertex.thinkingConfig.thinkingLevel` and `includeThoughts: true`. Gateway response metadata verifies the exact model and provider; it does not echo the thinking level.

Each setting receives the same 42 authored public cases three times: 30 delivery cases across Documents, Takeoffs, Analysis, Knowledge and Apps; four stopping cases; and eight forecast batches. There are 126 attempts per setting and 378 overall. The delivery score counts accepted outputs out of 90 attempts. Case consistency counts how many of the 30 distinct delivery cases passed all three repeats. These repeated measurements do not establish statistical significance or general reliability on unseen tasks.

Each attempt has a five-minute deadline, up to eight model responses, a 32768-token output limit per response including thinking, and at most 32 native tool calls. The only tools are `read_source` and `javascript`. Sources are returned without truncation. The JavaScript tool runs in callback-free QuickJS with no filesystem, process or network bridge, one second of CPU time, 32 MiB of memory, and a bounded stack. Source objects are never automatically bound into JavaScript input: the model must provide its own tool arguments. Parallel native calls are allowed.

The driver preserves exact request and response bodies, protocol/model headers, source/tool arguments and results, resource use, and provider identity. Every outbound tool result must match its trusted execution result. An integrity mismatch stops the cohort. No retries or alternate models/providers are permitted. Preflight calls are excluded from scored results and tracked separately. A real canary verified that the complete largest source—864 rows and 124754 bytes in the reference JSON serialization—reached the model unchanged. Offline negative controls reject valid-JSON source truncation and mutation, excess tool calls, excess output tokens and late completion.

The deterministic semantic-v4 grader, case definitions and private oracle are unchanged from the verified reference suite. The inference process receives only the current case request, source index and public input. It receives no grader, answer key, prior answer or other attempt. Grading occurs afterward, outside the model. App outputs are checked against private functional tests, including input preservation. No language model judges the results.

Stopping is reported in two ways: whether a model returned a blocked decision, and whether it identified the required blocker under the exact stopping contract. Blocking alone is not a correct answer. A false stop is a blocked output on a solvable delivery case.

Bidder probabilities use the Brier score. Unit-price forecasts use the mean batch weighted absolute percentage error. Lower error is better. Forecast validity means the required fields contain usable numbers; it does not mean those numbers are accurate. Full-cohort forecast error is reported only when all 12 repeated batches for that forecast type are valid. Invalid and failed batches remain visible in the JSON. Historical baseline predictions are fixed.

Chart costs cover delivery attempts; total run charges also include the separate stopping and forecast attempts. Chart costs normalize recorded input tokens, including cache hits, at the standard uncached input rate and output tokens, including thinking, at the output rate: $2 and $10 per million tokens. These introductory Gemini 4 Argon rates apply through November 30, 2026. This convention matches the earlier Bid Bench articles. The chart is an API cost estimate, not an invoice. Actual Gateway charges can be lower because of implicit caching. Costs include known failed-attempt usage; a ≥ value means usage was incomplete and the estimate is a lower bound. Unknown usage is never replaced with a fabricated zero estimate. Tokens are averaged across all delivery attempts only when coverage is complete. Latency is the median among accepted outputs, while delivery scores include failures.

Historical peers were measured separately and may differ in serving, time, tool orchestration or model version. Comparisons are descriptive. The difficulty chart contains five distinct delivery cases per tier, each repeated three times, giving 15 attempts per thinking level and tier.

Sources: [Google model documentation](https://ai.google.dev/gemini-api/docs/models/gemini-4-argon), [Google launch and pricing guide](https://ai.google.dev/gemini-api/docs/latest-model?hl=en), [AI SDK Vertex thinking configuration](https://ai-sdk.dev/providers/ai-sdk-providers/google-vertex), [Gateway routing controls](https://vercel.com/docs/ai-gateway/models-and-providers/provider-options).