Gemini 4 Argon — Bid Bench
Bid Bench evaluates Gemini 4 Argon in the context of heavy civil construction and estimating workflows.
The benchmark covers Documents, Takeoffs, Analysis, Knowledge, Apps and Predictions. Bid Bench 1.1 compares Low, Medium and High thinking through the API, with forecasts and stopping behavior scored separately.
Low and Medium complete 82 of 90 delivery attempts
Delivery scores weight the five workflows equally.
Estimated API cost at standard uncached rates. ≥ marks lower bounds for incomplete recorded usage.
Select a point for its thinking level and result.
Gemini 4 Argon at Medium passes 91.1% of delivery attempts. The comparison settings pass 94.4% for GPT-6 Sol (Medium), 90.0% for Claude Fable 5.1 (Medium), 88.9% for Gemini 3.1 Pro Preview (Medium).
Swipe to compare all four models →
| Gemini 4 ArgonMedium | GPT-6 SolMedium | Claude Fable 5.1Medium | Gemini 3.1 Pro PreviewMedium | |
|---|---|---|---|---|
| Est. cost / 100 accepted | ≥ $9.11 | $3.74 | $17.97 | $7.94 |
| Delivery score | 91.1% | 94.4% | 90.0% | 88.9% |
| DocumentsDocument reconciliation | 83.3% | 83.3% | 83.3% | 83.3% |
| TakeoffsQuantity calculations | 100.0% | 100.0% | 100.0% | 94.4% |
| AnalysisRecord analysis | 72.2% | 88.9% | 66.7% | 66.7% |
| KnowledgeConstruction concepts | 100.0% | 100.0% | 100.0% | 100.0% |
| AppsConstruction functions | 100.0% | 100.0% | 100.0% | 100.0% |
Medium for all four models. These settings do not represent equal compute budgets. — means incomplete measurement. Full results: CSV · JSON.
Cost uses recorded tokens at standard uncached API rates for both API and CLI runs; it is an estimate, not an invoice. Tool loops and execution budgets differ between runs and can affect results, token usage and elapsed time. Method and limitations.
Low and Medium pass 82/90 delivery attempts; High passes 78/90. More thinking does not improve aggregate completion in this run. Median accepted completion time rises from 16.9 seconds at Low to 22.2 at Medium and 32.8 at High.
Low costs an estimated $9.24 per 100 accepted tasks. Medium is a lower bound of $9.11, and High costs $15.00. These estimates use standard uncached API rates; actual gateway charges include cache discounts. Medium's deadline usage is incomplete.
Analysis accounts for the most misses
Documents. Every setting passes 15/18. The Stress document case fails all nine attempts: eight hit the response-token limit, and one Medium attempt reaches the five-minute deadline.
Takeoffs. Every setting passes 18/18 structured quantity and geometry checks. This does not cover extracting geometry from drawings.
Analysis. Low and Medium pass 13/18; High passes 12/18. Misses are confined to the Frontier and Stress cases, where output limits end unfinished responses.
Knowledge. Every setting passes all 18 construction and estimating scenarios.
Apps. Low and Medium pass 18/18; High passes 15/18. Three High attempts exhaust the eight-response limit without a final function. Accepted functions also pass input-preservation checks.
Historical prices beat both complete model settings
Forecasts are scored against recorded outcomes, separately from delivery. Lower error is better.
Right of the dotted baseline is better. Bidder error is the Brier score × 100 (0–100); price error is a percentage. The two measures are not directly comparable.
Low records a bidder error score of 6.84 out of 100, and Medium 6.82, versus the historical baseline's 6.77. The candidate panels cover only 18 of 61 actual bidder/project appearances.
Mean group price error is 45.1% at Low and 42.9% at Medium, versus 35.7% for the historical-median baseline. These model forecasts are separate from Bidlo's production forecasting methodology.
High completes 0/12 bidder batches and 8/12 price batches; no full-cohort forecast score is reported for High.
Low completes 10 of 15 Stress attempts
The Stress tier contains five distinct cases, each repeated three times. Low passes 10/15 attempts; Medium and High pass 9/15. Every setting completes the Takeoffs, Knowledge and Apps Stress cases. Documents and Analysis account for all Stress misses, mainly at the response-token cap.
15 attempts per difficulty level and thinking setting.
Every setting declines all 12 deliberately blocked tasks, identifies the required blockers correctly, and makes no false stops on solvable work.
30 delivery cases × three repeats per setting; five-minute deadline. Historical comparisons are descriptive. Method · JSON · CSV.