Claude Sonnet 5.5 — Bid Bench
Bid Bench evaluates Claude Sonnet 5.5 in the context of heavy civil construction and estimating workflows.
The benchmark covers Documents, Takeoffs, Analysis, Knowledge, Apps and Predictions. Bid Bench 1.1 compares delivery quality across thinking settings, with forecasts and stopping behavior scored separately.
High completes every delivery attempt
Delivery scores weight the five workflows equally.
Estimated API cost at standard uncached rates. ≥ marks lower bounds for incomplete recorded usage.
Select a point for its thinking level and result.
Claude Sonnet 5.5 at Medium passes 94.4% of delivery attempts. The comparison settings pass 64.4% for Claude Sonnet 5 (Medium), 94.4% for GPT-6 Sol (Medium), 91.1% for Grok 4.7 (Medium).
Swipe to compare all four models →
| Claude Sonnet 5.5Medium | Claude Sonnet 5Medium | GPT-6 SolMedium | Grok 4.7Medium | |
|---|---|---|---|---|
| Est. cost / 100 accepted | $12.47 | $30.41 | $3.74 | $10.28 |
| Delivery score | 94.4% | 64.4% | 94.4% | 91.1% |
| DocumentsDocument reconciliation | 88.9% | 83.3% | 83.3% | 100.0% |
| TakeoffsQuantity calculations | 100.0% | 77.8% | 100.0% | 88.9% |
| AnalysisRecord analysis | 83.3% | 61.1% | 88.9% | 100.0% |
| KnowledgeConstruction concepts | 100.0% | 72.2% | 100.0% | 100.0% |
| AppsConstruction functions | 100.0% | 27.8% | 100.0% | 66.7% |
Medium for all four models. These settings do not represent equal compute budgets. Historical peers were measured September 28–30 using different API/CLI conditions; these are descriptive comparisons. Full results: CSV · JSON.
API-equivalent estimates at standard uncached rates. ≥ marks incomplete recorded usage. Method.
High passes 90/90 delivery attempts. Low passes 84/90, Medium 85/90, XHigh 88/90 and Max 71/90. Median accepted completion time is 9.9 seconds at Low, 10.4 at Medium, 10.8 at High, 13.3 at XHigh and 33.0 at Max.
Low costs an estimated $9.54 per 100 accepted tasks at standard uncached API rates; Medium costs $12.47 and High $10.85. XHigh and Max are lower bounds of $11.69 and $30.23 because recorded usage is incomplete.
Max loses completions in Apps and Analysis
Documents. Low and Medium pass 16/18, High and XHigh 18/18, and Max 15/18.
Takeoffs. XHigh passes 17/18; every other setting passes 18/18. These are structured quantity and geometry checks, not geometry extraction from drawings.
Analysis. Low and Medium pass 15/18, High and XHigh 18/18, and Max 12/18.
Knowledge. Low passes 17/18; every other setting passes 18/18.
Apps. Low, Medium and High pass all 18 executable function tests. XHigh passes 17/18 after one response-budget failure. Max passes 8/18, with ten output-limit or deadline failures.
Medium beats the historical price baseline
Forecasts are scored against recorded outcomes, separately from delivery. Lower error is better.
Right of the dotted baseline is better. Bidder error is the Brier score × 100 (0–100); price error is a percentage. The two measures are not directly comparable.
Medium’s mean group price error is 33.7%, versus 35.7% for the historical-median baseline. Every other setting is worse than the baseline, ranging from 36.3% at High to 41.7% at Max.
No setting beats the historical bidder baseline. XHigh comes closest at 6.81 out of 100 versus 6.77. Max completes none of its 12 bidder batches, so its error score is withheld. The candidate panels cover only 18 of 61 actual bidder/project appearances. These forecasts are separate from Bidlo’s production forecasting methodology.
High clears all five Stress cases
The Stress tier contains five distinct cases, each repeated three times. Low, Medium and XHigh pass 13/15 attempts, High 15/15 and Max 7/15. Resource failures remain in the score.
15 attempts per difficulty level and thinking setting.
Every setting identifies all 12 deliberately blocked tasks correctly. Medium falsely stops once on solvable work; the other settings have no false stops.
30 delivery cases × three repeats per setting; five-minute deadline. Historical API and CLI comparisons are descriptive. Method · JSON · CSV.