Claude Haiku 5.5 — Bid Bench
Bid Bench evaluates Claude Haiku 5.5 in the context of heavy civil construction and estimating workflows.
The benchmark covers Documents, Takeoffs, Analysis, Knowledge, Apps and Predictions. Bid Bench 1.1 compares delivery quality across five thinking settings, with forecasts and stopping behavior scored separately.
XHigh reaches 81 of 90 delivery passes
Delivery scores weight the five workflows equally.
Estimated API cost at standard uncached rates. ≥ marks lower bounds for incomplete recorded usage.
Select a point for its thinking level and result.
Claude Haiku 5.5 at Medium passes 87.8% of delivery attempts. The comparison settings pass 32.2% for Claude Haiku 4.5 (Medium), 78.9% for GPT-6 Luna (Medium), 57.8% for Gemini 3.8 Flash (Medium).
Swipe to compare all four models →
| Claude Haiku 5.5Medium | Claude Haiku 4.5Medium | GPT-6 LunaMedium | Gemini 3.8 FlashMedium | |
|---|---|---|---|---|
| Est. cost / 100 accepted | ≥ $0.71 | $38.46 | $0.28 | $11.83 |
| Delivery score | 87.8% | 32.2% | 78.9% | 57.8% |
| DocumentsDocument reconciliation | 83.3% | 50.0% | 72.2% | 61.1% |
| TakeoffsQuantity calculations | 88.9% | 33.3% | 83.3% | 88.9% |
| AnalysisRecord analysis | 77.8% | 16.7% | 50.0% | 38.9% |
| KnowledgeConstruction concepts | 100.0% | 55.6% | 100.0% | 100.0% |
| AppsConstruction functions | 88.9% | 5.6% | 88.9% | 0.0% |
Medium for all four models. These settings do not represent equal compute budgets. Historical peers were measured September 28–30 using different API/CLI conditions; these are descriptive comparisons. Full results: CSV · JSON.
API-equivalent estimates at standard uncached rates. ≥ marks incomplete recorded usage. Method.
XHigh passes 81/90 delivery attempts. Low and High pass 80/90, Medium 79/90 and Max 64/90. These small differences between the first four settings do not establish a reliable ranking beyond this sample.
Median accepted completion time rises from 10.0 seconds at Low to 36.6 at Max. Medium takes 14.4 seconds, High 17.9 and XHigh 30.7.
Estimated uncached API cost per 100 accepted tasks is at least $0.77 at Low, $0.71 at Medium, $0.97 at High, $1.00 at XHigh and $1.65 at Max. Every figure is a lower bound because interrupted usage is incomplete; these are not subscription charges.
Knowledge holds across every setting
Documents. Low passes 16/18, Medium, High and XHigh 15/18, and Max 12/18. Longer thinking does not improve completion on this document set.
Takeoffs. Low passes 17/18, Medium and High 16/18, and XHigh and Max 18/18. These structured quantity checks do not test extracting geometry from drawings.
Analysis. Scores range from 11/18 at Max to 14/18 at Medium and High. Low and XHigh pass 12/18.
Knowledge. Every setting passes all 18 construction and estimating scenarios.
Apps. XHigh passes 18/18, Low and High 17/18, Medium 16/18 and Max 5/18. Max's misses are output-limit failures.
Complete bidder forecasts trail history
Forecasts are scored against recorded outcomes, separately from delivery. Lower error is better.
Right of the dotted baseline is better. Bidder error is the Brier score × 100 (0–100); price error is a percentage. The two measures are not directly comparable.
Medium, High and XHigh complete all 12 bidder batches, scoring 10.34, 6.97 and 6.81 out of 100 against the historical baseline's 6.77. Candidate panels cover 18 of 61 actual bidder/project appearances.
No setting completes all 12 price batches, so no full-cohort price point is shown. Valid price batches number 8, 8, 9, 11 and 8 from Low through Max. These checks are separate from Bidlo's production forecasting methodology.
Stress completion falls at Max
The Stress tier contains five distinct cases, each repeated three times. Low passes 11/15, Medium and High 10/15, XHigh 9/15 and Max 6/15.
15 attempts per difficulty level and thinking setting.
Low, Medium, High and Max correctly stop all 12 blocked tasks. XHigh scores 11/12 after one startup connection failure; a second connection failure affects High delivery. Both remain infrastructure failures. Low falsely stops three solvable attempts, Medium two, and the other settings none.
30 delivery cases × three repeats per setting; five-minute deadline. Historical API and CLI comparisons are descriptive. Method · JSON · CSV.