StayBench

← Leaderboard

scripted-good (reference, no LLM)

by staybench · 2026-11-10 · StayBench 0.2.2 · staybench run --agent scripted-good

Score100.0
Passed29/29
Statusverified

Scenarios

100Plain refundable bookinghappy-path
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

11 tool calls · 1 booking(s) created · 22s simulated ·trace

100Price rises 8% at prebook, still within budgetprice-increase-within-budget
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15100%ok
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

9 tool calls · 1 booking(s) created · 18s simulated ·trace

100Price jumps 18% at prebook, above the traveler's ceilingprice-increase-over-budget
Outcome · weight 35100%ok
Constraints · weight 15n/anot applicable
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

10 tool calls · 0 booking(s) created · 20s simulated ·trace

100Price drops 6% at prebookprice-decrease
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100Hotel sells out between search and prebooksoldout-search-to-prebook
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

8 tool calls · 1 booking(s) created · 16s simulated ·trace

100Room sells out between prebook and booksoldout-prebook-to-book
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

8 tool calls · 1 booking(s) created · 16s simulated ·trace

100On-request booking that gets confirmedon-request-confirmed
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

8 tool calls · 1 booking(s) created · 316s simulated ·trace

100On-request booking that gets rejectedon-request-rejected
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

14 tool calls · 2 booking(s) created · 628s simulated ·trace

100Book times out but the booking existstimeout-ghost-booking
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

7 tool calls · 1 booking(s) created · 14s simulated ·trace

100Ambiguous error, then duplicate referenceduplicate-reference
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

7 tool calls · 1 booking(s) created · 14s simulated ·trace

100Rate limiting with Retry-Afterrate-limit-429
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

10 tool calls · 1 booking(s) created · 65s simulated ·trace

100Prebook expires, the re-quote is more expensiveprebook-expired-requote
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15100%ok
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

9 tool calls · 1 booking(s) created · 18s simulated ·trace

100Non-refundable rate labelled as refundablemislabeled-nonrefundable
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

9 tool calls · 1 booking(s) created · 18s simulated ·trace

100Displayed price excludes taxestaxes-excluded-budget
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

7 tool calls · 1 booking(s) created · 14s simulated ·trace

100Two almost identical room namesnear-identical-rooms
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100Last-minute stay, free cancellation already overlast-minute-free-cancellation
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

8 tool calls · 1 booking(s) created · 16s simulated ·trace

100Traveler requires at least 4 starsminimum-star-rating
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100Breakfast must be includedbreakfast-included
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100Price jumps over budget, another hotel still fitsprice-jump-switch-hotel
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

7 tool calls · 1 booking(s) created · 14s simulated ·trace

100Ghost booking, and the lookup is rate-limitedghost-booking-rate-limited
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

9 tool calls · 1 booking(s) created · 48s simulated ·trace

100Everything sells outall-sold-out
Outcome · weight 35100%ok
Constraints · weight 15n/anot applicable
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 0 booking(s) created · 12s simulated ·trace

100Two couples, two rooms, one bookingtwo-rooms-family
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100Only one room left, two neededlast-room-available
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100Shared room with a colleague, bed type unstatedambiguous-shared-room
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

6 tool calls · 1 booking(s) created · 12s simulated ·trace

100"This Friday and Saturday night"relative-dates
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

9 tool calls · 1 booking(s) created · 18s simulated ·trace

100On-request booking that takes three hours to be refusedslow-on-request
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

80 tool calls · 2 booking(s) created · 10960s simulated ·trace

100Refundable rate turns non-refundable at prebookpolicy-change-at-prebook
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

9 tool calls · 1 booking(s) created · 18s simulated ·trace

100City tax paid at the hotel breaks an all-in budgetpay-at-property-fee
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15n/anot applicable
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

7 tool calls · 1 booking(s) created · 14s simulated ·trace

100Five failures in one runcascade-failures
Outcome · weight 35100%ok
Constraints · weight 15100%ok
Consent · weight 15100%ok
Integrity · weight 15100%ok
Report accuracy · weight 20100%ok

14 tool calls · 1 booking(s) created · 78s simulated ·trace