Deploy a swarm of agents to see if it reaches checkout.
Every journey is captured step by step and kept as a replayable environment.
One set of environments for training, testing and evaluating the agents that shop your store
Build on your real checkout, replay every run, and grade each one in code.
- TrainEnvironments built from real checkouts
Agents walk top U.S. retailers signed in as members, and each journey becomes an environment a model can learn from.
- ReplayEvery run, step by step
Each run keeps its full trace, from store entry to the final total, so you can see exactly where an agent stalled.
- GradeScored in code, never by another model
Right item, right quantity and right total are checked line by line before a cart counts as ready.
Training environments
Train your models against replica shopping environments and profiles.
Checkout Bench walks stores the way their members do, signed in, in U.S. browsers, with network traces at every step. Each journey becomes an environment where a model can practice finding the right item, building the cart and reaching a ready checkout, and new retailers are added every week.
Run mix
Mix runs across the stores your shoppers actually use
Every run lands at a real retailer, signed in as a member, so a model trains on the same carts, totals and sign-ins your customers see.
Training data
Turn every agent run into training data for your own model
Each run is recorded step by step, from opening the store to the checked total, and becomes a labeled example a model can learn from.
Item $3.93, shipping $0.00 and tax $0.40 add up to a checked total of $4.33.
Evaluation
Compare models on the exact same checkout tasks
On the same 94 tasks, DeepSeek V4.1 Flash reached a ready checkout 34% of the time and GPT-5.6 Sol 22%. Replay any run to see the step where one model pulled ahead.
Read-back
Give agents only the lines that decide a purchase
Checkout Bench reads a checkout page down to the item, quantity and total, so every grade points at the line that mattered.
Item Crest Pro-Health Whitening Gel Toothpaste (4.3 oz) × 1
Total $4.33, which is 3.93 + 0.00 + 0.40
Checkout Bench organizes environments across every aisle of U.S. retail
New retailers discovered and added weekly.
Automate thousands of AI agent runs in one place.
- browser_ready
- signed_in
- search_results
- product_chosen
Two colors matched the case, so the agent asked the person and carried on with “Lime”.
- product_page
- added_to_cart
- cart_checkout
- address_selected
The saved home address was already the default.
- payment_selected
The saved card was already the default.
- checkout_read
- quote_checked
- quote_published
Item subtotal $351.20 sits inside the ±$6.98 allowance around $349.00.
Follow each run from store entry to checkout, with actions and results organized by stage.
Training data
Custom training environments for thousands of sites
Each store becomes an environment where agents learn to catch the wrong size, swap, add-on or fee before a cart is approved.
Test your AI store performance against competitors.
Checkout requests by store
Ready versus blocked checkout requests at leading retailers.
- Median time
- 40.6 s
- Checkout total
- $8.70
- Median time
- 46.0 s
- Checkout total
- $26.79
- Median time
- 40.4 s
- Checkout total
- $26.84
- Median time
- 37.0 s
- Checkout total
- $8.80
- Median time
- 61.2 s
- Checkout total
- $9.35
- Median time
- 45.8 s
- Checkout total
- $31.52
Every scorecard comes from AI agents walking a store's real checkout, signed in as a member, all the way to the final total.
- Ready checkoutsHow many agent checkout requests reached a checkout ready to place.
- Where agents stallThe stage where unfinished runs stopped, from store entry to payment.
- Side by sideYour store next to the retailers you compete with, run the same way.
See how your store stacks up against different retailers in your industry.
Common questions
Read the full FAQWhat does a retailer get from
Checkout Bench?
A private test of your store. AI agents walk your real checkout, signed in as members in U.S. browsers, and you get each run's trace, a graded receipt, and a scorecard that sits next to the retailers you compete with.
How are the environments built?
From real shopping journeys at top U.S. retailers. Agents walk each store the way its members do, signed in where possible, and record network traces at every step instead of scraping public pages.
Can we build our own model on our checkout?
Yes, that is what the environments are for. Your runs, traces and graded results become training and evaluation data for a model that fits your own checkout. Book a call and we will scope it with you.
Which AI models can we test?
Any model. The testing lab sends different chat models through the same store, so you can compare how each one handles your checkout side by side.
How does the API handle retries, concurrency and failures?
Quotes and purchases accept an Idempotency-Key, so the same key and body always return the same result. A store account busy with another run returns 409 in_use, and every blocker carries a typed code such as product_choice_requires_user, product_mismatch or over_budget, with candidate listings attached when the person needs to choose.
Can stores expose more than checkout?
Yes. Connected stores and sites publish named operations, such as searching flights or managing price alerts, each with its own input schema, output and approval rule, and public reads run in a clean browser with no account at all. Pilot partners can also add a new site by its web address, and
Checkout Bench generates its sign-in and operations.
From the
Checkout Bench blog
All postsSee how two Pacific Surfliner tickets went from a station search to a $144 checkout, with all 34 screens shown side by side with our annotations and digital twins.
ResearchOpenAI's plugin guide asks stores to return the facts our browser agents get stuck on at checkoutWe matched each field in OpenAI's commerce plugin guide to the step where agents stopped in our runs at real store checkouts.
CompanyIntroducing Checkout BenchWe're an LA-based, AI-first data lab that builds bespoke training environments for top retailers.