Deploy a swarm of agents to see if it reaches checkout.

Open the lab

Every journey is captured step by step and kept as a replayable environment.

One set of environments for training, testing and evaluating the agents that shop your store

Build on your real checkout, replay every run, and grade each one in code.

  • TrainEnvironments built from real checkouts

    Agents walk top U.S. retailers signed in as members, and each journey becomes an environment a model can learn from.

  • ReplayEvery run, step by step

    Each run keeps its full trace, from store entry to the final total, so you can see exactly where an agent stalled.

  • GradeScored in code, never by another model

    Right item, right quantity and right total are checked line by line before a cart counts as ready.

Training environments

Train your models against replica shopping environments and profiles.

Checkout Bench walks stores the way their members do, signed in, in U.S. browsers, with network traces at every step. Each journey becomes an environment where a model can practice finding the right item, building the cart and reaching a ready checkout, and new retailers are added every week.

Run mix

Mix runs across the stores your shoppers actually use

Every run lands at a real retailer, signed in as a member, so a model trains on the same carts, totals and sign-ins your customers see.

Run mix0.0%0 stores
    Median time from request to quote

    Training data

    Turn every agent run into training data for your own model

    Each run is recorded step by step, from opening the store to the checked total, and becomes a labeled example a model can learn from.

    Real checkout runs
    open storesign insearchchoose productadd to cartread checkoutcheck totalopen storesign insearchchoose productadd to cartread checkoutcheck total
    Training set
    Labeled steps0
    Amazon · signed in Crest Pro-Health Whitening Gel Toothpaste (4.3 oz) × 1

    Item $3.93, shipping $0.00 and tax $0.40 add up to a checked total of $4.33.

    DeepSeek V4.1 Flash1 min 9 sReady checkout

    Evaluation

    Compare models on the exact same checkout tasks

    On the same 94 tasks, DeepSeek V4.1 Flash reached a ready checkout 34% of the time and GPT-5.6 Sol 22%. Replay any run to see the step where one model pulled ahead.

    Browser ready
    Signed in
    Search results
    Checkout read
    Quote checked
    GPT-5.6 SolBaseline
    0% ready on 94 shared tasks
    DeepSeek V4.1 FlashChallenger
    0% ready on 94 shared tasks
    +12

    Read-back

    Give agents only the lines that decide a purchase

    Checkout Bench reads a checkout page down to the item, quantity and total, so every grade points at the line that mattered.

    Buy Crest Pro-Health toothpaste, 4.3 oz, one tube
    amazon.com · checkout page Checkout Bench read-back

    Item Crest Pro-Health Whitening Gel Toothpaste (4.3 oz) × 1

    Total $4.33, which is 3.93 + 0.00 + 0.40

    Checkout Bench organizes environments across every aisle of U.S. retail

    New retailers discovered and added weekly.

    Big box12 retailers
    Grocery13 retailers
    Fashion21 retailers
    Sport & sneakers7 retailers
    Beauty & health6 retailers
    Home & hardware10 retailers
    Pets & auto5 retailers
    Travel9 retailers
    Tickets & dining8 retailers
    Marketplaces & resale8 retailers
    Artist stores5 retailers

    Automate thousands of AI agent runs in one place.

    Sign inNook
    • browser_ready
    • signed_in
    • search_results
    • product_chosen
    product_choice_requires_user

    Two colors matched the case, so the agent asked the person and carried on with “Lime”.

    CartShopify
    • product_page
    • added_to_cart
    • cart_checkout
    Address and identityPlaid
    • address_selected

    The saved home address was already the default.

    PaymentLink by Stripe
    • payment_selected

    The saved card was already the default.

    CheckoutCheckout Bench
    • checkout_read
    • quote_checked
    • quote_published

    Item subtotal $351.20 sits inside the ±$6.98 allowance around $349.00.

    Ready Checkout

    Follow each run from store entry to checkout, with actions and results organized by stage.

    Training data

    Custom training environments for thousands of sites

    Each store becomes an environment where agents learn to catch the wrong size, swap, add-on or fee before a cart is approved.

    Test your AI store performance against competitors.

    Checkout requests by store

    Ready versus blocked checkout requests at leading retailers.

    Anonymized
    Ready CheckoutBlockedStill preparing
    Amazon
    Scorecard
    323
    Ready Checkouts from 479 requests
    Median time
    40.6 s
    Checkout total
    $8.70
    Uber Eats
    Scorecard
    40
    Ready Checkouts from 159 requests
    Median time
    46.0 s
    Checkout total
    $26.79
    Instacart
    Scorecard
    24
    Ready Checkouts from 123 requests
    Median time
    40.4 s
    Checkout total
    $26.84
    Walmart
    Scorecard
    20
    Ready Checkouts from 68 requests
    Median time
    37.0 s
    Checkout total
    $8.80
    Chewy
    Scorecard
    10
    Ready Checkouts from 33 requests
    Median time
    61.2 s
    Checkout total
    $9.35
    Target
    Scorecard
    6
    Ready Checkouts from 42 requests
    Median time
    45.8 s
    Checkout total
    $31.52

    Every scorecard comes from AI agents walking a store's real checkout, signed in as a member, all the way to the final total.

    • Ready checkoutsHow many agent checkout requests reached a checkout ready to place.
    • Where agents stallThe stage where unfinished runs stopped, from store entry to payment.
    • Side by sideYour store next to the retailers you compete with, run the same way.

    See how your store stacks up against different retailers in your industry.

    Common questions

    Read the full FAQ
    What does a retailer get from Checkout Bench?

    A private test of your store. AI agents walk your real checkout, signed in as members in U.S. browsers, and you get each run's trace, a graded receipt, and a scorecard that sits next to the retailers you compete with.

    How are the environments built?

    From real shopping journeys at top U.S. retailers. Agents walk each store the way its members do, signed in where possible, and record network traces at every step instead of scraping public pages.

    Can we build our own model on our checkout?

    Yes, that is what the environments are for. Your runs, traces and graded results become training and evaluation data for a model that fits your own checkout. Book a call and we will scope it with you.

    Which AI models can we test?

    Any model. The testing lab sends different chat models through the same store, so you can compare how each one handles your checkout side by side.

    How does the API handle retries, concurrency and failures?

    Quotes and purchases accept an Idempotency-Key, so the same key and body always return the same result. A store account busy with another run returns 409 in_use, and every blocker carries a typed code such as product_choice_requires_user, product_mismatch or over_budget, with candidate listings attached when the person needs to choose.

    Can stores expose more than checkout?

    Yes. Connected stores and sites publish named operations, such as searching flights or managing price alerts, each with its own input schema, output and approval rule, and public reads run in a clean browser with no account at all. Pilot partners can also add a new site by its web address, and Checkout Bench generates its sign-in and operations.

    Build your next model on the checkout your customers already use

    Open the lab

    Book a call
    with us

    “There should be as many models as there are companies in the world. Because after all, what is a company? A company is a learning system.”.”

    Satya NadellaSatya NadellaChairman and CEO, Microsoft

    We build your private test on this site.