BlogResearch

Getting into a store is harder than buying from it

About half of agent sign-ins work, and once signed in at a big store, an agent rarely hits a bot wall at checkout.

Checkout Bench run log: fresh sign-ins fail at Ulta, United, Ticketmaster, Sephora and AXS, while a signed-in run reaches checkout

Agents fail at sign-in far more often than at checkout. In our production records, fresh sign-ins, where the agent logs in as a returning member, worked about half the time. In lab runs that started with a working session at Amazon, Walmart, Target, Chewy, Uber Eats and Instacart, almost every request got past sign-in, and bot protection stopped about 0.5% of runs. Most failures came after search, when the agent had to pick a product.

Sign-in is where agents stop

Some failures happen before the login form: Opus sometimes could not find the sign-in button on a store's homepage. On our single-visit walks to the payment page, several stores sent a one-time code to email or phone, and a handful used Google sign-in. Ulta required a password and an SMS code, United made the agent type into a popup, and Ticketmaster held the agent at sign-in for most of a day. Sephora refused sign-in in one browser where Ulta, on the same kind of trip, did not.

Bot checks are rare past the login

Most stores we walked had no bot check on the path to payment. The checks we did see came at or just after sign-in: Ticketmaster's hold, Sephora's refusal and AXS's repeated human checks. Dick's showed a robot check at Add to Cart, while Fanatics and JD Sports, selling a similar hat, did not. SeatGeek and StubHub let the agent straight in, and stores that easy to enter can take sales from rivals that block.

For retailers: fix sign-in for agents

Hidden sign-in buttons, login challenges and emailed or texted codes stop agents before they build a cart. To let agents shop your store:

We can run agents through your checkout and show where each one stops, sign-in included. Request a private test of your store.

For shoppers: be ready at sign-in

When an agent buys from a store where you have an account, expect to supply the code, Google prompt or password. After that, at a large store, the agent is more likely to stop with a question about the product than at a bot wall.

Sources

Markdown version of this page