Testing a Sticky Add to Cart Bar Without Polluting the Data

Test the bar on mobile first, where it is supposed to earn its keep, with add to cart rate and revenue per session as the readout, not clicks on the bar itself. Run it as a clean split with the bar as the only change, exclude the product pages where it overlaps a size selector or a chat widget, and pre commit to a minimum sample per device before looking. Most sticky bar wins evaporate when desktop traffic, where the bar adds nothing, is blended into the result.

Last quarter we shipped a sticky add to cart bar on a store I advise, watched the dashboard green for two weeks, and rolled it out. Then someone split the result by device. Mobile, the entire justification for the bar, was flat. Desktop had carried the win, on pages where the regular buy button was already visible most of the time. The bar had not helped anyone check out faster. It had just been present during a good desktop fortnight, and blended reporting happily signed the credit over.

That is the normal failure mode for this test, and it is a measurement failure, not a design one. Sticky bars are device specific medicine. Measure them blended and you will prescribe them on evidence that says nothing about the patients they treat.

Set the test up around the device split

Declare mobile the primary segment before launch and judge the test there first. Desktop can be reported alongside, but it does not get to rescue a failed mobile result, because nobody installs a sticky bar for desktop shoppers with a buy box in view. Set the primary metric as revenue per session with add to cart rate as the diagnostic underneath it. Clicks on the bar itself are a vanity metric; a bar everyone taps that converts no better is furniture that follows you around the page.

Run the split at the session level with the variant sticky across return visits, and exclude your problem templates up front: products with required variant selection the bar cannot represent honestly, pages where the bar covers the cookie banner's accept control, long pages where it fights a chat widget for the same corner. A test polluted by layout collisions measures your collisions, not your concept.

Guardrails: what the bar quietly breaks

Watch the metrics that do not appear on the test dashboard by default. Scroll depth and time on page tell you whether the bar is hurrying people or annoying them. Size selector errors and variant related support contacts tell you whether shoppers are adding the default variant from the bar without choosing, which shows up later as exchanges, the most expensive conversion a store can book. Returns rate on bar attributed orders deserves its own look two weeks after the test ends.

Accessibility is a guardrail, not a garnish. A bar fixed over content must not cover focused elements for keyboard users, needs a contrast passing price and button, and must not trap or reorder focus illogically when it appears mid scroll. Check it at high zoom too. The stores that skip this learn about it from a demand letter rather than a dashboard, which is the priciest analytics tool in ecommerce.

Reading the result and deciding

Give the test its pre committed sample per device segment, not per site, and ignore the first weekend, when novelty and your own team's traffic distort everything. If mobile add to cart rises but revenue per session does not, the bar is harvesting clicks that would have converted anyway. If both rise and the guardrails hold, ship it, on mobile, and resist the victory lap of enabling it everywhere.

And publish the negative result internally if it fails. We killed that first bar, retested a slimmer version six months later with the selector problem fixed, and got a small real mobile win. None of that learning survives if the first test is remembered as the time sticky bars worked. Negative results are the lab notebook. Write them down like they cost money, because they did.

Frequently asked questions

Should a sticky bar show on desktop too?

Only if your desktop data shows shoppers losing sight of the buy button, which on most product layouts they do not. Test desktop separately if you enable it, and do not let desktop results justify a mobile feature or the reverse.

How long should I run a sticky bar test?

Until each device segment reaches the sample size you committed to before launch, which for most mid traffic stores means at least two full weeks including two weekends. Calling it early on a blended green dashboard is how false wins ship.

What is the most common sticky bar mistake?

A bar that adds the default variant without a clear selector, generating wrong size orders and exchanges that never get attributed back to the bar. If the product has variants, the bar must make selection obvious or send the shopper to the buy area instead.

Which metric decides the test?

Revenue per session in the primary device segment, with add to cart rate as supporting evidence and returns, support contacts, and accessibility checks as guardrails. Clicks on the bar alone decide nothing.