October 4, 2026
At least one to two full business cycles, and usually a minimum of two weeks. Most Shopify tests that end after three days are measuring noise: weekday versus weekend behavior, a single traffic spike, or random variation. The right duration comes from your traffic and your conversion rate, not from impatience. Run the sample size math first, then commit to the calendar.
The temptation is structural. You launch a test, check it daily, and on day four one variant is winning by 20 percent. It feels wasteful to keep running. But early leads are mostly noise: with small samples, random variation routinely produces big apparent differences that evaporate as data accumulates. Peeking daily and stopping at the first green signal is how stores ship losing changes with full confidence.
The discipline that separates real testing programs from theater is deciding the duration before launch and honoring it. Pre-commitment beats willpower every time.
Two separate minimums apply, and your test must satisfy both. The statistical minimum is the sample size needed to detect the effect you care about: use a sample size calculator with your baseline conversion rate and the minimum lift worth shipping. A store converting at 2 percent needs far more visitors per variant than one converting at 6 percent to see the same relative lift.
The calendar minimum is one to two full business cycles, which for most stores means one to two weeks. Traffic behaves differently on weekends, paydays, and promotion days. A test that runs Monday to Thursday has never seen your weekend shoppers, and its result does not describe your business.
High-traffic stores, tens of thousands of sessions a week, can often reach significance in one to two weeks for meaningful changes. This is the only segment where short tests are legitimate, and even here a full two weeks is safer than one.
Mid-size stores, a few thousand sessions a week, should plan three to four weeks for most tests. Small stores, under a thousand sessions a week, face an uncomfortable truth: many tests need six to eight weeks, and testing tiny tweaks is not worth it at all. Low traffic means testing only big, bold changes where the effect size justifies the wait.
Stopping early is justified in exactly two cases. One: the test is clearly broken, a variant shows a massive drop caused by a bug, broken layout, or tracking error. Kill it, fix it, restart. Two: an ethical or legal problem surfaced. Everything else, including boredom and a very convincing early lead, is not a reason.
Never stop early because the result is flat, either. A flat result after the planned duration is a finding: the change does not matter. Stopping a flat test early to try something else just guarantees you never learn which changes are truly neutral.
Running tests during promotions, holidays, or traffic spikes contaminates the result, because the test measures the event, not the change. Either exclude those periods or accept that the test describes sale behavior only. Changing the test mid-flight, tweaking a variant on day five, restarts the clock whether you admit it or not.
Running multiple overlapping tests on the same pages is the other classic wrecker. Interactions between tests make both results uninterpretable. One test per page at a time, sequenced, beats three concurrent tests every time.
Decide duration from sample size and business cycles before you launch, write it down, and do not peek until it ends. Two weeks is the floor for most stores, longer for smaller ones. The tests that feel too slow are usually the only ones producing answers worth acting on.
Because early results are dramatic and usually meaningless. The first days mix novelty, weekday effects, and small samples, so a loser can lead by a wide, convincing margin. Ending on that margin adopts noise as strategy. The discipline of a preset end condition exists precisely to protect you from the most persuasive chart in the process: the early one.
For most small to mid sized stores, two to four weeks is the honest neighborhood, long enough to cover full weekly cycles and gather a few hundred conversions per variant on the metric being tested. Lower traffic stretches that window; bigger changes, with bigger expected effects, can shorten it legitimately.
When a variant is broken or harming customers: errors, wrong prices, or a checkout problem. That is an operational stop, not a statistical one. Clear harm to revenue on very large samples is the other rare exception. Everything else, excitement, impatience, a stakeholder's curiosity, is a reason to wait for the planned end.