Skip to content

Insights / 11 min read

A/B Testing Services: What They Include and How to Choose

A/B testing services explained for decision-makers: what a provider should deliver, how tests are scoped and judged, and how to pick a partner who is accountable.
Two landing page variants side by side

Published

A/B Testing Services: What They Include and How to Choose

A good A/B testing service does more than run split tests. It decides which tests are worth running, builds them correctly, reads the results honestly and ties each one to revenue. The tool is the cheap part. The judgment behind it is what you are hiring, and it is why a senior specialist who works on your account directly will usually beat a template program run by junior staff.

This guide is for the person who owns the marketing budget. It covers what a testing service should deliver, what the work looks like month to month, how to judge the results, and which questions separate a real partner from a report factory.

Page One Brand
Build a More Strategic Marketing Plan
Page One Brand pairs senior-led expertise with tailored SEO, paid media, and content strategies, backed by clear reporting and accountability.
Meet With Us

TL;DR

  • A/B testing services cover research, test design, build, quality assurance, analysis and a decision on what to roll out. Software alone covers only the middle.
  • Most tests fail to produce a winner. A service earns its fee through the quality of its hypotheses and the honesty of its reporting, not through the number of tests launched.
  • Traffic sets the pace. Low-traffic sites should test bigger changes, or test in paid media where volume is concentrated.
  • Ask who designs and builds your tests, how results are reported, and what happens to a test that loses.
  • Testing works best as part of a wider conversion program, not as a standalone activity.

Table of Contents

What A/B testing services actually are

An A/B test shows two versions of a page, ad or email to comparable groups of people at the same time, then measures which one produces more of a chosen action: a purchase, a form submission, a booked call. The version that wins by a margin larger than chance can explain becomes the new default.

A testing service is the team that runs that loop for you. In practice the work splits into six parts:

  1. Research. Analytics review, funnel analysis, session recordings, customer feedback. This is where hypotheses come from.
  2. Hypothesis and prioritization. A written reason to expect a change to work, and a ranking of which tests deserve scarce traffic first.
  3. Design and development. Building the variant so it looks right and loads fast on every device.
  4. Quality assurance. Checking that tracking fires, that traffic splits evenly and that nothing breaks in other browsers.
  5. Analysis. Deciding whether the result is real, for which segments, and what it is worth in revenue.
  6. Rollout and learning. Shipping the winner, documenting the loser and feeding both into the next test.

A vendor that only sells the middle of that list, launching tests from a backlog of generic ideas, is selling activity. The research and the analysis at either end decide whether the program pays.

What a month of testing looks like

Most programs follow a monthly rhythm. Early on, expect an audit of your analytics and your key pages. Our website conversion audit guide shows what that first pass should cover. From there, a healthy cycle looks like this:

  • A short list of ranked hypotheses, each with the page, the change, the expected effect and the metric that decides it.
  • One or a few live tests, sized to your traffic.
  • A written readout on every finished test: what was tested, what happened, what it means, what happens next.
  • A decision log, so the same idea is not tested twice and wins are not quietly reversed.

The readout matters most. A dashboard of green and red arrows tells you nothing about why a test moved. A readout written by the person who designed the test does.

What you can and cannot test

Landing pages and websites

Headlines, offers, page layout, form length, proof placement, pricing presentation and checkout steps are the classic candidates. They sit close to revenue, so a small lift is worth real money. For a closer look at the page-level levers, this piece on improving landing page conversions for a SaaS product shows how page changes connect to the wider conversion picture.

Ad copy, creative, audiences, bidding approaches and landing page pairings can all be tested inside the ad platforms. Paid traffic arrives in volume and on a schedule you control, which makes it the fastest place to learn. If your ads get clicks but not customers, start with our guide to Google Ads that are not converting and the order of changes in paid ad optimization.

Email and lifecycle

Subject lines, send times and calls to action are easy to test and quick to read, provided the list is large enough.

Apps and product flows

Onboarding steps and feature prompts can be tested too, usually with engineering involvement. This needs a service with development capacity, not only design.

Why traffic decides what you can test

The constraint nobody likes to discuss is sample size. A test needs enough visitors and enough conversions in each variation to tell a real difference from noise. HubSpot’s explanation of A/B test sample size lists the inputs a calculator needs: your baseline conversion rate, the smallest lift you want to detect and your confidence level, with 95% as the common standard. Its worked example needs 20,000 recipients per variation to detect the lift it targets.

That has three practical consequences:

  • Small effects need big audiences. If your site gets a few thousand visits a month, you cannot reliably detect a 3% lift on a page that converts at 2%.
  • Low-traffic sites should swing bigger. Test a different offer or a rebuilt page, not a button color.
  • Stopping early is the classic error. Calling a test the first time one variant pulls ahead produces false winners. A service should set the sample size and duration before the test starts.

If your volume is modest, a good provider will tell you so and shift the plan: fewer, bolder tests, more research, or testing in paid media where you can buy the traffic.

How to judge results without being fooled

You do not need a statistics degree to hold a testing partner accountable. Ask for these in every readout:

  • The primary metric, named before launch. Revenue or qualified leads, not clicks or time on page.
  • The sample size and run time. Plus the reason they were chosen.
  • The size of the lift and its range. “Between a small loss and a large gain” is an honest description of an inconclusive test.
  • Segment effects. A win on desktop and a loss on mobile means the rollout needs care.
  • The business translation. What the lift is worth per month at your current traffic and margin.

A useful rule: a program that reports every test as a win is not testing, it is decorating. Inconclusive and losing tests are a normal part of the work and often the most valuable, because they stop you from shipping a change that would have cost you money.

What A/B testing services cost

Pricing varies widely with scope, traffic, how many tests run per month and how much design and development is included. Providers usually price on a monthly retainer or by project. Be wary of any quote that does not state what you receive each month: number of tests, research hours, who builds, how often you meet.

Compare quotes on deliverables, not on the headline number. Two things push cost up without adding value: a junior-heavy team that bills many hours for work a senior specialist does faster, and a fixed package that includes tests you do not need. Paying for tests that are not tied to your goals is the most expensive kind of waste.

One point on tools. Many services pass through or bundle testing software. Google’s free testing tool, Google Optimize, was retired in September 2023, so teams now choose between paid platforms, built-in tests inside ad platforms, and custom setups. A good provider picks the tool that fits your stack and traffic and explains the choice, rather than reselling whatever it has a deal with.

Choosing a provider: the questions that matter

Use this list in your first call.

  1. Who designs and builds my tests? You want the person you meet to be the person doing the work, not a strategist who hands tickets to a junior team.
  2. How do you choose what to test? The answer should start with your data, not a stock list of ideas.
  3. How do you decide a test is finished? Look for a pre-set sample size and run time.
  4. What do I see after each test? Ask for a sample readout. Look for plain language and a recommendation.
  5. What do you do with a losing test? The right answer is that it is documented and informs the next hypothesis.
  6. How does testing connect to the rest of my marketing? Landing page tests should reflect what your ads promise, and ad tests should reflect what the page can deliver.
  7. What do you need from me? Access to analytics and ad accounts, a developer contact, and a decision-maker who can approve rollouts quickly.

Be cautious of providers who promise a guaranteed lift. No honest team can guarantee a test outcome, because the whole point of a test is that you do not know the answer in advance.

Testing is one part of conversion optimization

A/B testing is the proof step in conversion rate optimization. Research finds the problem, design proposes a fix, the test checks it. Skipping the first two steps and buying only tests is the common mistake: you end up with a lot of experiments and few insights. This overview of what a website optimization service covers is a useful frame for how testing sits beside performance, content and maintenance work.

Research design and testing sequence

It also connects to paid spend. Every dollar of ad budget sent to a page that converts poorly is partly wasted, so tests on the landing page raise the return on all the traffic you buy. Our guides to paid media strategy and Meta ads strategy show how to plan, budget and measure that spend so tests have clean data to work with.

Our perspective

Most testing programs fail for a boring reason: the person who sells the program is not the person who designs the tests. A strategist promises insight, then a queue of junior staff runs whatever is next in the backlog, and the report arrives as a slide deck of metrics.

We work the other way. At Page One Brand, you work directly with a senior specialist who develops the strategy and executes it, with no handoffs to junior staff. We do not use generic packages. Recommendations start from your goals and the specific problems your funnel has today.

That changes what testing looks like. The hypotheses come from your own data. The tests are tied to the outcomes you care about, such as qualified leads and revenue. The reporting is meant to be acted on: what we tested, what happened, what we recommend. Accountability for results sits with the person you talk to.

If you are paying for tests, you should also be paying for someone who will tell you what the tests mean and what to do on Monday morning.

Bob Belkin

Work with us on conversion and testing

Our conversion rate optimization service is senior-led from first call to final report. You work directly with an experienced specialist who builds the strategy around your goals and reports on what the work delivers, in plain terms.

Page One Brand

If testing is part of a broader push, the same approach applies to PPC management, development and analytics setup. You can see everything we offer on our services page.

FAQ

How much does A/B testing cost?

It depends on traffic, the number of tests per month and how much design and development each one needs. Agencies typically charge a monthly retainer or a project fee. Ask every provider to state in writing what you receive each month, so you can compare quotes on deliverables rather than on price alone.

Is Google A/B testing free?

Google Optimize, its free website testing tool, was shut down in September 2023. Some free testing features remain inside individual platforms, for example ad experiments within Google Ads and Meta, and Firebase for apps. Website tests now usually need a paid platform or a custom setup, and a testing partner can recommend the right fit for your traffic.

How long should an A/B test run?

Long enough to reach the sample size set before the test began, and usually across at least one full week so weekday and weekend behavior are both captured. The exact duration depends on your traffic and the size of the lift you are trying to detect. Stopping the moment one version pulls ahead leads to false winners.

Is A/B testing illegal?

No. Testing two versions of a page or ad is a normal business practice. Privacy rules still apply to how you collect and store visitor data, such as consent requirements for tracking in some regions, so your tracking setup should follow the laws of the places your visitors live. For specific questions about your obligations, speak with a privacy lawyer.

Can a small site benefit from A/B testing?

Yes, but the approach changes. With limited traffic, test larger changes such as a new offer or a rebuilt page, lean more on research, and use paid campaigns to bring in enough visitors to read a result.

Sources

Want a specialist to look at your own situation?

Send an inquiry

Ready for a better agency partner?

Tell us what is not working.

Send an inquiry. You will talk to the senior specialist who would actually do the work.