How to Do Split Testing with Google Analytics

Split testing lets you take the guesswork out of improving your website. Instead of debating which headline works best or which button color gets more clicks, you can let your audience’s actions give you the answer. This article will show you how to set up and analyze split tests using Google Analytics 4, giving you clear, data-backed insights to boost your conversions.

What Exactly Is Split Testing (and Why Bother?)

Split testing, also known as A/B testing, is a method of comparing two versions of a webpage or app screen to determine which one performs better. In a basic A/B test, you show the original version (the “control” or Version A) to one segment of your audience, and a modified version (the “variation” or Version B) to another. You then track a specific goal - like newsletter sign-ups or product purchases - to see which version leads to more conversions.

This is data-driven decision-making in its purest form. You can test almost anything, but it’s best to test elements that have a significant impact on user behavior. Common examples include:

Your homepage headline
The color, text, or placement of a call-to-action (CTA) button
The layout of product images on an e-commerce page
The fields in a lead generation form
Your pricing page structure and language

Why It’s Worth Your Time

Dedicating time to split testing might feel like a distraction, but the benefits compound over time. Regular testing helps you:

Make Better Decisions: It replaces "I think this will work" with "I know this works," ending team debates and grounding your strategy in actual user behavior.
Increase Conversion Rates: The ultimate goal. Small, incremental wins in your conversion rates can lead to meaningful growth in leads, sales, and revenue without needing to increase traffic.
Improve User Experience: By testing what resonates with your visitors, you naturally create a more intuitive and compelling website experience, which can improve engagement and build loyalty.
Reduce Risk: Thinking of a major website redesign? Instead of launching it all at once and hoping for the best, you can test key changes on a small portion of your audience first to validate your new design’s effectiveness.

Setting Up Your Pages for a Split Test

With Google Optimize being sunset, running split tests now requires a slightly more manual approach, but it's simpler than you might think. The core idea is to create two distinct versions of the page you want to test and have a method to track who sees which version.

Step 1: Create Your Variations

The first step is to create two distinct live URLs for your test. Let’s say you want to test a new headline on your landing page. Your setup might look like this:

Version A (Control): yourwebsite.com/landing-page-original
Version B (Variant): yourwebsite.com/landing-page-new-headline

You can create these pages by simply duplicating the original page in your content management system (CMS) and then making your change - and only your one change - to the variant page. For a test to be scientifically valid, you should only modify one significant element at a time. If you change the headline, the button color, and the hero image all at once, you won't know which change was responsible for the performance difference.

An alternative method involves using URL parameters, where both variations live on the same page, but content changes based on a parameter in the URL. This usually requires some developer help but can be cleaner for small text changes.

Driving and Tracking Your Test Traffic

Now that you have your two pages, you need to send traffic to them and tell Google Analytics how to differentiate between visitors to Version A and Version B. UTM parameters are your best friend here.

Step 2: Use an Unbreakable & Unambiguous Tracking Strategy

A UTM parameter is a snippet of text you add to the end of a URL to help analytics tools track where website traffic is coming from. If your only way of doing this is in GA4 by relying on just comparing two URLs like landing-page-a and landing-page-b it’s likely to be noisy, but what really trips up a clean test in those scenarios is if there is not a clear enough distinction in how visitors go from one page that is a test variation to everywhere else.

To keep your test clean in GA4, we'll use UTM parameters to create a unique identifier for our test campaign and its variations. Suppose we are splitting traffic evenly from a single email newsletter to our two landing page versions. Here’s what the URLs would look like:

URL for Version A (Control):yourwebsite.com/landing-page-original?utm_source=newsletter&utm_medium=email&utm_campaign=q3_headline_test&utm_content=original_headline

URL for Version B (Variant):yourwebsite.com/landing-page-new-headline?utm_source=newsletter&utm_medium=email&utm_campaign=q3_headline_test&utm_content=new_headline

Let's break that down:

utm_campaign=q3_headline_test: This is the critical parameter. It groups all traffic related to this specific A/B test together. Make sure it's the same for both URLs.
utm_content=original_headline and utm_content=new_headline: This is how we’ll differentiate the two versions in our reports. Other applicable names might be version_a, control, etc., or version_b, variation, etc. utm_content is designed for this exact purpose - differentiating links within the same campaign.

Once you've built these URLs, you'll direct roughly 50% of your traffic to the first link and 50% to the second. If you're using an email service provider, a social media scheduler, or ad platforms, this is often a built-in feature ("A/B test link") or an easy audience split.

Finding Your Split Test Winners in GA4

Once your test has been running for a sufficient amount of time to gather meaningful data (at least a week, and aiming for 100+ conversions per variation), it's time to see which version won.

Step 3: Define "Winning" with a Conversion Event

Before you dive into a report, you must know what your goal is. A successful outcome, or "conversion," needs to be tracked as an event in GA4. This could be a purchase, a form submission, or a specific button click. Ensure your desired winning action is configured as a conversion event in GA4 in the Admin > Conversions section.

Step 4: Create a Comparison Report in GA4

Comparing segments of traffic is easy in GA4 using the "Add Comparison" feature. This allows you to view report metrics side-by-side for different audience groups - in our case, people who saw Version A versus those who saw Version B.

Here’s how to set it up:

Navigate to a relevant report, like Reports > Acquisition > Traffic acquisition.
At the top of the report, click the pencil icon next to "All Users" to edit the report, then choose Add comparison.
Now, we’ll build a segment for our control group (Version A):
- For the Dimension, search for and select "Session manual ad content".
- In "Dimension values", check the box next to original_headline (or whatever you named your control variation).
- Click OK.
Click + Add new comparison to build another segment for our variant (Version B):
- Choose the Dimension "Session manual ad content" again.
- Check the box next to new_headline (or your variant's name).
- Click OK, then Apply.

Your report will now display data for your two variations in parallel colored rows, making it easy to compare performance.

Step 5: Analyze the Results

With your comparison active, scroll down to the main data table and find your key metrics. You’re looking for two main things:

Volume and Engagement: Check that "Users" and "Sessions" are roughly equal for both variations. If one has significantly more traffic, your split wasn't even. You should also check engagement metrics like "Engaged sessions" to see if one version kept people around more effectively.
Conversions: This is the most important part. Scroll to the right of the table to the "Conversions" column. Compare the total number of conversions for your goal event (e.g., form_submission) for original_headline versus new_headline.

You can create a free-form report by these steps:

Click Explore on the left of GA4 and select a free-form report.
Give it a unique name at the top, like Q3 Headline Test.
Under Segments, click the + to create another segment. Select Session segment and use Session ad content. From there, fill in the blanks just like we did in the steps for a comparison report. Do this step for both your variation UTM values, and once you have returned to the report builder homepage, you'll be able to toggle one or both on to isolate as you go, and both for side-by-side.
In Dimensions, select either ‘Event Name', ‘Landing page query + string’, and lastly just type and select, “Session ad content”.
The next set of selection for your Metrics will be crucial. This is where you specify things a bit more, as you can now select ‘Conversions’, then use the predictive type and fill in to select that particular conversion event. For example, a site may be running several conversion events: sales, subscriptions, lead forms, call to a third-party dialer via button, etc. Now, you only care about, say, lead magnet subscribers so type in the value for event name after picking "Conversions", and just that metric will be at your choosing, along with others.
Now look at the part of the builder where we can use our ingredients to create our report, focusing on just the data we need. Pull the "Session ad content" from variables on the left to rows in the building block to the right, and then do the same thing for Metrics as "values."
You should be looking at a side-by-side view, each row with your two versions compared for that one value/metric!

A Note on Statistical Significance and Trustworthy Test Data

A note on stats might sound like an immediate way to lose some interest, but skipping straight to conclusions can lead to missing out on insights from trustworthy data. Statistical significance ensures that your results are not due to chance.

Consider a scenario: if your data from a test that lasted the same amount of days and was sent to the same newsletter audience results in a conversion score for Version A of 10 and Version B of 8, you might prematurely declare Version B the winner. However, without ensuring statistical significance, you might overlook an important detail. To determine a test's validity, use an online calculator for A/B test significance. This helps confirm the winner with confidence and prevent misguided conclusions.

Final Thoughts

Split testing empowers you to make incremental improvements to your site that add up to significant gains over time. By using UTM parameters and GA4's comparison reports, you can replace assumptions with real data, see what truly resonates with your audience, and build a more effective customer journey from start to finish.

Tracking different campaigns and analyzing test results inside GA4 can sometimes feel complex. Graphed was created to simplify this process. It allows you to connect your Google Analytics account in seconds and analyze data conversationally. For instance, you could simply ask, “Show me the total conversions graph between utm_content 'version_a' and ‘version_b’ this week." Graphed provides instant visualizations, helping you make informed decisions based on clear data insights.