One of the worst mistakes you can make on your website is to implement major changes without running any tests.
Any major change you make on your site will either increase or affect conversions.
Wouldn’t you wanna know what’s working and what’s not?
There are a bunch of A/B testing tools you can use to validate what moves the needle and what doesn’t. Some of these tools are free, some are cheap and some are pricey. Not running A/B testing on the website is now more like a choice – but an expensive one.
Nowadays, having low traffic is not a valid reason to not conduct A/B testing. Top CRO experts say that you can still launch A/B tests on low traffic sites, you just have to:
- Limit the number of variations.
- Test pages with the most traffic.
- Test significant changes instead of small ones.
- Track micro-conversions, and not macro-conversions.
So, if you’ve not been conducting A/B testing, then you have to start now. It’s never too late. If you still have no idea what kind of a tool to go with, then you should continue reading this article.
In this article, I will walk you through what you need to do before you choose an A/B testing tool, mistakes you should avoid when choosing an A/B testing tool, and the top A/B testing tools you should consider.
What to do Before Choosing an AB Testing Tool
As mentioned in the intro, there are different types of A/B testing tools, each and every tool has its own set of pros and cons (will get to these later). This means that not every A/B testing tool is right for your site.
So, how do you know which tool is right for your site and which was is not?
Well, a good starting is to ask yourself the following questions:
- Which platforms need testing?
- What exactly do I need to test? Conversions? AOV? Revenue? RPV?
- How many tests will I run every month?
- What is my budget?
- Do I need a tool with available technical assistance? Or do I have a team of developers?
Answers to the above questions will give you ideas on what kind of a tool you need. At least, you won’t into the A/B testing marketplace with no idea.
Suppose you now have the answers to the above questions, the next step is to think of the features that make a good A/B testing tool.
I had a chat with Tim Mehta, the CRO Manager at Root Inc., about what factors to consider when choosing an A/B testing tool, and this is what he had to say:
Factors should include any data or niche case studies about how that tool worked with your CMS. So, you need to choose a tool that has been proven to work on a platform like yours.
Tim also adds:
You can have a lot of questions about the tool, which you don’t have the internal resources to answer. So, you will want a tool that has accessible training materials as well as customer representatives who are available to answer these questions.
Tim also mentioned a factor that may be significant to many people: the type of stats engine or statistical technology that exists in this tool. He said:
There are a lot of companies that personalization or auto personalization is becoming more necessary to keep up with the market, especially content delivery services that have a lot of data about users.
Those companies may lean towards a tool that better accommodates auto personalization.
Also, one of the factors that intrigued me and I asked Tim about was testing velocity.
Should it be determined by the type of tool you pick?
According to Tim, yes:
If you are using a tool that is not as easy to manipulate, using JavaScript and CSS might take longer to launch tests and QA because you will probably have more bugs.
As far as the build speed, if your developers are already familiar with a particular tool or have never built in a tool before, that will help you decide which tool to use to accommodate that resource.
That means that testing velocity can be a game-changer for companies who want to launch multiple variations in every test, but they don’t have the sample size to do it.
Mistakes to Avoid When Choosing an AB Testing Tool
According to Tim, one of the most common mistakes people make when choosing an AB testing is choosing a package plan that is too big for what they need.
His advice is to start with a plan that suits your traffic size and number of conversions, then gradually move to a bigger one as your numbers increase.
Another mistake worth mentioning is going for a tool simply because some of your friends use the same tool. If a tool is best for your competitors it doesn’t necessarily mean that it will be a good tool for you.
Don’t assume that all A/B testing tools are the same, they are different somehow.
In this regard, Tim says:
Other tools wouldn’t have existed if there was no opportunity for differentiation. So, I would say shop around multiple tools, not just the top two.
With all that said, now let’s take a look at the top AB testing tools you should consider:
Top AB Testing Tools
The following A/B testing tools have been chosen based on these criteria:
- Testing velocity.
- Pricing.
- Customer support efficiency.
- How easy it is to set up and use.
- Integrations.
- Targeting and segmentation capacity of the tool, etc.
FigPii
What is FigPii?
FigPii is a conversion rate optimization platform established in September 2016.
It can be considered a subsidiary of Invesp, established in 2006 since it used to be a tool within the latter.
FigPii offers a full suite of CRO tools, including AB testing.
FigPii Pros
- FigPii has both AB testing and Split URL testing.
- It offers native integration with Shopify and WordPress, which includes revenue reporting.
- The offer consists of advanced targeting.
- Custom JavaScript targeting.
- Unlimited Subdomains.
- Targeting by campaigns (query).
- Visitors targeting (New vs. Returning).
- Source targeting.
- It has several goals set up like custom events (JavaScript), Page visits, cross-domain tracking, and domain-wide goals.
- FigPii has 2 options when it comes to statistical models like
- Bayesian (A winner will be decided after a variation has a significant enough chance to win, variation weights are not changed).
- Multi-armed bandit algorithm (Variation weights are continuously redistributed to give more weight to better-performing variations)
- It has IP targeting for quality assurance services.
- It helps you launch and pause your tests at specific timings according to your preference.
- It has integration with Google Analytics, where you can use custom dimensions to analyze your data further on Google Analytics.
- FigPii offers unlimited concurrent tests.
FigPii Con:
- FigPii doesn’t support mobile application tests.
FigPii Pricing
FigPii Integrations
- For Google Tag Manager, Shopify, WordPress, and WooCommerce, they have a native integration with FigPii. This means that once you install the FigPii application, it will be integrated with your website automatically.
- For the other ones like unbounce, WiX, and others, here is a step by step guide on how to integrate FigPii with any website.
Technically, FigPii can integrate with any website.
Google Optimize
What is Google Optimize?
Google Optimize is a Google tool and the follow up to their previous A/B testing tool Google Content Experiments, built into Google Analytics.
Google Optimize is made for small to medium-sized businesses.
It lets you test different variations of your website and landing pages.
The selling point of Google Optimize is its integration with Google Analytics. It gives you automatic access to behavioral insights and various segments.
Google Optimize Pros
Easy to set up and use
Just like signing up for Google Analytics, all you have to do in Google Optimize is enter your email address.
Also, it is easy to use. You insert the link to the website you want to optimize, then list your variants.
Then, you will be asked to download a free Google Chrome extension so you can edit your website in live mode.
After you download the extension, you can change the parameters of each element during your website testing, like the color, size, opacity, etc.
Native navigation
You can link your Google Optimize account with your Google Analytics account.
Google Analytics will help you have data to assist you in Google Optimize. For example, about pages you need to test, for how long, which device to run the tests on; mobile or desktop or both, etc.
Price
This is another selling point for Google optimize; it is free!
As I mentioned before, all you have to do is have a Google account. There are not any hidden fees or in-app purchases.
Your free account offers not only A/B tests but also multivariate tests.
Just for your information:
An A/B test compares the conversion rates of two versions of a landing page based on live traffic. It tracks how visitors interact with the landing page they are on.
Multivariate testing uses the exact core mechanism as A/B testing but compares more variables and reveals more information about how these variables interact with one another. As in an A/B test, traffic to a page is split between different design versions.
A multivariate test aims to measure the effectiveness of each design. However, it works just like A/B tests. The only difference is that it compares between more variables.
Google Optimize Cons
Three experiments simultaneously
Google Optimize only allows you to run five experiments simultaneously, both mobile and desktop.
Despite not being an obstacle for small businesses that are just getting started and newly introduced to the concept of A/B testing, it can hinder medium-sized companies.
Multivariate testing for significant tests
Google Optimize only gives you 16 combinations in your multivariate tests. This can be challenging for big tests, not small ones.
Google Analytics segments
In Google Optimize, you can’t use Google Analytics segments unless you have a Premium Google Analytics 360 plan, which costs around $150,000 per year.
Segments is a personalization functionality that provides you with quality A/B testing.
They help you break down the data and narrow it down based on your needs.
Google Optimize Reviews
Here are some of the reviews about the Google Optimize A/B testing tool from G2 website:
You can also read: A/B testing tools: Omniconvert vs. Kameleoon
Adobe Target
What is Adobe Target?
Adobe Target is an Adobe Experience Cloud solution.
It is an enterprise tool that combines targeted testing and personalization.
Adobe Target offers two features, A/B tests and multivariate tests. You create test variations by selecting and swapping out, modifying, or dragging and dropping content.
Moreover, Adobe Target’s self-learning algorithm associate visitor behavior and content to give you the best results.
Adobe Target Pros
Adobe Target has some admirable features, like:
- Its reporting feature can tell you how many visitors have seen the particular experience, hits by the user on a specific page area, which country has the most views, the winner based on the report, and many more.
- It provides both form-based and visual-based testing, which is suitable for developers.
- You can create the audience that you wish to target. You can also categorize the audience based on the query string, country, region, and user Ids.
- You can create as many experiences under a single activity, which helps analyze customer behavior more efficiently.
- The WYSIWYG editor allows you to create experiments without coding skills. This is useful for UX people who are creative but lack coding skills.
Adobe Target Cons
As for the disadvantages of Adobe Target:
- Support is a bit slow, often not knowledgeable about local installation – If you have critical processes, there’s little ability to escalate support without multiple calls.
- While writing scripts in form-based type, the text area is minimal to write the code, so you might need to write on an editor then paste it in that area.
- It does not capture revenue out of the box. You need to deploy an additional tool called Adobe Experience Cloud Manager to get conversion data.
- Editing an activity on a password-protected webpage can be tricky. This happens because it loads the webpage within Target; you have to sign in via the embedded frame and then get it to load correctly, which has been problematic.
- The software is costly. You cannot segment your tests or campaigns by geo-targeting, for example, unless you pay extra for that portion of the package.
- The software is mainly centered around the “Target” portion of the product. Once testing completes, there are many options to push your winning campaigns live and target them to specific segments. Unfortunately, this adds to the expense of the software; you pay by impressions.
Adobe Target Pricing
The pricing for the Adobe Target testing tool can estimate as follows:
| Business size | Pricing |
| Small Businesses | Can pay from $15 to $25. |
| Medium Businesses | Can pay approximately $50.However, some other products can reach $150 per user monthly. |
| Large Businesses | Can pay from $400 to $1,500 per month. |
List of Integrations Supported by Adobe Target
Adobe Target integrates with various Adobe products like Adobe Analytics, Adobe Campaign, Adobe Audience Manager, etc.
Adobe Target Reviews
Here are some of the customer reviews about Freshmarketer, according to the Gartner Peer Insights website:
You can also read: A/B testing tools: VWO vs. Optimizely
Convert
What is Convert?
Convert is a high-powered testing and personalization tool.
It is a Climate Neutral Certified A/B testing platform.
Convert Pros
When it comes to the advantages of Convert:
- Convert has reliable and responsive support to fix problems and set up complex experiments.
- It’s pretty cheap compared with competitors.
- Powerful and affordable for growth-stage SMBs looking for high ROI
- The tool is very flexible and can be used for more straightforward tests right through to much more challenging and rewarding testing but does require a more thorough knowledge of the platform.
- Easy test setup with lots of pre-configured options and simple enough to construct your own.
- Easy to implement. Once the script is implemented, you can begin A/B testing immediately. Unlike other software, it’s not a long and arduous process to get the setup work done.
- Javascript-based tests are straightforward to set up and execute. Some of our simpler tests are functional in 30mins or less.
- Real-time reporting is very beneficial. You can share a Convert Report to any level of intelligence on your team, and they will understand what’s happening.
- It doesn’t take very long to set up your experiment.
- Advanced editing. Convert allows you to edit the variation code directly. This means that you can test any website (including AJAX and highly dynamic sites).
- Easy breakdown and visualization of daily conversion patterns.
- “Blink” handling. Convert takes great pains to reduce or eliminate the “blink” that comes with any javascript rewrite-based A/B testing software.
Convert Cons
As for the disadvantages:
- Experiments often don’t appear to you after saving new JS for up to 5 minutes. This can be a bit hectic if you’re going from Local Development > Staging > Production.
- There are some minor UI quirks inside the experiment editor if you’re using the more advanced features as we do.
- There can be a steep learning curve for uninitiated developers who want to create more than a simple A/B test. It takes some time to fully understand how the platform works, how changes execute in the browser, etc.
- You won’t segment the visitor data in the tool instead of relying on GA, which has a time delay.
- Convert lacks built-in features useful to large-scale, enterprise-level teams, such as project management, result segmentation, robust user administration.
- The user interface is not clean or has limits when building complicated tests. This makes users dependent on customer service.
- There is no segmentation option during or after testing the results by device, new users, etc., within the results page.
Convert Pricing
There are three packages in Convert:
| Kickstart | Specialist Plans | Leader Plans |
| $699 per month | Starting $879 per month | Starting $1899 per month |
List of Integrations Supported by Convert
Convert can integrate with:
- Analytic Call Tracking
- BigCommerce
- Bitium
- CallFire
- CallRail
- CallTrackingMetrics
- Calltracks
- Contentsquare (Clicktale)
- Clicky
- Crazy Egg
- DialogTech
- Drupal
- Google Analytics
- Hotjar
- HubSpot Marketing Hub
- Instapage
- Joomla!
- Kissmetrics
- Convirza
- Mixpanel
- PrestaShop
- SDL Tridion
- Shopify
- Tumblr
- Twilio WebRTC, part of Twilio CPaaS
- unbounce
- Volusion
- WooCommerce
- WordPress
- nopCommerce
- Zapier
- Magento Commerce
- part of Adobe Commerce Cloud
- Avanser
- LanderApp
Convert Reviews
Here are some of the customer reviews about Convert, according to the G2 website:
Wrapping Things Up
As you would have learned by now, there isn’t a good or bad AB testing tool; it depends on your website, needs, current conversion rate, traffic, etc.
So, whenever you decide that you need one, you can go back to this blog.
