Did you know that 75% of all conversion optimization work is (or has to be) research?
If you have enough traffic/conversions on your website, then A/B testing must be part of your research process.
The good news is there are dozens of A/B testing tools out there. The bad news is not all of them are reliable. Some are better than others in terms of data accuracy.
That’s why I have created this series of blog posts to talk about different A/B testing tools.
In this blog, we are going to talk about VWO vs. Optimizely.
Let’s get started…
What is VWO?
VWO (Visual Website Optimizer) is an A/B testing and conversion optimization platform.
It helps companies increase their conversions by assessing user behavior and web analytics.
VWO A/B Testing allows users to create and execute:
- A/B tests,
- Multivariate tests,
- Split tests.
VWO tool works best for Marketers, UX/UI professionals, Website teams, and Developers.
Like other AB testing tools, VWO has its advantages and disadvantages.
VWO Advantages
- It is easy to make no-code changes in existing pages to test variables like a copy, color, and button placement.
This will empower the users to run tests by themselves and validate hypotheses without technical help.
It also makes the learning cycles faster and enables them to implement changes smartly.
- You can run multiple types of tests on different URLs or via Javascript.
- It has many functions that allow you to easily test small changes like headlines, images, copy without needing developers to do so.
- VWO’s interface is laid out in a user-friendly format. It makes it great for UX teams to onboard stakeholders outside of the research department.
- The visual editor helps make quick changes. It cuts the development time in half and enables you to move quickly with your experimentation journey.
- VWO support team is always available via chat in the web app, and they’re very responsive (within minutes).
- Surveys, heatmaps, and session recordings are built-in.
- VWO has a feature to “check once” or “check continuously” to see if a user matches the test targeting criteria.
- The tool has a user-friendly UI that helps you navigate through all the experiences quickly.
- Within the report tab, you can adjust the time and recalculate the probability based on the duration you selected, which is a great bonus, especially when you have to make some changes to only some metrics in the middle of the test running.
VWO Disadvantages
As for the disadvantages:
- Some tests will inevitably require advanced implementation through the code editor rather than the visual editor.
- If you pause a campaign or test, the UI creates a prominent red warning that indicates it has been paused. This notification is red with a large exclamation point that makes it appear as if something is wrong with the test.
This notification is by design according to VWO cannot be dismissed. This creates some confusion when presenting or training new users.
- The platform tends to consume a large visitor allotment to find a result.
- It’s not easy to run a test on multiple pages at once. For instance, to test something on 10 product pages at once.
- The observation collection sample rate is a little low (for the price) – it takes a bit longer to collect a meaningful amount of data.
- Regarding AB test analysis, instead of giving you a specific number on the lift, it gives you a range and calculates “likeliness to beat control.”
This is due to the fact they use Bayesian statistics and not Frequentist. Having a specific numbered lift, like X% better/worse, would be extremely helpful in communicating the effect a test has to clients.
- There is no easy way to see how long a test has run. You must look at the date range on a different view in analysis and then count the days.
- You cannot compare conversion rates over time.
It gives you a line chart of conversions per variation over time. So naturally, the line chart keeps growing because conversions keep coming in.
VWO Pricing
As for the pricing, VWO starts from $99 to $467 per month for the web-based testing range.
They offer a FullStack plan for $1,999 per month.
Also, they offer a 30-day free trial. However, they don’t have a free version.
List of Integrations Supported by VWO
VWO can integrate with:
- WordPress,
- Drupal,
- BigCommerce,
- Mixpanel,
- Shopify,
- Google Analytics,
- Magento Commerce Cloud (formerly Magento),
- Joomla,
- Clicky,
- Kissmetrics.
VWO Reviews
Here are some of the customer reviews about VWO, according to the Trust Radius website:
What is Optimizely?
Optimizely is an American company based in New York City that provides A/B testing, multivariate testing, website personalization, and feature toggle capabilities.
It was founded in 2010 by ex-Google employees Dan Siroker and Pete Koomen.
Currently, Optimizely has been owned by Episerver since October 2020.
Optimizely Advantages
Just like VWO, Optimizely has many advantages like:
- Optimizely’s user interface and WYSIWYG editor are easy to use and don’t require coding experience but allow you to use JavaScript and CSS if necessary.
One of the great features is extensions, where a Developer can create a prototype template for a particular feature, for example, Popup/Lightbox.
Moreover, any non-dev person or a marketing team member can quickly utilize that by feeding the required information, for example, Popup/Lightbox text content.
- Test results are clear and easy to read.
They provide great graphs and tables to help you see how different variations performed, how likely a variation is to beat the baseline, and how well that stacks up against other variations.
Even with multivariate tests, the results are clear and concise.
- Traffic allocation options are convenient and easily split traffic into different variations.
- Optimizely integration is super easy. With one line of JavaScript, you can install Optimizely on your website and start testing. Minor configurations may be required within the tool based on product-specific requirements but never take much time.
- Optimizely’s results dashboard is very beneficial. It will allow you to get insights into the running of tests, like time to the significance and a view of metric changes over time.
- Their Stats Engine allows marketers to declare a winner sooner (with smaller required sample size) than traditional statistics would allow.
- It has a robust results engine. It automatically creates segments based on URL parameters and other standard criteria (new vs. returning) in addition to the predefined criteria you set up.
- The support of the Optimizely implementation team and customer care is beneficial.
They provide knowledge of how to use the tool and how to build a culture around testing and experimentation within your company.
- The drag and drop/click and pull functionality present throughout the tool allow for seamless changes and updates.
Optimizely Disadvantages
As for the disadvantages of this tool:
- Sometimes, the linting feature for JavaScript and CSS takes little time (15 – 30 seconds) to load large code and switch between CSS to JavaScript editor and vice-versa.
- It might slow down the rendering of a webpage because it’s a third-party script. If you set it to render asynchronously or after the page is rendered, you might see flickering.
- Their cost structure might be considered a bit expensive for small businesses.
- URL targeting does not have a mass upload option for lists of unrelated URLs. It does have substring and regular expression matches that cover most other needs.
- When clicking on an element with a ton of nested HTML, the HTML editor pane can often bug out and only show the first 10 to 20 lines of code before going completely blank.
- Reporting/exporting is a challenge if you want to report on data across multiple campaigns.
- Integration with Google Analytics can be problematic.
Optimizely Pricing
Optimizely offers Standard, Business, and Enterprise plans.
Their prices start from $50,000 per year.
List of Integrations Supported by Optimizely
Optimizely can integrate with:
- Google Analytics,
- WordPress,
- Jira,
- New Relic,
- Mixpanel,
- Segment,
- Marketo,
- Hotjar,
- Contentful,
- SessionCam,
- AdBasis,
- Oracle DMP (Bluekai).
Optimizely Reviews
Here are some of the customer reviews about Optimizely, according to the Trust Radius website:
Wrapping Things Up
And, voilà!
That is it for this series, folks.
I hope it was beneficial and informative for you, as I intended it to be.
