Dave drew on his 14 years experience in CRO, sharing actionable insights on transitioning seamlessly from hypothesis to test execution. His focus was on the challenge of taking a “supposed winning hypothesis” and making sure that it turns into a valid and successful test, which he describes as “walking over a slippery floor carrying a priceless piece of pottery”. 

The post Taking a strong hypothesis and turning it into a successful test appeared first on Browser to Buyer.

For example, for one of our Lead Generation clients, Scrap Car Comparison, we recently ran an email A/B test which increased their collections by 25%.

In this post we discuss how emails can be improved, the measures to look at and how these measures indicate what to change.

Why are emails important?

Emails can often be overlooked – your team may not regularly see them, unless they are in the habit of spot checking them regularly. But they are an important form of communication with your customers at various points in their journey with your brand – for example:

• Welcome flows
• Order confirmations (often one of the most important yet overlooked emails)
• Basket abandonments
• Product review requests
• Back in stock notifications
• Nurture flows
• Cross selling other relevant products
• Follow-up emails – for instance, your next vaccination dose is due in x weeks or reorder reminders for frequently purchased products that users may not necessarily be subscribed to.

So where do we start?

Firstly we create a visual board of all our client’s email flows – Miro is a good tool to use for this. This can also help identify opportunities for new flows.

Secondly, we wish to understand the volume of each email type – as we want to make the most impact, and higher volumes mean A/B tests will reach significance sooner.

Next we make sure all emails are being tagged correctly – for instance, UTM parameters are set correctly to make sure the traffic is all allocated correctly.

Then we look at the Click Through Rate (CTR) and the Open Rate for each email. This helps to indicate which ones could be improved and what type of improvements are needed.

What does a Low Click Through Rate (CTR) on high volume emails indicate?

If we see a low CTR, then the first questions we consider are:

• Is messaging a problem?
• Are the users being given a reason to click?
• Can we use urgency or scarcity to get people to start to take action?
• Is the text clear, appropriately sized and does it render gracefully on mobiles?
• Do the users have a clear Call to Action (CTA)?
• Are the users being given a reason to come back to your site, or have you given it all away in the email?
• Does it feel like a human is addressing their concerns on an individual level – for example, using their name and conversational language? Or does it feel like the email was generated by a system? This can be a big turn off.

What does a Low Open Rate indicate?

If users aren’t opening the emails the most likely areas to investigate are:

• Is the subject line intriguing?
• How compelling is the teaser content?
• Does the email feel personalised or tailored to their specific interests?

How do we fix these issues?

Once we have identified which emails show the most potential for improvement and understood what is causing users not to interact with them, our expert consultants and designers work together to craft a carefully worded and designed email, with clear CTAs and appropriate teaser content.

These new emails are then tested against the existing emails, as an A/B test, allowing the impact of the change to be measured, before winners are rolled out to 100% of users.

If you’d like to talk to us more about how we could run similar A/B testing for your emails and website in general, please get in touch and one of our consultants will be happy to discuss this with you.

The post Emails – Why and how should you A/B test them? appeared first on Browser to Buyer.

It’s typical to have this kind of thing across VPNs – so we expected it be a slam dunk to test this across the site.  It wasn’t!  

We put it on desktop, we put it on mobile.  The traffic split was 50:50.  

What was amazing about this was that on desktop this DROPPED the conversion rate by about 20%, however on mobile it INCREASED the conversion rate by 33%.  

Changes having completely opposite effects on mobile and desktop is something we’ve also found previously for another VPN site.  And it’s not only SaaS sites – for  one of our eCommerce clients, David Austin Roses, many tests have given significant improvements on desktop but not done so well on mobile. 

The danger

By breaking down into desktop and mobile, we KNEW that adding the bar on mobile would increase the mobile conversion rate by 33%.  That’s huge.  

We also KNOW not to add this bar to the desktop site.  

Traditionally people look at the overall results of a test across all devices and only break them down by desktop or mobile when they think there’s a usability difference.  

So if one goes up and the other goes down, they’ll think the impact is flat.  And not make the change.  

In this case they would have not implemented the new change on any device and missed out on a 33% increase on mobile.  

Think back to losing or flat tests you’ve run on your site.  Did you look at the breakdown?  Are you sure you wouldn’t increase conversion by running these just on one type of device?

But why are the results sometimes different on mobile to desktop?

1. People are in very different buying positions when they’re on a desktop device.  

Some products and services are too complex or expensive for people to complete their purchases on mobile. For example, for one of our clients, who offers a B2B service, users might do some initial research on mobile, but most of the purchasing is done on desktop. Similarly for high-ticket B2C items, often users are reluctant to actually complete the purchase on mobile. For others, however, like VPNs there is a more immediate need and lower cost barrier so many purchases are made on mobile

2. Different devices are used by different audiences.  

Younger audiences are more likely to be using mobile than older audiences and so rather than thinking about desktop and mobile as different ways of viewing your site, instead think about them as ways to differentiate between your audiences.  

3. Time, screen space and distractions differ between devices. 

Typically on desktop users have more time to appreciate what’s on the site.  They can read, they can research.  On mobile you need to be really careful not to distract your users. They won’t read for so long and have a very limited screen size. You really need to identify what the key messages are (use a heatmapping tool like CrazyEgg to help with this) and make sure these are the ones users see and read. Mobile users are more likely to be dual-screening and have shorter attention spans, and the page needs to account for this.

Going forward

When you’re designing your A/B tests, make sure you carry out your research across mobile and desktop and break down your analytics by device so that you understand how these are used differently for your products.  

When you run your A/B tests, make sure you analyse the test results by device for every test.  This way you’ll get all of the upside….and none of the downside. 

The post Don’t miss your biggest wins <br>– why A/B test results need device-specific analysis appeared first on Browser to Buyer.

One of the biggest reasons for users not converting on a website is usually price, or at least the user’s perception of price. If a user doesn’t feel that the product or service gives them value for money they’ll look for alternatives or simply won’t convert. Using a price anchor can help to overcome this, by making the price appear less expensive.

What is price anchoring?

Anchoring is a cognitive bias – an unconscious reaction in the brain to the outside world. What that means is that your brain is influenced without you realising it, changing your perception of the information you see around you. There are hundreds of these cognitive biases, but one which is often involved in purchase decisions is anchoring.

The concept is quite simple – when given new information, people will start with a first approximation (their anchor) of what that means and will then adjust this based on additional information they receive.

For example – if you see a new computer priced at $800, that will be your initial anchor price of the ‘value’ of that computer. If you then do more research  you may find others with similar specifications priced at $600, $750 and $685. This is the additional information you are taking in, which you will use to evaluate whether you think that $800 is a fair price for the computer.

This seems quite straightforward, but the cognitive bias comes in when we discover that the original anchor has more influences us more than the additional information  That means the original $800 price tag will subconsciously have more importance to us, than the three others prices we found when evaluating the value of the computer.

This may seem strange, but it has been supported by many experiments. For example, Tversky and Kahneman asked two groups of people to work out a calculation in their heads in just five seconds:

• Half the people were asked to calculate:
  1 x 2 x 3 x 4 x 5 x 6 x 7 x 8
• And the other half:
  8 x 7 x 6 x 5 x 4 x 3 x 2 x 1

Of course five seconds wasn’t enough to calculate the exact answer, so the participants had to estimate after the first couple of calculations.

Those in the first group had a median estimate of 512 however those in the second group estimated much higher: 2,250. The lower guess in the first group was because they were anchored to a lower number at the start (1 x 2 = 3) than the second group (8 x 7 = 56).

The real answer was in fact 40,320.

Interestingly, even the suggestion of a number, even one that is clearly nowhere near actual answer has an effect. Two groups of students were asked a question:

• Did Gandhi die before or after age 9?
  or
• Did Gandhi die before or after age 140?

They were then asked to guess the age he actually died. The first group guessed significantly lower (average of 50) than the second group (average of 67) – they had been anchored by the lower number.

These and countless other experiments have shown the anchor effect is real, so how can we use this to increase conversion?

How should you use anchoring to influence your visitors?

There are two main important ways:

• If people don’t have an idea of what something costs, they are very open to being influenced about value (they have nothing to compare to). Price anchoring can be very strong here in influencing how your products price and value will be evaluated.
• When comparing different price options, people will be influenced by the first price they see or read. If you have three price options, presenting them with the lower price first will anchor people lower:

Let’s now look at six examples of how anchoring can be used to influence your visitors:

1. Use a high monthly price to make the annual cost appear lower

Runkeeper, the hugely successful fitness app, uses price anchoring to make the cost of their subscriptions seem much cheaper.

The upgrade screen shows the monthly price (£5.99) followed by the annual price of £23.99:

This is a great approach. On its own, £23.99 may seem expensive for a subscription to an app but price anchoring means that the person has already subconsciously looked at the monthly cost and worked out that the annual price is a really good deal in comparison (equivalent to less than 5 months subscription, or an annual saving of about £48).

Runkeeper’s visitors have no initial concept of how much a year’s subscription should cost, but the anchor means that they perceive it to be much higher than Runkeeper actually want to charge – making the annual subscription seem like a bargain.

2. Reverse anchoring to sell a subscription

Amazon use the opposite principle to upsell users to a Kindle Unlimited subscription.

In search results, when a book is available as part of the subscription, they list the Kindle Unlimited price as £0.00 before the actual price to buy the book.

Again, users often don’t have a perception of how much they expect the book to cost. This isn’t a particularly expensive book, but once the user has been anchored to £0.00, the £3.72 price seems expensive. That makes them more likely to consider the subscription – a far more lucrative option for Amazon than a single sale.

3. Show a sale price first to make the product seem cheaper

One of the most commonly used, and effective methods of price anchoring is to use a “was” price or sale price. GoDaddy do this very effectively, by listing a “regular” price of £10.99 for a domain, with a sale price of £0.99:

This appears to be a fantastic deal as in the user’s mind the value of the domain has been anchored to £10.99. It doesn’t work so well if you try it the other way round as iPage do:

Although the price and discount are similar to Godaddy’s, the user is already anchored to $1.99 for a month of hosting before they see the discount, negating much of the effect.

Was/sale/regular pricing is hugely effective, but to get the most value from it, put the higher price first to anchor your visitors to the higher value.

4. Use any large number to anchor the user, not just a price

The great thing about the anchoring bias is that you don’t just have to use a price to anchor the user. On Hotjar’s pricing page they show the number of visitors included in the package first:

This has already anchored the user’s mind to a high number (20,000) before they start looking at prices. Although the 20,000 and €89 price aren’t related, as the “Ghandi” experiment  showed, this doesn’t matter. Users will already be anchored to the high number, so the price will seem more reasonable.

Investment company Nutmeg also approach this well by using a fee calculator. It shows the value of the portfolio first (a very large number), followed by the monthly fees (a comparatively very small number):

The user is anchored to a large number first so the fees seem far smaller.

If you don’t have multiple prices or packages to use for anchoring, finding another number (perhaps a feature of your product or service) to anchor the user at a higher value, can help to make your price seem more reasonable.

5. Make your anchor price stand out so that it’s looked at first

The key in anchoring a user to a specific value is to make sure it’s looked at first. Site5 make their “hostPro+Turbo” package stand out so much that the price is the first thing the visitor sees on the page. As it has the highest price, both other packages seem very reasonable in comparison:

The important factor here is that this is a cognitive bias. It happens unconsciously, so think very careful about the first number or price the user is seeing. Even if you can’t put the large number first on the page, draw the user’s attention to it using other design and usability techniques.

Booking.com have a very effective technique where they use a “loading” indicator for a few seconds when the sale price is being loaded. This gives the user’s mind time to anchor on the original price for the hotel, and also draws attention to the lower price when it is loaded:

Using design and UX techniques can be very effective in drawing the visitor’s attention to your anchor price and making it more effective.

6. Order prices and packages from highest to lowest

The most common layout of pricing pages, particularly for subscription services, is to offer three or more of packages, ordered from lowest to highest in price. These pages often try to use all sorts of psychological techniques to upsell users, such as labelling packages as “best value” or “most popular”, however simple price anchoring can be even more effective.

Compare A/B Tasty’s pricing page:

To that from competitor Convert:

Convert’s page shows the enterprise price first – anchoring the user. Their $499/month Lite package then appears like a great deal. In contrast, A/B Tasty’s $249/month seems relatively expensive as there’s no high anchor to guide the user.

When to be careful

Price anchoring is a very effective technique, but doesn’t come without risks. Some of the things to be careful of are:

• Don’t put users off with a high price too early. If users know little about your product and see  a high price early on they may just leave without investigating further. Make sure you have users’ attention and interest before using anchoring.
• In retail and travel, be particularly careful about very cheap products showing in search results. For example when searching for iPhone on Amazon’s site, several very cheap accessories are shown first. This will subconsciously anchor the user to this lower price, making the actual iPhone seem more expensive than it actually is.

• Test to work out what is right for your visitors. Although effective, this doesn’t work for everyone, so make sure you A/B test your changes to find out the effect they have on your users.

Conclusion

Price anchoring can be hugely effective when used in the right way. By carefully evaluating your visitors’ behavour on site, you can find the best and most effective places to use this technique.

If you’d like to see what price anchoring and other persuasion technqiues could do for your site, why, not get in touch for a free discussion.

The post 6 ways to increase conversion rates and revenue using price anchoring appeared first on Browser to Buyer.

21 May 2021 – Lead Generation World UK (Virtual)

Conversion Rate Optmisation (CRO) and A/B testing brings the potential to significantly increase leads from your site without needing to increase traffic. Drawing on experience with over 70 companies around the world, from tiny startups to global brands, Dave will cover the fundamentals of getting started with CRO. By using a structured approach to research, idea generation, wireframing, design and testing, any site can significantly increase their conversion rate. Dave will share his tried and tested approach, some of the best tools and techniques and some of his most surprising and interesting tests.

The post Dave Gowans speaks at Lead Generation World UK – How to set up a successful CRO programme appeared first on Browser to Buyer.

What is Purchases on Google?

Purchases on Google is a new offering for merchants which will allow users to buy directly from a product search result. Instead of being linked to your site, visitors will see a “Buy on Google” option. This will take them to a simple product page (with your branding) before allowing them to checkout directly without visiting your site at all.

Currently open as a beta program, the Buy on Google buttons will only appear for Android users who are using Google Wallet – a small but growing proportion of the marketplace. Google are recommending this as a seamless shopping experience but from a conversion rate optimisation (CRO) perspective, will this really improve your conversion rate and what will the impact be on your sales?

There’s very little data available at the moment due to the closed beta. Google have released a few examples from their partner retailers, for example, according to Google, Deckers Brands have seen a 50% increase in the conversion rate of their mobile shopping ads and a 25% decrease in conversion and over the first year of the program, Staples have seen an improvement in ROAS. These are two carefully selected examples, however, so what’s the real impact on conversion for your site?

Conversion Rate Impact – The Positives

1. Seamless three click shopping experience

From a usability perspective, it’s a very compelling proposition. The user can go from search results to a completed order in just three clicks. The product page and checkout are as simple as possible and, as these users all have Google Wallet, payment and delivery details are already prefilled. You can’t really fault the ease for users to make purchases and this may just make the difference for customers, especially on mobile. Its still surprisingly time consuming to enter purchase details on a mobile device and users may be more likely to convert through an experience which almost completely removes this barrier for them.

2. Your products will stand out against the competition

As a beta tester or early adopter of Purchases on Google, you will have a huge advantage over other retailers that your products will stand out. The Buy with Google button will draw users’ attention to your listings and that attention could easily be converted into a click and purchase. Combining this with the ease of making a purchase through Google’s checkout, early movers are likely to see a lot more volume coming through this route.

3.  You get the benefit of Google’s reputation

For smaller retailers, trust and reputation are a huge challenge. When listed alongside big, brand name retailers, it is difficult to compete, where users often choose a familiar name, even if a competitor offers better prices or service. It can extensive conversion optimisation to demonstrate your reputation well on your site and this gives a quick shortcut to build trust. With Google offering purchase protection and lending its brand to the sale, there’s a high probability that cautious users would choose the product “sold by Google” over an independent seller.

Conversion Rate Impact – The Negatives

1. Your brand and offering is secondary to Google

Similarly to selling on Amazon, by getting the benefits from Google’s brand and reputation you give up your own. Although your logo will appear on the product page and Google will allow users to opt into your marketing, fundamentally users will see this as a transaction with Google, not your store. If they want to think about their purchase, they’ll return through Google (not direct) and any future purchases are likely to be made through Google. You may make more sales in the short term, but at the cost of a long term loyal customer.

2. You lose the opportunity to compete on other areas except price

Currently retailers have a huge opportunity to sell users on the benefits of a product or why they should buy from a particular store. Whether that’s offering a great delivery and returns process, a loyalty scheme, an expert and curated service or a great backstory and history, these are all lost. The impact of these shouldn’t be underestimated. One ecommerce retailer I worked with saw a 16% increase in conversions from highlighting their history and story to customers. Without these tools, price becomes the only comparison point, and results in squeezed margins and a race to the bottom.

3. Lost opportunity of upsells, cross-sells and multiple item orders

As the customer never hits your site, you have no opportunity to give them potentially lucrative upsells and cross-sells. The value of these have been proven by retailers around the world, for example Comscore data shows that 75% of people have added a product to their bag to reach a free shipping threshold. There are so many opportunities to upsell shoppers (with add-ons, more expensive products or volume orders), inspire them with similar items (capturing lost sales where the product wasn’t quite right) and to cross-sell them with other items in your catalogue. A retailer with a focus on CRO will make a significant proportion of their revenue through these routes and by using Purchases on Google you will be giving up this opportunity.

4. Loss of data and customer insights

Google will pass customer data back to you and allow you to opt them into your marketing, so the data from your sales is safe, but the real loss is the data and insight you can gain from people who don’t buy. This is some of the most valuable data for improving your conversion rate. By running onsite surveys, using session recordings and examining your analytics data you can build up a picture of why users don’t buy from you and make changes to fix that. If potential shoppers never reach your site, you have no opportunity to learn from them, resulting in lower conversion rates and fewer sales.

Should I use Purchases on Google?

It’s very early days but for heavy users of product listings, Purchases on Google is a very interesting development. Like any major change to your site or offering you should run a split test.

It appears that Purchases on Google gives more advantages for smaller retailers than larger ones as the benefits of Google’s brand and purchase protection will be stronger. This is especially true if your purchase funnel is particularly long and complex, or your own data indicates you have a major problem with users dropping out during the funnel. By passing this responsibility to Google, you’ll get a quick fix, however resolving your onsite conversion issues should be the priority.

If you do choose to test the program, it’s important to watch out for any tests being skewed by the early mover advantage. If you are one of the first into the program, your ads will stand out more among others and you may find this increases both clicks and conversions. Make sure you continue to test regularly as the program matures to ensure that your choice is still valid.

Google will promote Purchases on Google as a foregone conclusion for increasing sales, and there are many good arguments for why it may do so, however make sure you have your own justification before joining and continue to test regularly. This space is changing fast and you may find that the results of your early experiments don’t continue as more retailers come on board. Don’t forget the long term impact too – remember you’re building a customer relationship and selling your business and brand to users – not just making a quick sale.

The post Should I use Purchases on Google? The impact on your conversion rate, sales and revenue appeared first on Browser to Buyer.

The aim is to stop so-called “subscription traps”, where people who have registered for a free trial get charged for services they didn’t expect. This led to speculation on what exactly the government was planning, with major newspapers suggesting the changes “could include stopping companies from taking payment details when customers sign up for a free trial.”

Thousands of sites and businesses rely on subscriptions and free trials from Software as a Service (SaaS) to entertainment, shopping, food and health. Whether it’s a 30 day trial for Spotify or a month free with accounting software, they’re a fundamental part of the subscription business model and any change could have a significant impact on how these sites operate.

So what do these changes actually mean and will online subscription businesses have to change their approach?

What is a subscription trap?

Over 21% of people in the EU who have tried a free trial have reported having a problem with it, the majority in the cosmetics and healthcare industry according to a study on subscription traps.

Fundamentally a subscription trap is a free trial where a user’s credit card details are taken at the start of the trial and it is not clear that they’ll be charged for the service at the end of the trial.

Typically when credit card details are taken, a Continuous Payment Authority (CPA) is placed on your card. This means that the company can charge your card at any time in the future without your permission and for any amount. It’s a perfectly reasonable approach to charging for a subscription – you don’t want to enter your details every month or approve each payment, but it is open to abuse.

Subscription Traps usually happen when a site doesn’t make clear that signing up to a free trial means that the user is also signing up to a paid subscription, or automatically taking payments for a future subscription without warning. In one example, a user of food delivery service HelloFresh was unexpectedly charged £38 after their first “free trial” box. A customer of dietary service Weight Watchers took a free trial subscription and was told a specific end date, but was still charged for the next month and the end date was extended.

The two key problems come from:

·         Users not realising they are signing up for a subscription

·         Users being overconfident about remembering to cancel

A free trial gives users the opportunity to try out a service without having to pay but  for some subscription based businesses a  large proportion of revenue comes from the inertia and forgetfulness of people – they sign up then just forget to cancel the trial.

This doesn’t mean to say that all free trials are “subscription traps” but businesses must be very careful about how they approach them.

What are the government doing?

At this stage, there isn’t any guidance on what the government will decide. They have committed to publishing a green paper where there will be a consultation on subscription traps along with other changes to consumer laws. Although newspapers have sensationalised this as an “end to free trials” there’s really no indication that this will really happen and the government is most likely to follow the guidance of consumer bodies such as the Citizens Advice Bureau.

The Citizens’ Advice Bureau recommends:

​“Companies offering trial periods for subscription products or services should remind consumers that they will enter into a binding contract at the end of the trial period. This should be done several days before the end of the trial and should make the cost of the subscription clear. “

The European Union have also conducted their own investigation, concluding that there are good practices, illegal practices and a grey zone where the legality is not clear. Their focus is about enforcing existing legislation and educating consumers and businesses, rather than making major changes.

So, for now, the free trial looks to be safe but nothing will be clear until the government has published its proposals.

What does this mean for conversion rates?

For any subscription based business, the free trial is the biggest factor in the conversion process. There are two stages:

1. Convincing users to sign up for a trial
   This is all about convincing people that it’s worth their time to find out about the service. Offering a “no credit card” trial is by far the easiest way to do this, and will result in the highest visit to trial conversion rates but risks pushing a lot of unqualified traffic into trials. Worse, some businesses, like email testing software Litmus, suffer from users repeatedly signing up for free trials and never paying for the product
2. Convincing trial users to pay
   This is the second conversion step and one that is far easier if the user has already provided their billing details. Even if a business isn’t trying to ‘trick’ customers into subscribing beyond the trial, there is always going to be inertia and difficulty in getting people to take action to subscribe after their trial, especially if it involves entering payment details.

What should you do?

Legally, there is nothing you need to do at this stage and any legislation will take months, if not years, to come into force. Now is the time to start preparing, though. When Citrix switched from taking credit card details to offering a “no credit card” trial it took over a year to complete the switch and ensure conversion rates weren’t harmed.

The key now is to make sure you’re following practical advice and testing your trials extensively:

1. Make it clear that users will automatically be charged at the end of their free trial

SEO software ahrefs is a great example of this. They make it very clear that you will be charged at the end of the subscription and how much that will be:

You may be concerned that the price will put users off at this stage, so its key to A/B test your messaging and ensure that it doesn’t harm conversion rates. Ideally, you should also be doing price testing to find the perfect price point for your service.

2. Notify users clearly when they are reaching the end of their trial

The most likely recommendation from the government will be that users must be notified a few days before their trial ends and told what will happen next. This could be as simple as an email to the users, such as this one from Lucidchart. The key is to make sure you keep selling the benefits to the customer here. This is their opportunity to cancel, so make your offering as compelling as possible.

3. Consider not asking for a credit card at all

This is the higher risk but most customer focused approach. By offering a trial with no credit card, you need to make sure that your trial is so good that people really want to subscribe. Additionally your trial to subscription route must be fully optimised to make sure you don’t lose valuable traffic there. Accountancy software Xero don’t ask for a credit card, with the hope that once you’ve invested the time and effort in setting it up, you won’t want to cancel:

When considering a “no credit card” trial, you need to be very careful about messaging. “No credit card” is a huge selling point for the trial and you need to make sure you are featuring this message throughout the site.

What next?

Fundamentally, all subscription businesses need to be running conversion optimisation on both their free trials and subscription processes. The key is to find the right combination of credit card/no credit card trial with messaging that is compliant with the “subscription trap” recommendations. Conversion rate optimisation and A/B testing approaches is the best way to approach this as you will have the opportunity to increase your conversion rate as well as complying with new legislation.

The post Will the UK ban websites from taking credit card details for free trials? appeared first on Browser to Buyer.

These aren’t our clients so we don’t know the hypotheses and we can’t see the data, but for each test I’ve added a few comments on what we’ve seen.

Zipjet – Trustpilot Badge

We’ll kick off with London based laundry provider, Zipjet. They’re currently running a homepage test where they’re adding a Trustpilot badge to the top right of the page:

This is a great test and one that any business can try. Social proof is an extremely strong persuasion principle and this is an effective way to get it in front of as many visitors as possible. A similar test for a leading healthcare company increased sales by 13.6%. This could be even more effective if Zipjet had a higher Trustpilot rating, but I’m confident that this will be a winner for them. Do you agree? Let us know in the comments section below.

Follow up tests could include making the Trustpilot score even more prominent on the homepage, or possibly even including it in the headline.

Hidden Hearing – New Landing Page

Our next example comes from UK hearing aid company, Hidden Hearing. This test is a major redesign of their landing page:

 

There’s a lot going on here, but I’ll break it down into the main changes:

• Imagery – the variation features a clear image of the main target audience for Hidden Hearing. Images such as this help people to identify with the brand and confirm that it is right for them, however can risk alienating non-core customers. Hidden Hearing may have many younger customers who could be put off by this.
• Key Benefits – these have been reduced from five in a graphic to three main points. Having a large number of benefits is good for conveying the Unique Selling Points (USPs) of a brand, but can overwhelm users. The original small text could easily be missed, whereas focusing on three key points makes sure these are clearly visible.
• Headline and form header focuses on “local branch” – this can work well in increasing trust (brands with a real-world local presence are seen as more trustworthy), however people may be concerned that there isn’t a branch local to them.
• Focus on Experience – the expert testimonial has been changed and the qualifications and certifications Hidden Hearing have received have been promoted at the bottom of the page. Clearly experience and trust is important to visitors, and by promoting these, Hidden Hearing aim to reassure them.
• Testimonial Quote – adding the quote “The free hearing test … has made a significant difference to our lives” is a strong message as it focuses on outcomes. Users of products such as these are looking to solve a problem and this aims to address that. It could be even more effective with a more specific outcome “I can hear the birds when I wake up in the morning now and I never could before”. Additionally, giving a name and location of the person who said it would help to add credibility

This is a big change on a very significant page of the site and I would expect it is giving some very clear results. How would you have done this differently? Let us know below!

Wageday Advance – Button Colours

Unfortunately this is an example of one of those conversion optimisation myths that just won’t go away. For many years sites have been trying to get small increases in conversion rates through testing button colours and Wageday Advance is no exception:

‍

Wageday Advance might get a small result from this test, although it could be due to chance so I’d strongly recommend testing it to 99% confidence. Button colours are unlikely to strongly influence users. Ultimately there are three key things a button should do:

1. Be clear and obvious on the page (in this case the orange button is already)
2. Explain what the next action is (this does a good job)
3. Connect to the user’s end goal (it may be worth adding something to connect to the user’s goal of getting approved for a loan for example)

Button colour tests can be effective if the button isn’t clear on the page, but Wageday Advance could get much bigger increases from testing the copy of the button, or even better the surrounding content to ensure that users’ concerns are overcome.

Have you had success running button colour tests? Let us know in the comments.

Wufoo – Form On Mobile Landing Page

Wufoo is a web form provider owned by SurveyMonkey. They are currently running a test on mobile:

This is a nice, effective test from Wufoo with a clear goal. In Variation #1, the form is shown by default, making it clearer what the next step is, removing an interaction from the signup process (no need to click the button) and reducing the number of options available (just sign up for free without a pro option). This is risky, however, as there is no information at the top of the page to make it clear what the service is.

Often for SaaS products such as this, getting the user signed up and using the product is the key step, so this may work well, however I would want to look at scrolmaps for this page to understand exactly which features people are reading about in both versions.

Has this approach worked for you before? Let us know in the comments.

FreeAgent – Homepage

FreeAgent are running a very big test at the moment with three very different variations:

This is a great test as it uses three very different ways to convince the user:

• Control – This focuses on the things that users want to achieve (the text “VAT returns” is dynamic and changes regularly to show other use cases for the tool). This is a really strong approach to use if your tool or service has many different uses and has been one I’ve used successfully with a financial services client.
• Variation #1 – This is centred around social proof. The large statistic of “50,000 small businesses” is very compelling to show the scale of the service and adding in real tweets is an extremely effective way to make the social proof more effective. People identify more strongly with social proof when it’s presented as being from identifiable individuals and this should work well. Free Agent could make this stronger by using a specific figure for the number of small businesses, which could give more credibility.
• Variation #2 – This focuses on the individual user’s frustrations. A good approach to convincing someone they need your service is to focus on the pain you’re helping them to avoid.

This test really aims to understand what motivates FreeAgent’s users. It’s not just about adding something to the site. When they have the results they will know clearly which approach is best to use with their customers.

Have you run a test like this before? Let us know in the comments below.

Summary

This week we’ve seen five very different tests from across the web including some great examples of brands running tests to really understand their users, and some which are going to be less effective. Check back next week for five more real world test examples.

The post CRO In the Wild – 5 real world A/B tests we’ve seen this week appeared first on Browser to Buyer.

Having used Optimizely for over four years, running optimisation campaigns and hundreds of experiments for clients worldwide, I was keen to find out how the new platform stacks up. After six months using Optimizely X, I can now share some of the key differences, improvements and frustrations to help you to make the decision whether to make the switch.

So what’s changed in Optimizely X?

Almost everything! Optimizely X is a completely new platform that’s been built from the ground up. The principles are the same and you can still do everything you could in Optimizely Classic (the new name for the existing web testing tool), but the structure, approach and even naming of many common features have changed significantly.

The most exciting developments are areas such full stack testing, personalisation and recommendations. The changes have included a full rebuild of the web testing platform, however, so for typical Optimizely users, though, there’s a big change if you choose to switch. The only area where there hasn’t been significant change has been experiment results where Optimizely’s Stats Engine, already a key part of Optimizely Classic, is still used.

How do Optimizely X and Optimizely Classic compare?

Let’s take a look at some of the biggest changes and differences between the two platforms:

New approach to site and experiment setup with more upfront work required

One of the biggest changes in Optimizely X is the structure of experiments and the workflow for getting them live. This is primarily to fit with the new personalisation platform and to enable seamless integration between the two. What this means for users is a more structured approach to setting up and defining your site within the platform, with a bit more upfront work.

Traditionally Optimizely was focused entirely on experiments – you created an experiment, chose what pages and audience to target it to and defined your goals. That’s great for small scale testing but when you scale up to many experiments, it brings a lot of risk to the process. For example many experiments reuse the same targeting conditions over and over again e.g. example running an experiment on your Product Page involves targeting the same set of URLs every time. Every time you set up an experiment these need to be defined, though, which brings a risk of an error every time you set up a new experiment.

Optimizely X requires you to define the structure of your site first, then set up experiments (“Campaigns” if you’re using the new terminology) later. You use the platform to set up Pages (single pages or set of pages), which you can then use to target experiments. Each page can also have a set of Events (similar to goals in Optimizely Classic), used for tracking and targeting.

For an existing user of Optimizely, this seems like a time consuming process at first, but I was surprised at how much easier it is to get experiments live once you have your whole site defined within the platform. By going through the one-off task of setting up pages and events (including any custom coding which is needed), experiments are now trivial to set up, even for non-technical users. For anyone launching more than one or two experiments, the improvements in speed of creating tests and the reduction of risk from having pre-defined pages is clear.

New experiment workflow adding change management to reduce bugs

One of the biggest improvements in Optimizely X is the new workflow, which allows for draft changes to be made without publishing them live to all users. In Optimizely Classic it’s very simple, an experiment can either be live or paused. Although it is possible to preview a paused experiment, there’s no way to make changes to the code/layout without either stopping or cloning the experiment. Although it’s possible to use preview links set up QA cookies to fully test experiments before they go live, it’s a bit of a hack and means that its not always clear what the status of an experiment is on the dashboard.

Although changing a live experiment will invalidate your results, there are plenty of times when it is unavoidable, for example fixing bugs, making small changes to copy etc. due to changes outside your control or simply because most sites will serve winning experiments to 100% of traffic while they’re being implemented and changes are unavoidable.

In Optimizely X, the published version of an experiment is unaffected when you make changes. The changes are saved (and can be previewed using the new QA tool), but are kept in a staging environment until you choose to publish them. This far more closely matches a typical software development livecycle and again reduces risk of errors significantly.

Vastly improved visual editor with responsive support

When Optimizely was first launched, the editor was revolutionary, allowing point and click changes to sites without having to write code. Over time, however, it has become less able to cope with modern sites, with conversion optimisation agencies and advanced users preferring to make the changes in pure JavaScript.

The new editor brings full responsive support, allowing you to easily and quickly create responsive tests. The point and click functionality is fast, easy to use and integrates closely with the definition of events for tracking experiments, making many changes easier to do in the visual editor than by writing code. For anyone using the visual editor, this is a clear reason to upgrade.

‍

Technical changes with a new coding style

For users who develop their tests in pure JavaScript code, there are huge improvements to the native code editor. Optimizely have clearly focused on this aspect. The code editor now includes syntax highlighting, simple error checking and is far faster to load.

Optimizely Classic uses a complicated jQuery-like syntax to try to reduce flicker on pages, which, although effective, required developers to learn a new style of coding. For anyone who didn’t, pages tended to suffer from a flickering effect, harming the user experience and results.

Optimizely X does away with this, instead giving the option to run the code synchronously (as part of the Optimizely snippet) or asynchronously (as the page loads), with a few useful utility functions (for example one which waits for a specific element to appear on the page). Although it still requires a good developer to take advantage of this, it allows for a lot more flexibility when coding changes and is a welcome improvement to the platform.

Powerful QA tool to test experiments for bugs

QA is one of the most vital parts of any testing program. Any bugs or errors in tests can cause massive damage to a site and undo a lot of the good work in testing. Running QA for a test in Optimizely Classic has always been a little painful, requiring either query string parameters or special cookies, or the use of a limited QA tool which takes over the bottom of the page, reducing its effectiveness.

Optimizely X brings a new approach. You can still use your old methods, however the new QA tool is impressive:

Loading as a “button” on top of the site, it offers the ability to enter into different variations of multiple tests, preview different audiences and, usefully, provides a preview of the events and actions the tool is taking, very helpful in testing tracking and evaluating why experiments don’t run.

For more technical users, the full log is now shown in the console, and has been improved from Optimizely Classic, further helping to diagnose any issues with tests.

“Always on” tracking

One of the biggest changes to the platform, and a very welcome one, is “always-on” tracking. Previously, results were tied to specific experiments, so a goal only tracked when it was added to an experiment. If you realised later that you’d forgotten to track a key page or metric, adding it to the experiment would only give you data from the moment you’d added it. Frustrating, especially if you have seen a trend and want to get more data on it.

Optimizely X splits tracking from experiments. Your events (goals) are set up independently, so always track users. That means you can add an event to an experiment half way through and you’ll have data right back to the beginning of the experiment, overcoming one of my own biggest frustrations with the old platform.

Results themselves haven’t changed a lot with Optimizely’s stats engine powering the largely similar results page. Stats engine is a great improvement on traditional statistics, using a Bayesian approach which means that the results are statistically valid whenever you look at them. Unfortunately you still need some common sense when looking at results you will still get results declared when clearly there isn’t enough traffic:

New software which still has some teething issues

When considering the switch, it’s worth remembering this is still a new tool. Although more than six months old, I’ve still come across some bugs in the platform. Some are relatively minor (occasional cryptic error messages), others are bigger and more frustrating (experiments not live when the platform says that they should be). Although Optimizely’s focus will be on Optimizely X for future development, you’re still choosing a 6-month old tool, rather than one that has been tried and tested by thousands of companies over many years.

Personalisation and testing beyond the web

When considering whether to make the switch, a key consideration is the integration of Optimizely X with Optimizely’s personalisation platform along with other new tools such as Optimizely Full Stack and recommendations. The personalisation tool is simple to use and powerful, using the same point and click editor as the experiment platform to set up personalised experiences for customers. This, combined with the recommendations tool means Optimizely X is now a fully featured platform for far more than just web optimisation.

The introduction of Optimizely Full Stack is an exciting development. Offering SDK integration for many programming languages, it allows companies to test the backend systems of their site, not just JavaScript changes on pages. Whether that means testing features and functionality that couldn’t be tested on the frontend site or testing changes to apps and software, this is a great opportunity for mature optimisers to go beyond the web.

If you’re considering using these features in the future, it makes sense to switch to Optimizely X to have an integrated conversion optimisation platform.

PCI DSS compliance to allow testing on payment pages

Another big change which may affect some businesses is that Optmizely X is now fully PCI DSS compliant. Many businesses and payment providers will require compliance for any tools used on payment pages on sites. This has always been a challenge for Optimizely Classic, leaving some companies unable to use the tool or test their payment pages and funnels. With full compliance for Optimizely X, these companies should now be able to benefit from A/B testing throughout the funnel.

How easy is it to switch?

Unfortunately the biggest challenge for many companies will come with the switch between the two platforms. Although Optimizely have published some useful guides, the reality is that you need to completely move your testing from one platform to another. There’s no change to the snippet of code on the site, but everything else will need to be set up from scratch.

That means creating the Pages, Events and Audiences in Optimizely X, which for a large or complex site could be time consuming. Because the platforms are separate, none of your tests will be transferred from Optimizely Classic either. That means any tests which are live (or being served to 100%) will either need to be recreated (and restarted) in Optimizely X, or you will need to run both platforms alongside each other (known as a “phased rollout”). This takes away some of the pain, but is only really a short term solution and brings a complexity of managing tests across two platforms.

The biggest challenge when switching is likely to be the learning curve, especially for developers. For advanced users and experienced developers the changes to the platform, although welcome, will take some time to get used to. This especially true of the new coding styles, and I’d strongly recommend spending time getting familiar with building tests in the new platform before attempting to switch.

Ultimately, the switch will be harder for more advanced and active users as the change impacts the advanced features more than basic functionality.

Will Optimizely force you to change platforms?

At the moment, Optimizely is supporting both platforms, and will continue to do so for some time, however previous experience has shown that they are willing to push through changes (such as new packages and pricing) without a long consultation or transition period. The large volume of sites using Optimizely Classic should mean that there is support for the near future, but with personalisation as the company’s focus, an enforced switch to Optimizely X in the future is almost a certainty. Planning and making the switch on your own terms is a sensible approach.

Should I switch from Optimizely Classic to Optimizely X?

If you’re keen to test backend features which you can’t do with JavaScript, you’re interested in trying personalisation or aren’t currently actively testing, I suggest making the move straight away.

If you’re currently happy with Classic, you need to weigh up how important the differences outlined above are to you. There’s certainly a cost (both in time and effort) to switch platforms, particularly in the setup of Pages and Events and for developers to change to the new coding style, however there are significant improvements and the likelihood is high that Optimizely will force the switch sometime in the future.

Ultimately, I’d recommend starting to plan for the switch and looking into the impact on your site over the next few months. There are huge benefits from Optimizely X and, despite some pain in changing, you will ultimately be using a far better platform for your testing.

What about other platforms?

When making the switch, it might get you considering other A/B testing platforms. With a typical Optimizely subscription now costing between $2,000/month and $4,000/month the big question is whether it’s still worth the money or whether other tools offer better value. In a future post I’ll be comparing the leading optimisation tools to find which is the best choice for your site.

The post Optimizely X Review – Should you switch from Optimizely Classic? appeared first on Browser to Buyer.