Travel & Hospitality
A/B test your product
Design, run, and analyze an A/B test to determine what marketing tools are generating the most ROI.
1.Design your test
First, you need to define your hypothesis. Example: If we add a live chat to our website, the subscription rate will be higher than a version of the site without a chat.
Next, define key properties about the experiment:
- Placing of the A/B test variable (e.g. chat widget). Show it on all pages, homepage or a specific landing page.
- Audience targeting. Define who is eligible to be in the experiment – first time or returning visitors.
- Success metric. If you’re an eCommerce company, a “conversion” might be when something is purchased from your online store. If you’re a B2B SaaS company, a conversion might be defined as someone subscribing for the product.
2. Set up Intempt Web Tracker
Install a web tracker on your website to start capturing visitor interactions.
3. Create events for A/B test
Set custom events to track how users are interacting with different A/B test variables. Let‘s use an example of the chat widget. We have 2 landing pages: one is with a chat widget another is without the chat widget. We will need to create events to track clicks on „Subscribe now“ (both pages) and clicks on the chat widget.
5. Run the test for 1-4 weeks.
Gather as many events as possible to make an informed decision. Minimum recommended sample size for 95% statistical significance and the minimum detectable effect of 20% is 6900.
6. Analyze data
Compare the A/B test results in the event graph page. Select event “Viewed page A” (without chat widget) as a base event and compare it against the success metric “A test subscription”. Note the subscription numbers.
Next, choose the other test event “Viewed page B” (with chat widget) as a base event and compare it with “Clicked on a chat widget” and a success metric “B test subscription”. Analyze which A or B test pages perform better and if the chat widget has improved the success metric performance.