Running A/B Tests for In-App Messages

A/B testing for In-App messages requires special consideration due to how they are delivered to mobile devices. Unlike email or push notifications, In-App messages with trigger rules are cached locally on the device only when the user meets predefined targeting conditions.

This caching behavior means audience segmentation must be resolved before the trigger event occurs. If segmentation is determined too late, the message may not be cached and won’t appear as expected.

Pre-Segmentation Settings

Build an automation workflow that randomly tags users as A or B.

1. Create an additional contact field to store the group’s value.
2. Build the workflow:
   • Add the **Split ** block
   • Connect each branch to the **Update custom fields ** blocks
   • In the Update custom fields blocks’ settings, specify the created field (e.g., A/B test) and values to update them (e.g., A/B)

You can trigger this workflow using the ApplicationInstalled, SubscriptionStarted, or another early-stage event. The matching In-App message will be cached once the group is assigned and the user meets the segment condition.

Triggering Settings

1. Create target dynamic segments and specify test groups in their conditions.

2. Create two messages:
   • Variant A → targeted at users in group A
   • Variant B → targeted at users in group B

Each message will be cached only for users who meet the segment condition in advance.

📘

Check out the guide on managing In-App campaigns to learn more about setting up triggering rules for various business goals