Understanding how your landing page experiments perform is essential to optimizing conversion rates and making data-driven marketing decisions. Instapage provides two dedicated modes for running tests, Manual Experiments and AI Experiments, alongside standard Page Analytics.
This guide breaks down how data is tracked across Manual and AI Experiments and clarifies why numbers in Experiment Analytics may differ from your general Page Analytics dashboard.
Manual Experiment Analytics
Manual Experiments (traditional A/B or multivariate tests) allow you to test specific page variations against a control version using a fixed, static traffic split, as we can see here: https://d.pr/NcZjar.
Key Features & Metrics
Traffic Split: You manually define the percentage of traffic routed to each variation (e.g., 50/50, 30/30/40).
Unique vs All Metrics: Manual Experiment Analytics can filter based on Unique Visitors, Unique Conversions, and Conversion Rate for each variation.
Device Filtering: Filter results by All, Desktop, or Mobile using the top-right device selector.
Performance Over Time: Interactive line charts display daily traffic distribution and conversion trends across all live variations.
Experiment Statuses & Controls
Running vs. Ended: View active experiments or filter past tests by selecting the category menu in the top-right corner. You can also see draft and archived experiments.
Editing Safeguards: If you edit a variation while an experiment is active, a system warning will appear on the experiment analytics graph: "Experiment has been updated. The results may be affected." To keep test results reliable, avoid editing variations while an experiment is running.
Selecting a Winner: When ending a test, you choose a winning variation. That variation automatically takes over 100% of future traffic on the live URL.
AI Experiment Analytics
AI Experiments use artificial intelligence to power dynamic traffic allocation. Rather than maintaining a fixed split, the AI system continually shifts incoming traffic toward higher-performing variations to maximize conversions while the test is active, as we can see here: https://d.pr/F2fXDR.
How Data Evolves
-
Warmup Phase: The test starts with an equal traffic split. During this phase, variations display a status of WARMUP.
Warmup Criteria: The experiment must run for a minimum of 5 days, with at least 250 visitors per variation and at least 1 conversion per variation.
Burn-In Phase & Allocation: Once a minimum visit threshold is met (50 visits on average per variation), the AI algorithm incrementally reallocates traffic (from max 60% up to 90% and higher to the leader) based on conversion probability.
-
Variation Statuses:
LEADER: The top-performing variation based on visitor count and conversion rate.
CONTESTANT: The strongest competing variations actively participating in the experiment.
TIE: Variations that share equivalent conversion performance scores.
AI Experiment Data Views
In the AI Experiment dashboard, you can monitor:
Real-time metrics per variation: Visitors, Conversions, Conversion Rate, and current AI Traffic Split (%).
Traffic Split over time visual graph showing dynamic traffic shifts.
Conversion Rate over time visual graph illustrating performance trends.
Termination Reason: When the AI determines a clear winner (or if performance criteria indicate no clear winner will emerge), the reason is displayed under the Experiment Info section and on the variation card.
Differences Between Page Analytics and Experiment Analytics
It is common to notice minor discrepancies when comparing the numbers on your Page Analytics tab with the numbers inside Experiment Analytics. These differences are expected due to variations in time tracking, visitor filtering, and ad-blocking technologies.
1. Unique Visitors vs. Total Visitors
Page Analytics: Tracks Total Visits by default (though Unique Visitors can be toggled via filters).
Experiment Analytics: Shows Unique Visitors and Unique Conversions per variation to provide statistically accurate test results.
2. Experiment Start Time vs. Full-Day Tracking
Page Analytics: Displays data starting from the very beginning of the selected calendar day (00:00 UTC).
Experiment Analytics: Tracks data strictly from the exact minute the experiment was started or launched.
Example: If you launch an experiment on March 27th at 14:00 UTC, Page Analytics will include traffic to the page from 00:00 to 14:00 UTC (before the test went live), whereas Experiment Analytics will only count visitors who arrived after 14:00 UTC.
3. Timezone Discrepancies
Page Analytics: Always operates strictly on UTC (+0) timezone.
Experiment Analytics: Displays data according to your local browser's timezone.
Impact: A single visit occurring late in the evening might fall into two different calendar days depending on whether you are viewing the Page Analytics tab (UTC) or the Experiment Analytics tab (Local Time).
4. Ad Blockers and Privacy Extensions
Visitors using aggressive ad blockers, privacy extensions, or browser restrictions (like Safari's ITP) may block test scripts or experiment tracking cookies.
These users may see a variation of your page and be counted in general server/page analytics, but they might be excluded from experiment analytics if the experiment tracking script is blocked from executing.