Website Personalization Engine: Testing the Accuracy of Tracking Ad Impressions

Note: Updated on 2023-02-05 with seven days of data. See the results and conclusion sections.

The website personalization tool we are working on had an algorithm change recently to track the impressions of the ads more accurately. This article will discuss the results of our digital marketing experiment to test the accuracy of impressions tracked. This will also demonstrate the ability of the personalization system to conduct digital marketing experiments.

We are currently using the website personalization tool to optimize the ads and slots that we have on our website. I will post more on such experiments in the coming days.

For this experiment, we selected a high-traffic page on our website. The page was analyzed using a heat map tool, Microsoft Clarity, to identify areas most visitors tend to scroll. Based on the analysis, three ad slots were created in these areas, and two different ad copies were created for testing.

Objective

This experiment's objective was to test the recent changes in tracking ad impressions. The message delivery system can now track the number of times an ad is visible to a user rather than simply counting the times the ad is loaded. This ensures that only impressions seen by the user are counted.

Method

Using Microsoft Clarity, we conducted a heatmap analysis on one of the popular pages on our website to identify areas where most visitors scrolled. Based on this analysis, we created three advertisement slots in these areas. We have taken the report on the impressions on these three slots before and after the code change.

Results

The experiment results showed that the new algorithm accurately tracks the impressions of the ads.

Data Before the Experiment

Here the impressions tracked for each slot are the same (equal to the page views tracked by GA). The total impressions are bloated as the impressions are counted for the ads users have not viewed.

Data After the Experiment

Here total page views tracked by google analytics for the reporting period was 6501. Out of 6501 page views,

• 4596 (70.70%) reached the first advt slot
• 3280 (50.45%) reached the second advt slot
• 1832 (28.18%) reached the third advt slot

These numbers match the analysis done on the heatmap.

Seven days data

In this period, total page views were 24311

• 14222 (58.50%) reached the first advt. slot with a CTR of 0.15%
• 8395 (34.53%) reached the second advt slot with a CTR of 0.04%
• 4167 (17.14%) reached the 3rd advt slot with a CTR of 0.1%

Conclusion

The results of this experiment demonstrate that our new algorithm that uses IntersectionObserver can accurately track ad impressions. Tools like Clarity can help you identify where to put advertisements on the website. However, experimenting with a system like our personalization engine can further measure efficiency and optimize the location and ad placements.

Also note that to get accurate results, you need to find pages with enough visitors and run the experiment for a sufficient number of days to get statistically accurate results. The example here shows 2-day data, which is insufficient to conduct a serious digital marketing experiment. This is enough for our purpose, as we thoroughly tested the system using google analytics debug mode. The video recording of testing this with the GA debugger is available in the article Tracking and Reporting Impressions and Clicks on the Personalized Messages 

UPDATE: We ran the experiment for seven days and updated the result in the section above.

1. The advertisements placed are not good enough to capture the user's attention. Most people scroll past the advertisement (the advt impressions are tracked only if it is visible to the user for 1 sec). We need to change the message and size of the ad.
2. The second slot CTR is very low comparing the other two slots. We need to move the second ad slot to a different position.

We will continue the experiment with the new message and position. Our target is to get a CTR of 2%

Let me know if you want an early preview or more details about this project.