Monitoring recommendation Premium plan

General information on the recommendations can be found on the page Personalize:

• The general report shows how the revenue changes from the moment the first recommendation was launched to the current day or to the day selected in the Period to field. These are summarized data for the entire list of active recommendations.
• The recommendations list includes draft, testing, and active recommendations. Each row is a recommendation for a specific application.

Recommendation report

After you run the recommendation, you will have access to a separate report for each application. To view the report, click on the app name or press the button .

The report includes the following data:

• Graph (1) shows the revenue from recommendations and the control group. The Y axis shows the key metric selected when creating the recommendation. On the X axis is the period from the recommendation launch to the current day or the day selected in the Period to field.
• Blocks with the results of recommendations (2): status, time required to achieve a statistically significant result, extra revenue, result, split, number of users.
• Blocks confirming the significance of the obtained results (3): confidence, power, p-value, expected result.

• Recommendation parameters (4): the key metric to evaluate the result, offer placements, selected user segment and type of recommended object.
• Offer list (5): one offer for the control group (A) and several offers for the recommendation group (B1, B2).

Recommendation statuses

The names of statuses correspond to the stages of recommendation formation: Draft → Testing → Learning → Significant result (or Failed) → Permanent result.

Each step has a different set of actions available:

Draft

Draft is an inactive recommendation that has not been finalized. The draft is saved automatically when you go to the second step of recommendation creation.

You can add a draft by clicking the button or delete it by clicking the button .

Testing

Testing is the status of an inactive recommendation at the testing stage. The status is assigned automatically after the recommendation is created and changes after it is run.

The testing page will be available for all subsequent statuses except for recommendations in the archive. For more details, see Testing and running recommendation.

Learning

Learning is the status of an active recommendation. Users of your app are already receiving offers on the recommendation, but it takes time to evaluate the result.

Learning completion time is the time until the statistically significant result is reached, which is displayed on the recommendation view page in the Statistical significance block.

Significant result

Significant result is the status of an active recommendation, the revenue from which can be increased. The status confirms that users in the recommendation group generate more revenue than the control group. You can change the split to increase revenue.

Changing the split

You can change a split in an active recommendation once. If you do not have permission to change a split, you can contact the account owner or the MyTracker team.

1. On the recommendation view page, click Change split or the button in the Split block.
2. In the opened window, use the slider to increase the share of the recommendation group to show recommended offers to more users and increase revenue. The control group is needed to evaluate the result.
3. Click ОК.

The recommendation status will change to Permanent result.

Permanent result

Permanent result is the status of an active recommendation that generates permanent income. Split control and recommendation groups are not available for editing.

Failed

Failed is the status of an active recommendation, the result of which is not statistically significant. Users continue to receive personalized recommendations, but due to the small amount of data, it is impossible to evaluate the confidence of the results.

You can stop the recommendation and create a duplicate based on a larger segment using the Stop and duplicate button.

Archived

The status Archived means recommendation is stopped and moved to the archive. Users no longer receive offers for this recommendation. You can duplicate the recommendation or restore it from the archive; other actions are not available.

To view the archive, enable the Show archived option in the recommendations list.