Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dominik/ntp search bar experiment #3419

Open
wants to merge 5 commits into
base: dominik/ntp-search-bar
Choose a base branch
from

Conversation

ayoy
Copy link
Collaborator

@ayoy ayoy commented Oct 18, 2024

Task/Issue URL: https://app.asana.com/0/72649045549333/1208262570807667/f

Description:
This change adds pixels for tracking NTP Search Box experiment.

Steps to test this PR:

  1. Update DefaultNewTabPageSearchBoxExperimentCohortDecider.cohort to return .control
  2. Run the app and enable internal user state, then close and reopen window.
  3. Verify that you're not seeing the search bar and the logs say "User assigned to cohort control"
  4. Update the code to return .experiment cohort.
  5. Clean app data and repeat step 2.
  6. Verify that you're seeing the search bar and the logs say "User assigned to cohort ntp_search_box"

Verifying experiment pixel:

  1. Filter Xcode console logs by m_mac_initial-search.
  2. Search for something from NTP search bar, verify that m_mac_initial-search-day-1 is sent with a count = 1 and from = ntp_search_bar
  3. Search for something from NTP address bar, verify that m_mac_initial-search-day-1 is sent with a count = 2 and from = ntp_address_bar
  4. Search for something from another non-NTP tab, verify that m_mac_initial-search-day-1 is sent with a count = 3 and from = address_bar
  5. Repeat searches until you reach count = 10
  6. Search again and verify that no pixel is sent.
  7. Set the date in the operating system to +1 day.
  8. Do a search, verify that that m_mac_initial-search-day-2 is sent with a count = 1 and appropriate from parameter.
  9. Before continuing with the last step, complete Verifying autocomplete pixel parameters now.
  10. Set the date to +7 days.
  11. Do a search, verify that no pixel is sent, and autocomplete pixels don't have extra parameters.

Verifying autocomplete pixel parameters:

  1. Type a phrase in the address bar, NTP search box or NTP address bar.
  2. Select an autocomplete suggestion (phrase, website, bookmark, favorite or history item)
  3. Verify that an appropriate autocomplete pixel is sent (m_mac_autocomplete_*) with the correct from parameter, and the experiment cohort.

Definition of Done:


Internal references:

Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation

Copy link

Warnings
⚠️ PR has more than 500 lines of code changing. Consider splitting into smaller PRs if possible.

Generated by 🚫 dangerJS against d71795c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants