Implement NTP Custom homepage ("super referral") #4866

simonhong · 2020-03-06T11:00:35Z

NTPBackgroundImagesServicemanages background provider(SI or SR).
By default, it registers SI component and SR component if it's super referral install.

It's global service because SI or SR component is browser-wide component and it manages their data itself. Which data should be used for webui or android NTP will be determined by ViewCounterService. ViewCounterService will provide proper data (SI or SR) to client (like webui) by checking current profile's preference.

NTPBackgroundImagesSource handles bg data request. It can handle SI and SR data simultaneously.
SI data's url starts with chrome://branded-wallpaper/sponsored-images/... and SR data is something like chrome://branded-wallpaper/super-referral/logo.png.
I think we can expand this in the future. For example, chrome://branded-wallpaper/default/wallpaper.jpg for default background.

To make SR data available quickly, SR data is cached. With this caching, client can get SR data
before component registration is ready. It's important because SR's NTP always should display its image.

In this PR, kNewTabPageSuperReferralThemesOption pref is added.
Client should use it as a switch for turn it on/off SR component in runtime in super referral installs.
If it's turned off, SI or default bg will be displayed. This can be changed in settings page.

Regarding to flags, existing New Tab Page Branded Wallpapers flag will act as a top level switch.
If it is off, SI and SR are all disabled.
If New Tab Page Branded Wallpapers feature switch is on, SI component is enabled by default.
New Enable Brave Custom Homepage is introduced for SR component. If this is on, SR component will be installed when current install comes from super referrer.

To test locally, we can pass different local test data by using ntp-sponsored-images-data-path and ntp-super-referral-data-path. Also can use demo components. It becomes available when demo flag is on.

Settings page with Super referral

Settings page with non Super referral

NTP Pages with Super referral

Submitter Checklist:

Test Plan:

Simply test with demo data => SR component test is not supported by Demo flag.
1. Launch browser with fresh profile
2. Turn on New Tab Page Demo Branded Wallpaper in brave://flags and relaunch
3. Open brave://components and check SI and SR component is registered (NTP Sponsored Images (DEMO) and NTP Super Referral (Technikke))
4. Open NTP and check SR is loaded. If top sites are visible, it means SR component is loaded
Note: With demo data, theme settings are not properly updated. To see the themes option change, please follow below step.

Check fresh install with super referrer

Create empty profile dir
Create promoCode text file in root profile dir and that file has TECHNIK string
Launch Browser and check NTP Super Referrer (Technikke) component is registered in brave://components
Load new tab and topsites are visible for every NTP loading
Open settings page and check themes option is changed in appearance settings
Click theme option and check new themes sub page.
Check super referral bg is not displayed after selecting Brave default images option.

Check fresh install with non referrer

Launch browser with clean profile
Check NTP SI component is only registered

Two scenarios that check this change doesn't affect existing users

Scenario 1 (User has default code or non super referrer code)

Create user dir with other builds
Launch browser with this PR with created user data at step 1
Check NTP Sponsored Images component is only registered

Scenario 2 (User has empty code because of 90 days passed)

Copy old user data dir that used more than 90 days
Start browser with that copied user dir
Check NTP Sponsored Images component is only registered

Reviewer Checklist:

New files have MPL-2.0 license header.
Request a security/privacy review as needed.
Adequate test coverage exists to prevent regressions
Verify test plan is specified in PR before merging to source

After-merge Checklist:

The associated issue milestone is set to the smallest version that the
changes has landed on.
All relevant documentation has been updated.

simonhong · 2020-03-12T00:08:53Z

Note: This is not yet ready to review. This PR works well for fresh install but not work properly for existing users. Implementation should be slightly modified after fixing brave/brave-browser#8603

simonhong · 2020-03-12T12:05:47Z

Ready to review.

simonhong · 2020-03-17T03:51:07Z

~~NOTE: Need to rebase after #4807 is merged.~~

browser/BUILD.gn

components/ntp_background_images/browser/ntp_background_images_source_unittest.cc

It should ignore for in-effective images data when it gets update.

W/o this, we could display SI data for super referral.

To do this, we should check existing install's promo code status properly. at launching time. If referal code is empty and referral service's is not initialized state, we should wait to get promo code. If referal service has already initialized state at launch time, we can set this is not SR install. SR is only effective for fresh install.

NTP displays SR data always. So, caching it and use it instead of using data from component folder. If not, we have to wait to get SR data until component is ready.

So far, current code only can recover fetching mapping table failure. but browser can be crashed while waiting component fetching after mapping table is downloaded. So, we should run recover logic if browser doesn't finish initial component downloding due to crash.

Android needs super referrals code during the campaign to generate QR code. Backend will clear it when campaign ends.

When registering demo, it skips some steps that sr component registration does. It can ruins some preferences. Instead of this, we can demo SR component easily by inserting promoCode file in user data dir.

ViewCounterService will provide SR data even if NTPBGImages pref is false.

Regardless of background image option, NTP will show SR images if user choose SR theme in settings.

Use same manifest filename and determine it's SI or SR by its contents. And delete utils. APIs that in utils should be in service impl file.

simonhong · 2020-04-17T09:12:12Z

Filed f/u issue - brave/brave-browser#9289

bsclifton · 2020-04-20T17:53:48Z

if/when uplifted, we'll need to grab this one too:
#5290

Implement NTP Custom homepage ("super referral")

GeetaSarvadnya · 2020-04-22T17:23:39Z

Verification passed on

Brave	1.10.3 Chromium: 81.0.4044.113 (Official Build) nightly (64-bit)
Revision	e3225dafb0475864a1812a374d73a92e391635ac-refs/branch-heads/4044@{#936}
OS	Windows 10 OS Version 1803 (Build 17134.1006)

Check fresh install with super referrer

Verified NTP Super Referrer (Technikke) component registered in brave://components
Verified topsites are visible for every NTP loading
Verified that themes option is changed in appearance settings
Verified Background images and Sponsored images settings are hidden in Dashboard settings in NTP
Verified that Technikke theme new images are displayed in NTP, verified all the 3 sets of images
Verified that super referral bg is not displayed after selecting Brave default images option in Themes
Encountered Extract correct resolver for ens domains (uplift to 1.27.x) #9353 and Fixes unexpected error messages in logs - follow up to #16744 #9354

Check fresh install with non-referrer

Verified that NTP SI component is only registered in brave://components when non-referrer is used

Two scenarios that check this change doesn't affect existing users
Scenario 1 (User has default code or non-super referrer code)

Created 1.7.x profile and moved profile to 1.10.x build which has referrer code, verified that NTP Sponsored Images component is only registered

Scenario 2 (User has empty code because of 90 days passed)

Verified NTP Sponsored Images component is only registered

Verification PASSED on macOS 10.15.3 x64 using the following build:

Brave | 1.10.3 Chromium: 81.0.4044.113 (Official Build) nightly (64-bit)
-- | --
Revision | e3225dafb0475864a1812a374d73a92e391635ac-refs/branch-heads/4044@{#936}
OS | macOS Version 10.15.3 (Build 19D76)

Check fresh install with super referrer: - PASSED

ensured that TECHNIK images are being displayed whenever a NTP is opened while brave://settings/themes is set as Technikke
ensured that clicking on the TECHNIK logo opens https://brave.com/?from-super-referreer-demo (checked on the three different images)
ensured that the TECHNIK images are NOT being used in PB/Tor/Guest windows
ensured that you can unpin/remove the pre-set top tiles from the NTP
ensured that promoCode under brave://local-state appears as TECHNIK

"initialization": true,
"promo_code": "TECHNIK"

Check fresh install with non referrer:

Ensured that installing Brave without a super referral promoCode uses the current NTP SI component and loads the correct images.

Scenario 1 (User has default code or non super referrer code)

used my personal profile which is using BRV001 (default Brave) as the promoCode and ensured that NTP Sponsored Images (CA) - Version: 1.0.34 was being used.
ensured that the current NTP SI were being downloaded/used.

"promo_code": "BRV001",
"timestamp": "13227605020492579"

Scenario 2 (User has empty code because of 90 days passed)

Used RED194 as the promoCode and ensured it's appearing under brave://local-state:

"initialization": true,
"promo_code": "RED194"

Moved the machines date ~90 days ahead and ensured that NTP Sponsored Images (CA) - Version: 1.0.34 was being used and promoCode was removed from brave://local-state. However, received the following error in the terminal which is expected as 30 days haven't technically passed on the server side, hence not receiving the +1 for a RED194 referral.

[82450:775:0901/172939.144138:ERROR:brave_referrals_service.cc(366)] Referral is not ready, please wait at least 30 days

simonhong added this to the 1.7.x - Nightly milestone Mar 6, 2020

simonhong self-assigned this Mar 6, 2020

simonhong force-pushed the ntp_super_referral_images branch 4 times, most recently from 7fe98a6 to 7c6dfeb Compare March 6, 2020 16:27

simonhong marked this pull request as ready for review March 6, 2020 23:28

simonhong requested review from bridiver and petemill as code owners March 6, 2020 23:28

simonhong force-pushed the ntp_super_referral_images branch 2 times, most recently from dadc4ef to ed7bcf0 Compare March 7, 2020 02:23

simonhong added the CI/skip Do not run CI builds (except noplatform) label Mar 9, 2020

anthonypkeane mentioned this pull request Mar 10, 2020

Desktop :: Implement Custom Homepage brave/brave-browser#8218

Closed

6 tasks

simonhong modified the milestones: 1.7.x - Beta, 1.8.x - Nightly Mar 10, 2020

simonhong force-pushed the ntp_super_referral_images branch from f25f3c5 to 1826181 Compare March 11, 2020 02:21

simonhong requested a review from bsclifton March 12, 2020 06:21

simonhong force-pushed the ntp_super_referral_images branch from bd0743a to 8a5aefb Compare March 12, 2020 11:57

simonhong removed the CI/skip Do not run CI builds (except noplatform) label Mar 12, 2020

simonhong force-pushed the ntp_super_referral_images branch 2 times, most recently from 8022ac4 to 83226b8 Compare March 13, 2020 00:44

simonhong added the CI/skip Do not run CI builds (except noplatform) label Mar 13, 2020

simonhong force-pushed the ntp_super_referral_images branch from f6c73cd to bf9ea53 Compare March 16, 2020 01:20

simonhong removed the CI/skip Do not run CI builds (except noplatform) label Mar 16, 2020

simonhong requested a review from fmarier as a code owner March 17, 2020 00:40

bsclifton reviewed Mar 17, 2020

View reviewed changes

browser/BUILD.gn Outdated Show resolved Hide resolved

bsclifton reviewed Mar 17, 2020

View reviewed changes

components/ntp_background_images/browser/ntp_background_images_source_unittest.cc Show resolved Hide resolved

simonhong added 19 commits April 17, 2020 12:49

Fix wrong check for effective ntp images data

26ae16e

It should ignore for in-effective images data when it gets update.

Fetch map-table data with more strict request flag

e57aa60

Make SI data availalbe only after referrals service is initialized

8c9a636

W/o this, we could display SI data for super referral.

Use alias for mapping table url

d2b94bb

Improve handling for failure during first launching

c06275b

Add API for getting theme name

9afbb06

Use cached NTP SR component image resource

fea05ab

NTP displays SR data always. So, caching it and use it instead of using data from component folder. If not, we have to wait to get SR data until component is ready.

Cache super referrals code

5dd91a1

Android needs super referrals code during the campaign to generate QR code. Backend will clear it when campaign ends.

Add more comments

8754a09

Don't load SR component via demo flag

f139bb1

When registering demo, it skips some steps that sr component registration does. It can ruins some preferences. Instead of this, we can demo SR component easily by inserting promoCode file in user data dir.

Use changed prefs name

16f827e

Enable SR regardless of NTPBGImages option

f25125d

ViewCounterService will provide SR data even if NTPBGImages pref is false.

Always show SR images if current profile choose SR theme

c7fe7f5

Regardless of background image option, NTP will show SR images if user choose SR theme in settings.

Update CODEOWNERS

4c1b822

Don't use custom homepage term. Use super referral

8f1f9b7

Use component for getting mapping table

8b37618

Don't use different manifest file name for SI and SR component

a3eeb48

Use same manifest filename and determine it's SI or SR by its contents. And delete utils. APIs that in utils should be in service impl file.

simonhong force-pushed the ntp_super_referral_images branch from caa2905 to a3eeb48 Compare April 17, 2020 05:10

simonhong merged commit 24dc9ac into master Apr 17, 2020

simonhong deleted the ntp_super_referral_images branch April 17, 2020 08:08

simonhong mentioned this pull request Apr 20, 2020

Don't show branded wallpaper notification in NTP for Super Referral brave/brave-browser#9329

Closed

brave-builds mentioned this pull request Apr 20, 2020

Implement NTP Custom homepage ("super referral") (uplift to 1.8.x) #5296

Merged

4 tasks

bsclifton pushed a commit that referenced this pull request Apr 20, 2020

Merge pull request #4866 from brave/ntp_super_referral_images

c7e8ff0

Implement NTP Custom homepage ("super referral")

bsclifton mentioned this pull request Dec 16, 2020

Super Referral builds should be able to specify NTP widgets brave/brave-browser#13212

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement NTP Custom homepage ("super referral") #4866

Implement NTP Custom homepage ("super referral") #4866

simonhong commented Mar 6, 2020 •

edited

Loading

simonhong commented Mar 12, 2020 •

edited

Loading

simonhong commented Mar 12, 2020

simonhong commented Mar 17, 2020 •

edited

Loading

simonhong commented Apr 17, 2020

bsclifton commented Apr 20, 2020

GeetaSarvadnya commented Apr 22, 2020 •

edited by kjozwiak

Loading

Implement NTP Custom homepage ("super referral") #4866

Implement NTP Custom homepage ("super referral") #4866

Conversation

simonhong commented Mar 6, 2020 • edited Loading

Settings page with Super referral

Settings page with non Super referral

NTP Pages with Super referral

Submitter Checklist:

Test Plan:

Two scenarios that check this change doesn't affect existing users

Reviewer Checklist:

After-merge Checklist:

simonhong commented Mar 12, 2020 • edited Loading

simonhong commented Mar 12, 2020

simonhong commented Mar 17, 2020 • edited Loading

simonhong commented Apr 17, 2020

bsclifton commented Apr 20, 2020

GeetaSarvadnya commented Apr 22, 2020 • edited by kjozwiak Loading

simonhong commented Mar 6, 2020 •

edited

Loading

simonhong commented Mar 12, 2020 •

edited

Loading

simonhong commented Mar 17, 2020 •

edited

Loading

GeetaSarvadnya commented Apr 22, 2020 •

edited by kjozwiak

Loading