This repository has been archived by the owner on Dec 11, 2019. It is now read-only.
YouTube - Player Update - Authenticated users seeing ads #7230
Assignees
Labels
Comments
lukemulks
added
feature/adblock
help wanted
|
I removed labels for OS as the issue affects all of the platforms. |
|
Thanks!
…On Feb 13, 2017 7:02 PM, "Suguru Hirahara" ***@***.***> wrote:
I removed labels for OS as the issue affects all of the platforms.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
<#7230 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIkDKAtwCtmhsYkrqTU0Qp8IUzA8b6fIks5rcRk5gaJpZM4L__ss>
.
|
|
Was able to pull a debug report from the player after catching an ad: |
|
Ok, @bbondy @bridiver I'm officially at my breaking point with the YT ad situation. There's a ton of info braindumped above, but I'll recap below: 02.10 - YT updated their player. Only users that are logged in experience the issues w/ad blocking in YT. I've compared uBlock, AdBlock and AdBlock Plus blocking to ours, and am curious about the following: -When was the last update for our Easylist file (or does this auto-update)? After comparing the traffic, I went into about:adblock in brave and made the following custom rules to match (minus cosmetic element hiding, etc): There are some listed above that are potentially duplicates, but I went overboard just in case. I started exploring the extensions in Chrome (dev mode for extension - ctrl+f8 dev tools), and in uBlock's advanced settings: Ublock origin has some |
|
Moving this issue to |
This was referenced Mar 1, 2017
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Did you search for similar issues before submitting this one?
Yes
Describe the issue you encountered:
Youtube updated their player on 02.10.2017, working in some additional ad blocking flags and countermeasures that had been resolved w/Brave v0.13.0.
After the update on 02.10.17, unauthenticated users are still able to block ads, but authenticated users are seeing ads serve with shields up.
Embedded iframe youtube players on non-yt domains are also failing to play with shields up. Once shields are down, the embedded content plays.
Platform (Win7, 8, 10? macOS? Linux distro?):
MacOS, Win10, Linux
Brave Version (revision SHA):
v0.13.2
Steps to reproduce:
Actual result:
Ads
Expected result:
No ads (match behavior for unauthenticated users)
Will the steps above reproduce in a fresh profile? If not what other info can be added?
Yes.
Is this an issue in the currently released version?
Yes.
Can this issue be consistently reproduced?
Yes
Breakout of investigation so far:
QUIC disabled in v0.13.0
02.10.2017
The Problem: Many of the pieces involved overlap between content and ads. I'm going to lay out the investigation below, along with some additional areas we could test further.
Client side blocking doesn't appear to work so well from traditional blacklisting methods.
Recommending that we try blocking VAST and VMAP, which are specific to ads, and shouldn't impact content.
The challenge with VAST/VMAP blocking is that this is typically done in pure XML, but appears to be encoded to work with the gRPC config.
Rewrites, etc. might work well as an alternative
============================================
Notes from the previous investigation: 02.10-02.13.17
gRPC (observed when QUIC is disabled)
It was discovered that Google is using gRPC for RPC method bundling for Google APIs (location, ads, books, maps, social).
Each of these JS requests contain a different
gapi_loaded=[VALUE]for a different set of Google APIscb=gapi.loaded_0:Full Request:
https://apis.google.com/_/scs/abc-static/_/js/k=gapi.gapi.en.ht56ryQzVZ8.O/m=googleapis_proxy/rt=j/sv=1/d=1/ed=1/rs=AHpOoo9BOneF1L6M_FnbxCRKqYNTbnMzBg/cb=gapi.loaded_0cb=gapi.loaded_2:Full Request:
https://apis.google.com/_/scs/abc-static/_/js/k=gapi.gapi.en.ht56ryQzVZ8.O/m=client/exm=card,gapi_iframes,gapi_iframes_style_slide_menu/rt=j/sv=1/d=1/ed=1/rs=AHpOoo9BOneF1L6M_FnbxCRKqYNTbnMzBg/cb=gapi.loaded_2The IMA code from the response:
Annotations are used for XML nodes and tracking. Code from response.
OAuth postMessage relay
Google is also using OAuth postMessage relay calls w/gRPC that are packaging ad request data
Request URL:
https://accounts.google.com/o/oauth2/postmessageRelay?parent=https%3A%2F%2Fwww.youtube.com&jsh=m%3B%2F_%2Fscs%2Fabc-static%2F_%2Fjs%2Fk%3Dgapi.gapi.en.ht56ryQzVZ8.O%2Fm%3D__features__%2Frt%3Dj%2Fd%3D1%2Frs%3DAHpOoo9BOneF1L6M_FnbxCRKqYNTbnMzBg
Client Side Network Traffic
The VMAP and VAST Ad Manifest framework preloads w/this request:
https://www.youtube.com/watch?v=r0n-ED3Oxr0&t=2s&spf=navigate
"vmap": "\u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\u003cvmap**:VMAP** xmlns:vmap=\"http:\/\/www.iab.net\/videosuite\/vmap\" xmlns:yt=\"http:\/\/youtube.com\" version=\"1.0\"\u003e\u003cvmap:AdBreak breakType=\"linear\" timeOffset=\"start\"\u003e\u003cvmap:AdSource allowMultipleAds = \"false\"\u003e\u003cvmap:VASTData\u003e\u003**cVAST** version=\"3.0\"\u003e\u003cAd\u003e\u003cWrapper\u003e\u003cAdSystem version=\"1\"\u003eYT:DoubleClick\u003c\/AdSystem\u003e\u003c**VASTAdTagURI\**u003e\u003c!VASTAdTagURI= the XML node that contains the URI to the video ad.
These all appear to be encoded for the gRPC configuration. Traditionally they'd be in XML.
Google's JS API autoload call for their ad module.
This request includes an ad module flag within the query string parameters (Google uses his same call for other modules):
Request URL:
https://www.google.com/jsapi?autoload=%7B%22modules%22%3A%5B%7B%22name%22%3A%22ads%22%2C%22version%22%3A%221%22%2C%22callback%22%3A%22(function()%7B%7D)%22%2C%22packages%22%3A%5B%22content%22%5D%7D%5D%7DDecoded Query String Parameter values:
autoload:{"modules":[{"name":"ads","version":"1","callback":"(function(){})","packages":["content"]}]}This is used to write and load the IMA into an iframe: :https://www.google.com/uds/api/ads/1.0/4088387d0f85fb494bbd48b881b424a7/content.I.js
This is used for what would typically be XML Tracking Extension nodes:
https://www.youtube.com/annotations_invideo?instream_ad=*This is used for the instream ad stats api:
https://www.youtube.com/api/stats/ads?*Blacklisting/Client Side Blocking Tests:
I tried blocking each of the above using blacklisting and URL block rules, and didn't get anywhere. I think this is due to the calls being formed via gRPC, which is using XHR with fallbacks when errors are thrown by blocking.
Player Resources (for rewirting, etc).
I dug deeper into the resources for the player and ads (Dev Tools / Sources) so I could catch the failovers and other ad stuff within each of the client side resources.
The screencap below is marked up to include points of reference for the YT Ads and Player config (taken when an ad was playing).
The heart of the client-side ad config is within the following resource files:
top / www.youtube.com / yts / jsbin/...
...player-enUS-vfl8LqiZp/remote.js
(contains cast_sender.js extension calls for proxy requests)
...player-enUS-vfl8LqiZp/annotations_module.js
(contains VAST references)
...player-enUS-vfl8LqiZp/base.js
(VAST references)
...www-en_US-vflzPvQ1b/base.js
(ad configuration)
...www-en_US-vflzPvQ1b/watch.js
(majority of ad references, including ad blocking references)
...www-en_US-vflzPvQ1b/common.js
(several ad references)
...www-en_US-vflzPvQ1b/watch_autoplayrenderer.js
(no direct ad references, but prefetch references)
...www-en_US-vflzPvQ1b/watch_speedyg.js
(invokes the spdy player config)
...www-en_US-vflzPvQ1b/watch_transcript.js
(appears to be where the VAST XML is parsed)
I need assistance with:
-Figuring out the best method of attack.
-Implementing it in a frictionless way.
-Testing in MacOS
-I'm down and available to test in Windows.
The text was updated successfully, but these errors were encountered: