Reload when playback is unexpectedly frozen with encrypted but only d…

…ecipherable data in the buffer

Note: This may be a controversial commit :p, don't hesitate to exchange
in the linked PR if you see problem with this logic or if you have
ideas for improving it.
This should only impact contents relying on DRM.

---

The great majority of our bugs are now DRM-related and device-specific,
generally platform lower-level (CDM, browser integration...) bugs.
Even if we very frequently exchange with partners to obtain fixes,
this is not always possible (sometimes because there are too many
people relying on the older logic and thus risks in changing it,
sometimes because there are a lot of higher priorities on their side).

We are now encountering frequently, for some contents with DRm, what
we call a "playback freeze": playback does not advance despite having
data in the buffer, all being known to be decipherable (we know that
we have pushed a license and that their linked key has a key id set to
"usable").

Recently we've seen this similar issue on Microsoft Edge, UWP
applications (Windows, XBOX...) and LG TV.

In all three cases, what we call "reloading" after the license has been
pushed always fixes the issue. The action of reloading means principally
to re-create the audio and video buffers, to then push again segments on
it.
Other work-arounds may work on some platform, but do not seem to work on
other.

Although we prefer providing more targeted fixes or telling to platform
developpers to fix their implementation, this issue is so frequent that
we're now wondering if we should provide some heuristic in the RxPlayer,
to detect if that situation arises and reload in that case, as a fallback
mechanism.
The logic being the following: we prefer to reload over having an infinite
loading or buffering phase.

What we are most afraid here is the risk of false positives:
falsely considering that we are in a "decipherability-freeze" situation,
where it's just a performance issue on the hardware side or an issue in
a segment.

To limit greatly the risk of false positives, here are the rules that
will lead to a reload under that heuristic.
All sentences after dashes here are mandatory:
  - we have a `readyState` set to `1` (meaning that the browser
    announces that it has enough metadata to be able to play, but does
    not seem to have any media data to decode)
  - we have at least 6 seconds in the buffer ahead of the current
    position (so compared with the previous dash, we DO have enough data
    to play - generally this happens when pushed media data is not being
    decrypted).
  - The playhead (current position) is not advancing and has been in this
    frozen situation for at least 4 seconds
  - The content has at least 1 audio or video Representation with DRM.
  - One of the following is true:
      1. There is at least one segment that is known to be
         undecipherable in the buffer (it should not happen, the RxPlayer
         already preventing this situation elsewhere, this was added as a
         security)
      2. There are ONLY segments that are known to be decipherable
         (licenses have been pushed AND their key-id are all `"usable"`)
         in the buffer, for both audio and video.

If all those conditions are `true` we will reload. For now with no limit
(meaning we could have several reloads for one content if the situation
repeats).
Note that at the request level, this only might influence segment
requests (which will have to be reloaded after 4 seconds) and not
DRM-related requests not Manifest requests.

We tested that logic with success on the problematic devices.

We're now in a testing/reviewing phase where we are testing and
considering the false positive risks, we're still adding and removing
tweaks to the conditions.

This is only planned to be added on the v4 for our official release, for
API-breakability reasons BUT we plan to test it internally at Canal+ in
some special v3 pre-releases, to ensure that this works well.

src/core/init/directfile_content_initializer.ts

@@ -84,6 +84,7 @@ export default class DirectFileContentInitializer extends ContentInitializer {
      * events when it cannot, as well as "unstalled" events when it get out of one.
      */
     const rebufferingController = new RebufferingController(playbackObserver,
+                                                            null,
                                                             null,
                                                             speed);
     rebufferingController.addEventListener("stalled", (evt) =>

src/core/init/media_source_content_initializer.ts

@@ -448,9 +448,20 @@ export default class MediaSourceContentInitializer extends ContentInitializer {
 
     const rebufferingController = this._createRebufferingController(playbackObserver,
                                                                     manifest,
+                                                                    segmentBuffersStore,
                                                                     speed,
                                                                     cancelSignal);
-
+    rebufferingController.addEventListener("needsReload", () => {
+      // NOTE couldn't both be always calculated at event destination?
+      // Maybe there are exceptions?
+      const position = initialSeekPerformed.getValue() ?
+        playbackObserver.getCurrentTime() :
+        initialTime;
+      const autoplay = initialPlayPerformed.getValue() ?
+        !playbackObserver.getIsPaused() :
+        autoPlay;
+      onReloadOrder({ position, autoPlay: autoplay });
+    }, cancelSignal);
     const contentTimeBoundariesObserver = this
       ._createContentTimeBoundariesObserver(manifest,
                                             mediaSource,
@@ -732,11 +743,13 @@ export default class MediaSourceContentInitializer extends ContentInitializer {
   private _createRebufferingController(
     playbackObserver : PlaybackObserver,
     manifest : Manifest,
+    segmentBuffersStore : SegmentBuffersStore,
     speed : IReadOnlySharedReference<number>,
     cancelSignal : CancellationSignal
   ) : RebufferingController {
     const rebufferingController = new RebufferingController(playbackObserver,
                                                             manifest,
+                                                            segmentBuffersStore,
                                                             speed);
     // Bubble-up events
     rebufferingController.addEventListener("stalled",

src/core/init/utils/rebuffering_controller.ts

@@ -30,7 +30,7 @@ import {
   IPlaybackObservation,
   PlaybackObserver,
 } from "../../api";
-import { IBufferType } from "../../segment_buffers";
+import SegmentBuffersStore, { IBufferType } from "../../segment_buffers";
 import { IStallingSituation } from "../types";
 
 
@@ -54,6 +54,7 @@ export default class RebufferingController
   /** Emit the current playback conditions */
   private _playbackObserver : PlaybackObserver;
   private _manifest : Manifest | null;
+  private _segmentBuffersStore : SegmentBuffersStore | null;
   private _speed : IReadOnlySharedReference<number>;
   private _isStarted : boolean;
 
@@ -72,12 +73,14 @@ export default class RebufferingController
    */
   constructor(
     playbackObserver : PlaybackObserver,
-    manifest: Manifest | null,
+    manifest : Manifest | null,
+    segmentBuffersStore : SegmentBuffersStore | null,
     speed : IReadOnlySharedReference<number>
   ) {
     super();
     this._playbackObserver = playbackObserver;
     this._manifest = manifest;
+    this._segmentBuffersStore = segmentBuffersStore;
     this._speed = speed;
     this._discontinuitiesStore = [];
     this._isStarted = false;
@@ -154,6 +157,10 @@ export default class RebufferingController
         Math.max(observation.pendingInternalSeek ?? 0, observation.position) :
         null;
 
+      if (this._checkDecipherabilityFreeze(observation)) {
+        return ;
+      }
+
       if (freezing !== null) {
         const now = performance.now();
 
@@ -215,7 +222,7 @@ export default class RebufferingController
           this.trigger("stalled", stalledReason);
           return ;
         } else {
-          log.warn("Init: ignored stall for too long, checking discontinuity",
+          log.warn("Init: ignored stall for too long, considering it",
                    now - ignoredStallTimeStamp);
         }
       }
@@ -358,6 +365,87 @@ export default class RebufferingController
   public destroy() : void {
     this._canceller.cancel();
   }
+
+  /**
+   * Support of contents with DRM on all the platforms out there is a pain
+   * considering all the DRM-related bugs there are.
+   *
+   * We found out a frequent issue which is to be unable to play despite having
+   * all the decryption keys to play what is currently buffered.
+   * When this happens, re-creating the buffers from scratch, with a reload, is
+   * usually sufficient to unlock the situation.
+   *
+   * Although we prefer providing more targeted fixes or telling to platform
+   * developpers to fix their implementation, it's not always possible.
+   * We thus resorted to developping an heuristic which detects such situation
+   * and reload in that case.
+   *
+   * @param {Object} observation - The last playback observation produced, it
+   * has to be recent (just triggered for example).
+   * @returns {boolean} - Returns `true` if it seems to be such kind of
+   * decipherability freeze, in which case this method already performed the
+   * right handling steps.
+   */
+  private _checkDecipherabilityFreeze(
+    observation : IPlaybackObservation
+  ): boolean {
+    const { readyState,
+            rebuffering,
+            freezing } = observation;
+    const bufferGap = observation.bufferGap !== undefined &&
+      isFinite(observation.bufferGap) ? observation.bufferGap :
+      0;
+    if (
+      this._segmentBuffersStore === null ||
+      bufferGap < 6 ||
+      (rebuffering === null && freezing === null) ||
+      readyState > 1
+    ) {
+      return false;
+    }
+
+    const now = performance.now();
+    const rebufferingForTooLong =
+      rebuffering !== null && now - rebuffering.timestamp > 4000;
+    const frozenForTooLong =
+      freezing !== null && now - freezing.timestamp > 4000;
+
+    if (rebufferingForTooLong || frozenForTooLong) {
+      const statusAudio = this._segmentBuffersStore.getStatus("audio");
+      const statusVideo = this._segmentBuffersStore.getStatus("video");
+      let hasOnlyDecipherableSegments = true;
+      let isClear = true;
+      for (const status of [statusAudio, statusVideo]) {
+        if (status.type === "initialized") {
+          for (const segment of status.value.getInventory()) {
+            const { representation } = segment.infos;
+            if (representation.decipherable === false) {
+              log.warn(
+                "Init: we have undecipherable segments left in the buffer, reloading"
+              );
+              this.trigger("needsReload", null);
+              return true;
+            } else if (representation.contentProtections !== undefined) {
+              isClear = false;
+              if (representation.decipherable !== true) {
+                hasOnlyDecipherableSegments = false;
+              }
+            }
+          }
+        }
+      }
+
+      if (!isClear && hasOnlyDecipherableSegments) {
+        log.warn(
+          "Init: we are frozen despite only having decipherable " +
+          "segments left in the buffer, reloading"
+        );
+        this.trigger("needsReload", null);
+        return true;
+      }
+    }
+    return false;
+  }
 }
 
 /**
@@ -581,6 +669,7 @@ class PlaybackRateUpdater {
 export interface IRebufferingControllerEvent {
   stalled : IStallingSituation;
   unstalled : null;
+  needsReload : null;
   warning : IPlayerError;
 }