v3.24.0

Release v3.24.0 (2021-04-01)

(No, it's not an april fools joke, unless it doesn't work, in which case we will pretend it was)

A new release is here! Amongst the new features and improvements:

• a new inbandEvent event is now triggered when event metadata is encountered in a segment (for example through "emsg" boxes in an ISOBMFF segment), just before that segment is pushed

• a new singleLicensePer option now allows to optimize the loading of contents with multiple decryption keys, to be able to perform only a single license request.

• we improved a lot the compatibility of the RxPlayer with several devices, most importantly many Tizen versions (e.g. Samsung TVs) and the PlayStation 4 (as to the PlayStation 5, it should already be handled).

• multiple optimizations have been added to allow lower content loading times

• many other bug fixes and improvements (you can see the changelog at the bottom of this release note)

A new player event: inbandEvent

"Inband" events

Streaming formats specify different ways to emit events through a content.
Those events can be sent at any time when streaming the content, and can be used to signal generic or format-specific information to a player or the application (e.g. signaling about parental ratings on the current content or about the Manifest's validity).

The RxPlayer already supported one kind of event: the MPEG-DASH manifest EventStream node, allowing an event to be declared directly in the Manifest.
But instead of only having events in the manifest, those can also be found inside media segments.

MPEG-DASH uses such format, by exploiting the "emsg" box of the ISOBMFF standard. This box is included in the metadata part of ISOBMFF segments (as in: it is not included in the media data itself) and is usually related to the media presentation time.

Those type of events, which are included directly in the segments, are usually called "inband events".

In the RxPlayer: inbandEvents event

The RxPlayer now parses these boxes (only the version 0 of it for now) and :

• Either the event has a specific meaning for a player (generally to give information
 about the manifest validity, allowing the player to consider when it may refresh the manifest) in which case it will handled it by itself.
• Either, the event is generic, or specific but not yet handled by the player. In which case it will be parsed and emitted 
by the RxPlayer through an event.

This new event is called "inbandEvents".
It emits one or more parsed inband events, as several "emsg" boxes may be found when parsing a unique media segment.

The "inbandEvents" event is emitted just after the corresponding segment is parsed, unlike the already-existing "streamEvent", which is sent at the presentation time of the concerned event.

And as an "emsg" box is associated with a specific segment, the player send it at the moment it gets it, before that segment is actually pushed.
This is important as it may be used to signal that the current segment contains content that may be subject to rating restrictions.
In that case, the client may need to interrupt playback before the segment's data is buffered.

This new event and its format is documented here.

singleLicensePer option: single license request for contents with multiple keys

Previous situation

When playing encrypted contents, the RxPlayer has a simple logic to know when it needs to perform a licence request:

1. Let's say that the player switches to a new Representation/quality
2. It looks at that quality's linked encryption initialization data if it exists:
   • if it was already encountered, it does nothing in particular with it
   • if it wasn't yet encountered, it goes to the next step
3. The player creates a new decryption session (called MediaKeySession) associated to that data and ask that session to generate a license request with it
4. The browser sends the player an event back with a "challenge" allowing it to perform a license request by calling the getLicense callback
5. Your application fetches the license and give it to the RxPlayer, which then pass it down through the MediaKeySession.update API
6. The RxPlayer then looks at which keys were contained in that license and reacts accordingly (it might fallback for unsupported qualities etc.)

This works pretty well, but it generally leads to multiple license requests when a content has different keys depending on the quality chosen.

This is because Representations linked to different keys will generally have different initialization data. The RxPlayer will thus create a new MediaKeySession and perform a new license request for each of them.

Optimization: performing only a single license request

We've looked at possible ways to optimize this logic. The most evident solution is to only perform a single license request.

Yet that would mean that the license returned contains all needed keys.
As such, this optimization relies on two things:

1. the license server has to be aware of this "trick" and return all keys even when a single one is asked for
2. the RxPlayer has to only perform a single license request.
   It could still technically perform multiple ones but this would be pointless as the first one already gave everything needed.

For that second part, we added the new singleLicensePer properties to the keySystems option of the loadVideo API.
By setting it to "content" (more possible values might come in the future), you can signal that there's only a single license available for the whole content.

It can be used as such:

rxPlayer.loadVideo({
  // ...
  keySystems: [{
    // ...
    singleLicensePer: "content",
  }]
}

This option is documented inside the keySystems documentation.

Remaining problem: what about missing keys?

But if the RxPlayer only decided to perform a single license request and did not change anything else in its behavior, it might lead to an other issue: what should happen if one of the keys is not in the license?

This might happen when the CDM, the module plugged in the browser allowing content decryption, is not trusted enough by the license server, resulting in the latter withholding some keys (the more restrictive ones).

The decision we've made was that, in the case an awaited decryption key isn't present in the pushed license, the player will automatically decide to mark the corresponding Representations as "undecipherable".

In this situation, those qualities won't be played. If segments corresponding to that quality were already pushed (we load media segments in parallel of the license request as an optimization measure), the RxPlayer will first try to remove them and might in some very rare conditions switch to the "RELOADING"state while doing so.

Improved compatibility with several platforms

At Canal+ Group, we recently had a wave of new platforms on which the RxPlayer was ported (I'll just name-drop the PlayStation 5, PlayStation 4 and Samsung TVs but there's others).
This was the occasion to improve our records on portability.

The main issues we encounter when porting the RxPlayer are subtle browser behavior differences, sometimes still specification-compliant and sometimes not, most of them being related to their respective MSE/EME implementations.

There was a lot of changes on our side (and multiple times on the implementer's side! We don't always want to fix another application's mess :p).

Most notably, they're fixing:

• A mysterious infinite loading situation when beginning a live content on some platforms (amongst which, Samsung TVs), most likely due to an overflow on the browser-side after setting the duration (#914)
• A situation where both the browser and the RxPlayer wanted to seek at different times, leading to a conflict (#922)
• decoding issues we encountered when pushing segments on top of the current position (#925)
• A leak of MediaKeySession on platforms where the generateRequest API is especially long (#920)
• Persistent MediaKeySessions being thrown away because too much time passed before the player was aware of its linked keys (#928, #926)

Loading time improvements

In this release, we also began (and still continuing) trying to heavily optimize the initial content loading time:

• One of the main improvements is that now when calling loadVideo when a content was already playing, the new content's manifest will be loaded in parallel of the old content being stopped (implementation details: #894)

• Another one, only for encrypted contents, is to depend only on the encryption initialization data in the Manifest if it already contains the right one (instead of the default behavior, which waits for a segment to be fetched first).

  This optimization was a complex one to implement because it implied recognizing which encryption initialization data is needed, whereas the previous implementation just used all available data - a solution which always work (implementation details: #911, #919).

  If you rely on persistent licenses, this optimization won't be available to you by default. You will need to set the disableRetroCompatibility property to true in the licenseStorage option (itself part of the keySystems `loadVi...

v3.23.1

Release v3.23.1 (2021-02-01)

This very small release only fixes a single issue, brought with the v3.23.0 release published this same day, which made it impossible to play encrypted contents on Safari.

Sorry about that, we normally test that specific use case, but we didn't after a quick code update* we did just before releasing the v3.23.0.

We will try to be more careful from now on and always test that use case before new releases, no matter how small the update!

* For the more curious readers, the incriminating update we're speaking of here was about throwing better errors when playing encrypted contents on platforms which do not support decryption : ed282ee. While writing that, I didn't realize that the else condition I was replacing was for a much more general case than what I replaced it with.

Changelog

Bug fixes

• Fix support of encrypted contents on Safari (v3.23.0 regression)

v3.23.0

Release v3.23.0 (2021-02-01)

This release adds multiple new methods and options to the RxPlayer, but also many other improvements.

Most importantly:

• A reload method has been added, to be able to manually quickly re-load a content after something when wrong (or even when nothing went wrong at all)

• a whole new discontinuity management system has been written, which allows the player to automatically detect and step over most "holes" in a content, even when those were not declared as such in the Manifest.
  Those type of holes are most often encountered between Periods in multi-Period DASH contents but can also be encountered in poorly encoded/packaged contents.

• A onCodecSwitch loadVideo option has been added, so you can ask the player to reload automatically when switching between incompatible video codecs or audio codecs, which you might want to do on some devices/browsers.

• multiple methods and options have been added to indicate a minimum audio and/or video bitrate to which the player should switch to when adaptive streaming is enabled

• Fallbacking to another Representation due to an in-exploitable decryption key (only when the keySystems[].fallbackOn loadVideo option is set) could be non-functional on some devices since the v3.21.1. This is now fixed.

A new reload method


We’ve noticed that there are some cases where an application wants to load again the last loaded content.
Some of these reasons could be:

• A fatal error occurred due to exceptional reasons, and thus may not reproduce after reloading the content.
• A decoding error occurred at a given playback time (e.g. the corresponding media chunk is defective), and provoked a crash. Here we can re-load while "skipping" the faulty segment.

For now, if an application developer wants to reload the exact same content, even with the same options than before, he/she needs to call the loadVideo API again. Then, the loading process will be decomposed into several steps, some of which are time-consuming: loading the manifest, communicating with the media browser APIs, etc.

This implies that after re-loading a content, all these steps will be taken for a second time. In some cases this is completely unnecessary, as fetched or parsed resources may not change from the first loadVideo to the next.

For now, we have identified one major step that can be avoided: fetching the manifest and turn it into our internal manifest format.
This operation can take a certain amount of time, which could be removed here by using the previously loaded manifest.

Thus, we’ve introduced a new API, the reload method.

It can be called at any moment, after the LOADED state for the last loadVideo has been received, even if the content is still playing.
The aim of the API is to propose to reload the last loaded content as fast as possible, by default at the last playback position it was set to.

The API options allow setting an absolute or relative (to the last playback position) position to reload playback:

// Reload the content at the last position it was set to
rxPlayer.reload();

// Reload video at a specific position
rxPlayer.reload({ reloadAt: { position: 55 } });

// Reload 5 seconds AFTER the last position it was set to
rxPlayer.reload({ reloadAt: { relative: +5 } });

// Reload 5 seconds BEFORE the last position it was set to
rxPlayer.reload({ reloadAt: { relative: -5 } });

The reload method is documented here.

Improved discontinuity management

This release contains a completely revamped discontinuity management logic.

A "discontinuity" is the name we give to holes in a media content.
For example, we could have a live content with an imperfect transition between ads and the show coming just after, where a few seconds of video would be missing.
Here the player has to do something - most likely step over that hole - to avoid getting stuck on it.

Most discontinuities were already handled in previous versions, but there were still issues we had - most often in contents concatenating multiple sub-contents (multi-Period DASH contents, "ad-switched" contents, MetaPlaylists...).

Previous discontinuity management logic

In previous versions of the RxPlayer, two types of discontinuities where handled, through a very simple logic.

When the player was stuck, it basically asked itself two questions:

• is there a very small (less than 1 second) hole at the current position? -> If yes, seek over it.
• does the Manifest tells us that there's no segment there? -> If yes, seek at the start of the next segment.

Those worked with most type of discontinuities, which are often either very small or clearly announced in the Manifest.
Yet it still had important drawbacks:

1. It was possible to have false positives: we could be stepping over what we thought to be small holes but which were just yet-to-be-downloaded segments

2. For discontinuities not announced in the Manifest specifically, the size of the "hole" had to be known before being able to seek over it.
   This meant we had first to load and push the first segment coming after that discontinuity.

   This could lead to a deadlock when loading the next segment is not yet possible.

3. Discontinuities defined in the Manifest might not reflect the exact reality once the segments are pushed: discontinuities might actually be bigger or smaller than initially announced

Solution: changing the whole logic

So we completely scrapped what we did before and thought about what the optimal way of handling discontinuities should be.

The answer we came with was the following:

1. The RxPlayer can reliably know which "holes" in the audio and/or video buffers won't ever be filled by any segment. When such a situation is encountered, it is signaled internally.

2. When one of this hole is finally encountered, playback may be stuck.
   When that happens, the player will cross-reference the position at which it is stuck with the currently signaled "holes". When a hole corresponds to that position, it automatically seeks over it.

This should completely eliminate the risk of false positives and handle almost all discontinuities whether they were clearly defined in the Manifest or not.

That new logic also comes with a smarter handling of boundaries between multiple Periods, which is where most discontinuities are encountered.

A new DISCONTINUITY_ENCOUNTERED event

Because an application might want to be be notified when discontinuities are seeked over, we also added a new warning event, emitted when this happens.
Emitted with that warning event, will be a MEDIA_ERROR instance with the "DISCONTINUITY_ENCOUNTERED" code.

You can thus know when a discontinuity has been seeked over by writing:

rxPlayer.addEventListener("warning", function (warning) {
  if (warning.code === "DISCONTINUITY_ENCOUNTERED") {
    console.log("A discontinuity has been encountered:", warning);
  }
});

Codec switching behavior

It is sometimes possible to encounter multiple different audio and/or video codecs in the same content.

For example, you might have a high-quality HEVC video track and a lower-quality AVC video track. You might also have a multi-Period DASH contents, where some Periods contain a Dolby Digital Plus audio track but others do not.

As a consequence, the RxPlayer could need to transition between two completely different codecs.
For example, the application could chose to select an HEVC video track while it was playing an AVC one, or a codec switch could just occur while entering a new Period.

The previous strategy when doing that was to just concatenate both type of segments on the same buffer while using the changeType API if available.
We found that strategy to be unreliable: depending on the device and browser, switching the codec that way could either work or totally fail. We have multiple examples of problem we encountered when doing just that on some devices: decoding error, error while appending and infinite rebuffering issues.

Another strategy would be to just reload the content any time an audio or video codec switch is encountered.
This should work for all platforms and browsers, but it means that for a small period of time, the player will go through the RELOADING state, during which the screen becomes black and most APIs are not usable. This leads to a much less pleasant experience.

From those elements, we thought that the best possible solution was to choose the first strategy (just continue pushing) when it worked and use the second one (reload) when it didn't.
We first planned to automatically detect the right strategy to take in the player (e.g. based on the presence or not of the changeType API) but we found out that doing this is very difficult and may not even be possible.

In the end, the solution we propose in this release is to let the application select the correct strategy through a new option.
An application should have a better idea of supported platforms and browsers, and the behavior when switching codecs can generally easily be checked.

This takes the form of a new loadVideo option, called onCodecSwitch, which is documented here.
It can take two values: either "continue" which is the default non-disruptive strategy (which may not work for all targets) we already have, or "reload", the disruptive strategy known to work for all targets:

// Reload every time a new codec is enc...

v3.22.0

Release v3.22.0 (2020-11-17)

Overview

This release adds multiple features and improvements:

• add an audioTrackSwitchingMode property to loadVideo to select the RxPlayer's behavior when switching the audio track (to choose between smoothness or instantaneity)
• add the initialManifest loadVideo option to avoid loading the Manifest when not necessary
• add an enableFastSwitching loadVideo option to enable or disable optimizations doing segment replacement in the browser's buffer
• the TextTrackRenderer tool is not experimental anymore. It can be used without fearing that its API will change
• A new version of the experimental local Manifest which allows much more advanced behavior, such as playing a content which is still downloading.
• A fix for an issue which made it impossible to play encrypted contents when in directfile mode
• saner defaults for TTML subtitles (white, centered text)

audioTrackSwitchingMode: behavior when switching between audio tracks

Until now when switching to a different audio track, the RxPlayer tried to make the transition as smooth as possible.

It doesn't interrupt playback of the old track, but load in parallel the new track and once done, replace the old by the new - most of the time without needing to pause playback.
Because this means that the user would still play in the old track for some time, a time limit (now of 2.5 seconds) is set from the point when the user chose a different track. If he/she has played until that limit, the player will enter a BUFFERING phase until the new track is available.

This behavior allows a very smooth experience but has two drawbacks:

1. It means that the switch won't happen right away.
   For example when switching between languages, you might prefer to interrupt the track in the previous language to play the right one at the same position you were in when you switched, even if it means interrupting playback for a while.

2. On some peculiar devices, multiple issues could arise when replacing those tracks smoothly. Sometimes it is not taken into account, sometime it doesn't even work.
   Note that we're not speaking of the usual web targets here but only of a minority of embedded devices.

Thus, we now let the possibility to adopt different strategies depending on how the audioTrackSwitchingMode API is set:

• Either, "seamless", which is the default behavior, where the transition will appear smooth and will not interrupt playback but could take time to be effective.
• Or, "direct", it will be much more instantaneous but it is possible that the player will go in "RELOADING" state in the meantime and so, may interrupt playback during a brief time.
  Note that we try to avoid going into the "RELOADING" state. We only do it when it is really necessary.

This property is a loadVideo option documented here.

For example to go into a more "direct" track switching, you can do:

rxPlayer.loadVideo({
  // ...
  audioTrackSwitchingMode: "direct",
});

initialManifest: provide the Manifest file if already downloaded

At Canal+, we encountered situations where the application calling the RxPlayer needed to process data from the Manifest before loading a video.

As such, the Manifest would be loaded twice. Once by the application beforehand and then by the RxPlayer.
This mean that we would perform an unnecessary request which would be both a waste of time and resources.

To load faster when the Manifest is already available, we consequently just added the initialManifest option to the transportOptions of loadVideo:

rxPlayer.loadVideo({
  // ...
  transportOptions: {
    initialManifest: myManifest,
    // ...
  }
});

As using it has some implications for live contents, we advise to first read the corresponding API documentation, in the transportOptions before using it.

enableFastSwitching: avoid segment replacement on some targets

The RxPlayer always try to play the best possible quality by default.

For example, when you begin to play a content at a low quality (because of poor bandwidth or because this is the initial content) but the bandwidth indicates that we can now download segments of a much higher quality, the player will try to directly replace the low-quality segments it has in its buffer by the higher quality segments it can download.

This lead to a much faster visible change in quality than if the player only loaded and pushed segments coming after those already loaded.

This optimization - replacing low-quality segments with higher quality-segments for raising faster in quality - has been integrated in the player for multiple years, but was given a name only recently: "fast-switching" (name admittedly taken from another player, dash.js as they developped a similar optimization).

By default, "fast-switching" is enabled and has been for a long time. But it's now possible to disable it through a new loadVideo option: enableFastSwitching.
By setting it to false, the player will just push new segments at the end of the buffer, regardless of the quality they are in.

The main reason you would want to disable that feature, and the main reason behind making that option configurable, is that some specific environments react very poorly when segments already in the buffer are replaced.
For example, we noted that some rare environments do not consider the new segments (making that optimization useless). We also worked recently on an environment which reacted very poorly to this optimization, leading sometimes to the video "freezing".
This was for a very specific environment which was still in heavy development and they since fixed that issue, but it was the main trigger behind making this behavior configurable, as anyone could encounter the same situation on a new device.

So you should probably not put it to false (even if there's no problem in doing so beside slower change in quality) but it can be useful when you know what you're doing.
You can consider this as an "expert" option!

This option is documented here.
It is set to true by default. To disable fast-switching, you can thus do:

rxPlayer.loadVideo({
  // ...
  enableFastSwitching: false,
})

The TextTrackRenderer is not experimental anymore

The TextTrackRenderer tool, which was added in the v3.18.0 version of the RxPlayer, allows to show any subtitles in the supported formats on top of a media element (it both parses and synchronizes them), even when the content is not played through the RxPlayer.

Until now, the API was marked as "experimental" which meant that its API could change at any time.

It now has a stable API, meaning it's not experimental anymore!

Note that if you already used it, you now have an action on your side to be compatible to the v3.22.0.
When before you imported the TextTrackRenderer as such:

import TextTrackRenderer, {
  TTML_PARSER,
  VTT_PARSER,
  SRT_PARSER,
  SAMI_PARSER,
} from "rx-player/experimental/tools/TextTrackRenderer";

You now have to get rid of the /experimental tp instead write:

import TextTrackRenderer, {
  TTML_PARSER,
  VTT_PARSER,
  SRT_PARSER,
  SAMI_PARSER,
} from "rx-player/tools/TextTrackRenderer";

As usual, its API documentation is available here.

Default TTML style

We encountered an issue where a user of the RxPlayer library basically told us that we badly formatted TTML subtitles, he couldn't read them well as they were in a black color and they were displayed on the top left of the screen.

After some investigations, we found out that we had no big problem with TTML formatting but that the real issue is that we do not make styling decisions if no style is present in the TTML file itself (and generally, when you are using TTML subtitles, the style is included!).
This means that by default, only the already-present styles of the concerned HTML elements are considered.
On most cases, this means black and on the top left of the screen!

Resolving that issue was not straightforward, because no default style seems to be defined in the TTML specification.
So we weren't sure of what to do here.

After looking at what other players were doing and at what the most sensible thing to do in that situation would be, we now consider that if no style at all is declared in a TTML file, we should apply a default style at its place.
Therefore, if no styles are defined for regions in the TTML, but other styles are (e.g. on HTML elements), we decide not to apply the default style. Indeed, we consider the style to have been defined on purpose, and we may interfere with the overall look of the cue if we apply a different style.

That default style objective is to display text cues in the clearer and cleaner possible way: at the bottom, centered and with white text

serverCertificate optimizations

The serverCertificate property of the keySystems option (in loadVideo) allows to communicate a server certificate.
This is DRM-specific data usually needed to encrypt message between a Content Decryption Module (or CDM) and a license server.

It appears that setting that certificate can take some time: up to 1 second on some embedded devices.
We thus decided to allow a new optimization: when we know a serverCertificate is alr...

v3.21.1

Release v3.21.1 (2020-09-21)

Overview

The v3.21.1 adds many small bug fixes and improvements:

• we fixed an issue that made it difficult to switch between key systems (e.g. from Widevine to PlayReady) on devices and browsers that support it
• we improved the support of DASH MPDs, in particular we now support contents with multiple levels of SegmentTemplate declaration
• we fixed some issues with the manifestUpdateUrl API, which allows to optimize the performance when playing very long Manifest files.
• we brought several small improvements to reduce the delay at which a track is switched and seek are performed and to improve the experience when switching quality
• we updated our adaptive logic so it favours even more smooth playback over maximizing the quality
• many other things, see our changelog on the bottom of this release note for more information (now with links on related PRs and issues!)

Switching between key systems

To be able to play encrypted contents on the browser, a CDM - or Content Decryption Module is used.

That module is usually an external software, plugged to the current browser, whose task will be to do what's needed to decrypt the content.
They are provided by DRM technologies whose names you have probably heard of if you play encrypted contents: Widevine, PlayReady, FairPlay etc.

 How encrypted contents are decrypted via the RxPlayer (simplification):

     +===================+
     |    Application    | Provide wanted DRM configuration(s)
     +===================+ and the logic to fetch a licence
              |
              V
        +----------+
        | RxPlayer | Try to open a session with a CDM having the
        +----------+ wanted configuration.
              |      Once opened, fetch and give the licence - allowing
              |      to play the content - to the CDM.
              V
  +-------------------------+
  |       Web browser       | Provide APIs to interact with CDM(s)
  +-------------------------+
              |
              V
           +-----+
           | CDM | Decrypt the content
           +-----+

Because multiple CDMs exist and because they might have different capabilities depending on the device on which the content plays (for example, devices made for video streaming might have more copy-protection mechanisms), not all browsers or devices use the same ones.

For the most part, we're stuck with only one possibility. For example most Chrome and Firefox builds on a desktop/laptop computer rely on the Widevine CDM and no other.
But other situations exist where multiple CDM can be available at the same time. For example the new Edge browser on desktop will usually allow us to interact either with PlayReady or Widevine. That exact situation is also found on other environments, like on recent Samsung TVs.

  +-------------------------+
  |       Edge Browser      |
  +-------------------------+
       /            \
      /              \
     |                |
     V                V
 +--------+      +---------+
 |  CDM   |      |   CDM   |
 |Widevine|      |PlayReady|
 +--------+      +---------+

The RxPlayer API let you choose which CDM you want by specifying what it calls a "key system": you can play a content with a Widevine CDM by claiming that you want the "widevine" key system.

All that work well as long as the corresponding key system is available. However, there was one situation where the RxPlayer was not so much reliable: when switching from one key system to another one, with the same RxPlayer instance.

Some browsers unexpectedly failed when "switching" the key system from one to another. After some initial tests, we thought that the situation was unsolvable: you were stuck with a key system once you've chosen one on some devices.

                 +-------------------------+
                 |         Browser         |
                 +-------------------------+
                      /            x
                     /              x
                    |                x
                    V                V
                +--------+      +---------+
                |  CDM   |      |   CDM   |
 CDM chosen  -->|Widevine|      |PlayReady|<-- We can't use that
 to play the    +--------+      +---------+    one anymore
 first encrypted
 content

However, we finally found out that by re-ordering some API calls, such error disappeared.
This is not the first time where we find that re-ordering DRM-related API calls unexpectedly unblock some situations on some devices. Because of this, we now follow a very particular sequence of calls that we know to work on all our tested platforms.

The result is that you can now reliably choose a different key system depending on the content played.

An in-depth explanation of the solution might rely on multiple technical terms (most of all MediaSource and MediaKeys) so I wont explain it in that release note. If you're still curious about it (and know what those terms roughly mean), you can look at the corresponding issue (#766) and PR (#744).

Reducing the delay of track switching and some seek operations

One of the main task of the RxPlayer is to schedule the right media data requests at the right time.

The basic behavior behind it is to download in parallel the next most needed audio, video and text (if subtitles are enabled) media "segments" (which are basically small decodable chunks of the whole media data).
When a segment is loaded, it is stored in a buffer so it can be decoded later (as soon as we need it). At the same time, we begin the request for the next most needed audio, video or text segment so we can maximize the bandwidth usage and provide smooth playback with the best possible quality.

 Queues of segment scheduled for download

     AUDIO QUEUE           VIDEO QUEUE          TEXT QUEUE
 +-----------------+   +-----------------+  +-----------------+
 | audio segment 1 |   | video segment 1 |  |  text segment 1 |
 +-----------------+   +-----------------+  +-----------------+
          |                     |                    |
          V                     V                    V
 +-----------------+   +-----------------+  +-----------------+
 | audio segment 2 |   | video segment 2 |  |  text segment 2 |
 +-----------------+   +-----------------+  +-----------------+
          |                     |                    |
          V                     V                    V
 +-----------------+   +-----------------+  +-----------------+
 | audio segment 3 |   | video segment 3 |  |  text segment 3 |
 +-----------------+   +-----------------+  +-----------------+
          |                     |                    |
          V                     V                    V
         ...                   ...                  ...          

However if we did just that, we could quickly encounter a problem. For example, audio segments are generally much smaller in size than video segments, and as such they are loaded much more rapidly. If we loaded the next audio segment as soon as the previous one's request ended, we would be limiting the bandwidth for the more urgent video segment request happening in parallel.

                      AUDIO QUEUE           VIDEO QUEUE     
                  +-----------------+   +-----------------+
distant, not  --> | audio segment 7 |   | video segment 1 | <-- close, really
urgent segment    +-----------------+   +-----------------+     urgent segment
                           |                     |            
                           V                     V            
                  +-----------------+   +-----------------+
                  | audio segment 8 |   | video segment 2 |
                  +-----------------+   +-----------------+
                           |                     |            
                           V                     V            
                  +-----------------+   +-----------------+
                  | audio segment 9 |   | video segment 3 |
                  +-----------------+   +-----------------+  
                           |                     |            
                           V                     V            
                          ...                   ...                                     

To solve this issue while still making the best use of the user's bandwidth, the RxPlayer make a compromise.

First, it assigns to each segment a priority.
When a media segment is loaded, the RxPlayer will first check the priority of the next most needed segment for that type (e.g. "audio"). It will only start a request for that segment if/when its priority is at least on par with the priority of the most needed segments for the other types. This sounds complicated, but it allows to download urgent segments with all available bandwidth while still being able to parallelize segment requests once urgent segments are all downloaded.

    AUDIO QUEUE           VIDEO QUEUE     
                      +-----------------+
                      | video segment 1 |
                      | (high priority) |
   Waiting until      +-----------------+
  higher priority              |            
  video segments               V            
    are loaded        +-----------------+
                      | video segment 2 |
       ...            | (high priority) |
                      +-----------------+
                               |          
                               V          
+-----------------+   +-----------------+
| audio segment 7 |   | video segment 3 |
| (lower priority)|   | (lower priority)|
+-----------------+   +-----------------+
         |                     |         
         V                     V         
        ...           ...

v3.21.0

Release v3.21.0 (2020-06-17)

Overview

The v3.21.0 brings multiple features amongst which:

• the RxPlayer now handle DASH EventStream's Event elements to emit timed events defined in a DASH MPD
• it is now possible to apply track preferences to the content(s) / Period(s) that are already playing.
• WebVTT "settings" are now considered when parsing subtitles in the html textTrackMode, thanks to @mattiaspalmgren for the contribution! Before, we only relied on WebVTT's CSS extensions to allow stylization of subtitles in this mode.
• the RxPlayer will now emit "warning" events when it detects minor errors in a DASH MPD, such as optional data in the wrong format.
• and a lot of other bug fixes and improvements - you can look at the changelog for more information.

Support of DASH EventStream's Event elements

What stream events are for

What we call "stream events" here is metadata synchronized with the content being played. There are several reasons why you might want to include those.

At CANAL+ for example, we started to study the implementation of ad-insertion and tracking solutions for live contents. Ad-insertion can work on the server-side by generating an MPD which will replace media segments by ads, for example by adding Period elements.

On the client-side (the media player), we then may want to know where an ad can be encountered and when the final user begins to see it (or when he/she skip it) for tracking purposes.

Here, DASH EventStream's <Event> elements can be used.
Such elements define timed events associated with the current content. When the user begins to play at a position within the time boundaries of that event, the corresponding Event element will be considered.
This element can contain specific user-defined information, such as the URL of an ad-tracking server.

But DASH <Event> elements are not restricted to advertising use cases, you could use one anytime you want to be notified when an user is at a particular position in the stream.

How the RxPlayer handle DASH EventStream elements

DASH's <EventStream> are elements of the MPD, inside the corresponding <Period> element.
Each EventStream can carry one or more <Event> elements, whose format and inner data depends on an attribute of the EventStream element called the schemeIdUri:

    <Period id="1">
      <EventStream schemeIdUri="urn:uuid:XYZY" timescale="1000" value="call">
        <Event presentationTime="20000" duration="10000" id="1">
          Some data
        </Event>
        <Event presentationTime="40000" duration="10000" id="2">
          Some other data
        </Event>
        <Event presentationTime="60000" duration="10000" id="3" />
      </EventStream>
      <!-- ... -->
    </Period>

Each Event element generally has an assiociated "presentation time" (the initial time the event should be triggered at), some specific data and, sometimes, a duration.

The RxPlayer acts like a passthrough with the Event's data.
Its only role here is to send "streamEvent" events when a user goes into the time boundaries of such events. By catching that event, an application can also be notified when the user "exits" the same event's time boundaries:

// Do something when the user enters a time area linked to an Event element
rxPlayer.addEventListener("streamEvent", (evt) => {
  console.log("Beginning of an event:", evt);

  // Catch the moment when we are exiting the event.
  // Note that only Event elements with a duration (and thus an end) can
  // have an "exiting" notion.
  if (evt.end !== undefined) {
    evt.onExit = () => {
      console.log("End of the event:", evt);
    };
  }
});

The user could also "skip" an event, like when seeking after it. As an application, you may want to know when that happens.

For this reason, the RxPlayer also reports when an event has been "skipped" through a new "streamEventSkip" event:

rxPlayer.addEventListener("streamEventSkip", (evt) => {
   console.log("An event has been skipped.", evt);
});

Other types of events

For now, only DASH EventStream are handled by the RxPlayer but several types of events exist.
DASH also has in-band events that are in the media containers themselves like for example in "emsg" ISOBMFF boxes.
Other streaming protocol have their own event format(s) as well.

We want to support more of them in future versions, the available APIs here were also written with those in mind.
The management of "emsg" boxes, for example, is something we are today considering to add in our next versions.

More information on those APIs

Because the concept and APIs could seem complex we have added both those APIs to our API documentation and added a new tutorial about it.

We advise people wanting to integrate stream event management to first read the tutorial and to then come back to the API documentation for checking what are the arguments awaited.

The tutorial is here.

And here is the documentation of the new APIs:

• the streamEvent event for when an event is encountered
• the streamEventSkip event for when an event is "skipped".

We also export two new types (which can be imported via "rx-player/types"):

• IStreamEvent: the object sent with a streamEvent or streamEventSkip event
• IStreamEventData: the value of the data attribute of an IStreamEvent.

Those are documented here.

Applying track preferences to the current content

The two APIs for track selection

When it comes to track selection in the RxPlayer, we have two set of APIs:

• the "classic" track selection APIs - like getAudioTrack, setVideoTrack, getAvailableTextTracks - will tell you which tracks are available and allow you to choose a specific one from that list

• the track preferences APIs - like setPreferredAudioTrack - allow to define a general wanted choice. For example, we could indicate that the RxPlayer should choose by default a french audio track and/or english subtitles if those are available.

The APIs in the first set list the available tracks, so the application can then choose a precize one.

The APIs in the second set let the RxPlayer do the choice itself based on criterias. They are also used in some internal RxPlayer optimizations, like when pre-loading a future content - where the right track to download is for now "guessed" by the RxPlayer.

The problem with track preferences

The track preferences APIs have thus many advantages, but they have a limit: they only apply preferences for future contents and periods.

The content which is currently played and the Periods (for DASH) or sub-contents (for MetaPlaylist) that we played previously will still remain with their last set track configuration.

This is by design, we didn't want to change the current tracks after a setPreferred... call because this could seem unexpected.
So we are left in what we could refer as a "hole" in our API:

• We can define the track for the current Period (or sub-content) but this won't affect any other Periods
• We can define a preference for new Periods but this won't affect the current one or the one that have already been played.

What if we just want to set a new global preference, that we want to apply retroactively (to the current and already-played Periods)?

We could call both APIs, but we would still be left with the previous Periods in their previous track configurations.

This problem can be seen for example when you want to enable an "audio-only" mode (by disabling the video track) for multi-period contents:

// The old way

// disable the current track
rxPlayer.disableVideoTrack();

// Update the preferences for the next content / periods
rxPlayer.setPreferredVideoTracks([null]);

But here seeking back to a previous period is going to display the video track again. To work-around that, we have to call disableVideoTrack() again when seeking back to them.

A solution

We did many drafts on how we could improve our track selection APIs so that all possible use-cases are handled.

Among those, we thought of giving "more power" to the classic track selection APIs, so they could also change the track for other Periods.
However, we found that that approach would necessitate a very complex API. For features as simple as applying a global preference, we thought that the API should stay simple.

In the end, we decided to add an optional boolean as a second argument to those track preferences methods:

• setPreferredAudioTracks
• setPreferredTextTracks
• setPreferredVideoTracks

When set to true that preference will be applied retroactively.

For example, disabling the video track globally now can simply be done this way:

rxPlayer.setPreferredVideoTracks([null], true);

A new tutorial on track selection

As you can see, the RxPlayer allows applications to have a complex track selection management. However, this can come at the cost of ...

v3.20.1

Release v3.20.1 (2020-05-06)

Overview

This is a small release which mainly contains bug fixes.

It most-of-all fixes a small regression brought in our v3.20.0 release.
This issue could lead users to not be able to play multiple encrypted contents. It was though relatively rare to encounter, as it was only triggered in the following situations:

• when using Internet Explorer 11 and playing multiple encrypted contents

• when linking more than RxPlayer instance to the same media element - after calling rxPlayer.dispose() to dispose the last one - and playing encrypted contents in more than one of such instance

For applications which did not dispose and re-instanciate the RxPlayer on the same media element and did not target IE11, this issue should have had no impact. Likewise, applications not playing encrypted contents are not concerned by this issue.

We also added other fixes to this release not linked to this (or any) regression.

Regression when disposing of a MediaKeys instance

The main issue here (summarized in the overview) was due to the way we changed our DRM-related code to better support Safari.

When doing that, we updated the code attaching MediaKeys instances (the main interface involved in content decryption) to the media element to be adapted to both the regular and Safari way of doing so.

However, we poorly considered and tested the possiblity of setting a null MediaKeys, which is usually what you want to set when disposing of the current instance attached to the element.
When wanting to set a null MediaKeys, the RxPlayer just ignored it instead.

This was done in two situations:

• between each content when playing in Internet Explorer 11, to work-around an issue on this platform
• after calling the rxPlayer.dispose() method, to dispose of that resource (the MediaKeys instance)

This wrong behavior only became an issue when loading a new encrypted content on the same media element, where the same MediaKeys logic would unexpectedly throw.

Fixing this issue was just a matter of better handling MediaKeys disposal, which is now done.
We do not detect an issue on IE11 or on applications calling dispose anymore.

DRM issue for contents with multiple keys

This is a rare and subtle issue that could have impacted very sporadically some applications playing encrypted contents - with several keys per content.

As such, I'll try to be very descriptive on what the problem was and how it could have had an impact.

What's a MediaKeySession?

A MediaKeySession is the browser API allowing to communicate decryption keys to the Content Decryption Module integrated in the browser, which tasks is to decrypt the content.

We create a single MediaKeySession per initialization data received, which roughly means one per license needed to decrypt the content.
In most encountered cases where we only have one key per license, we can thus simplify this concept by saying that we could need to create as much MediaKeySession for a single content as the number of keys in that content.

The MediaKeySession cache

We try when possible to keep old MediaKeySessions in a cache even when other unrelated key(s) are needed. Doing so has several advantages, like speeding-up the loading of an already-played content, the ability to avoid license requests we already have done etc.

This cache will store a maximum of 50 MediaKeySessions at the same time.

The problem

It all becomes more complicated when we consider another cache maintained by the RxPlayer, listing which encryption information has already been encountered in the current content.

That second cache is needed for a similar (but different enough) reason than the MediaKeySession cache: to only trigger some logic when new encryption data is received, and avoid duplicating such logic in other cases, for example when the same data is received two times in a row.

Where the problem lies is that the MediaKeySession cache has a logic to evict old stored data but the second cache - let's call it the InitializationData cache - has none, and there is some dependency between them due to how the RxPlayer consider both.

To explain this, let's consider an example:
We're playing a content with 51 keys, each one associated to a different license. Each time we're encountering a media segment needing a new key - and only in that case - we would be creating a new MediaKeySession, then loading its license and so on.
When reaching data needing the 51st key, we will remove the MediaKeySession linked to the 1st key to make place for the one linked to the 51st.
If then we encounter data related to the 1st key again, the InitializationData cache would ignore it, as it has been already encountered. But here because its linked MediaKeySession is now closed, we're not able to play that data. We will be left waiting for a decryption key which will never come.

This case could be considered extremely rare because encountering that much encryption data on a single content is in itself extremely rare.
But this issue could arise in more subtle ways, such as when zapping between multiple contents and playing them again, as those 50 MediaKeySession do not need to be from the same content.

Note that this problem was only an issue when playing a content needing several licenses. In any other cases, even when switching a lot between contents, everything should have worked fine.

The solution

The behavior for both cache stays the same, we do not limit the InitializationData cache as it wouldn't make sense when considered alone.

To resolve that problem in an other way, the RxPlayer core logic now removes manually entries from the InitializationData cache when the corresponding MediaKeySession cache entries are evicted. As both caches are not handled in the same place, this is done thanks to an event.

Avoid hash collision in DRM logic

Another issue was close to the one we just described: some encryption data could be ignored because the RxPlayer wrongly thought it has already been encountered.

This is because we do not compare the data itself, we first hash it (through a custom hashing function) and then use that hash for comparison. If two different encryption data give the same hash, the RxPlayer will wrongly consider both to be the same.

This is however limited by the fact that such collision should be very rare, as far as we know this case has never been encountered. Still, the cost of comparing the encryption data itself is very low, so low that we considered this "optimization" of only checking the hash as an unnecessary risk.

Now the initialization data is still hashed, but when the same hash is encountered, we still compare the encryption data linked to that hash.
That way we still profit from the speed at which a hash can be used for lookup while still protecting against collisions with a very small cost.

Switching back to the preferred audio, video or text track in a Directfile content

There was a minor issue with directfile contents and track management where an audio, video or text track set by the user could be replaced by the most "preferred" one (according to the preferred{Audio,Text,Video}Tracks APIs).

This was because on directfile contents, we run a logic setting the optimal track everytime the list of available ones change. The definition of "optimal" there was just based on the preferred tracks, without considering that the user could already have selected a track through the set{Audio,Text,Video}Track APIs.

In the case where that track is still available, it would be a weird behavior to reset the choice because a new unrelated track has been removed or added in the process.

This is now fixed. Now when a track was manually selected through the set{Audio,Text,Video}Track APIs, we stay on that track as long as it stays available.

Changelog

Bug fixes

• eme: fix OTHER_ERROR issue arising when loading a new encrypted media when a previous since-disposed instance of the RxPlayer-played encrypted contents on the same media element
• eme: fix OTHER_ERROR issue arising on Internet Explorer 11 when playing more than one encrypted content
• eme: fix issue where more than 50 active MediaKeySessions at the same time could lead to infinite rebuffering when playing back a previous content with multiple encryption keys
• directfile: for directfile contents, don't reset to the preferred audio, text or video track when the track list changes
• eme: remove any possibility of collision in the storage of EME initialization data. The previous behavior could theorically lead some initialization data to be ignored.
• eme: fix typo which conflated an EME "internal-error" key status and an "output-restricted" one.

v3.20.0

Release v3.20.0 (2020-04-22)

Overview

The v3.20.0 brings new features and improvements:

• With the addition of the disableVideoTrack method, it's now possible to play in an "audio-only" mode

• We added a preferredVideoTracks RxPlayer option, as well as setPreferredVideoTracks and getPreferredVideoTracks methods to specify which video tracks you would prefer based on its codecs and accessibility status. It's also possible to use those APIs to begin playing without any video track active.

• We added (thanks to @mattiaspalmgren) a signInterpreted property to video track APIs which signal when a video track contain a sign language interpretation (according to the DASH-IF IOP).

• We improved the audio preferences APIs to here also be able to specify a preferred audio codec.

• We added new optimizations for supporting very large MPDs (DASH's manifest)

• The Safari-specific EME implementation (for playing contents with DRM) should now be finished. Previously, you might not have been able to play DRM-ized contents with Safari.

disableVideoTrack

The ability to play in an "audio-only" mode was something we wanted to provide for some time now.

However, removing or adding a video buffer during playback is not something well supported on web browsers. In more exact terms, all SourceBuffers had to be created before we can start to push media contents to either of them. For this reason adding the possibility to disable and re-enable the video track while playing is usually not doable.

Thankfully, we already added the ability to "reload" the content (and its associated "RELOADING" state) in the v3.6.0 (august 2018) release. By simply doing that when the user disable or re-enable the video track - and some other code changes - we were able to provide this feature (those "code changes" being actually a little more complicated than how it sounds as it's the reason why we waited so long :p).

Disabling the current video track is now possible by calling the disableVideoTrack method:

function enableAudioOnly() {
  player.disableVideoTrack();

  // If you don't want subtitles to display, which may or may
  // not be something you want, you can also do
  // player.disableTextTrack();
}

Please note that doing so might lead the RxPlayer to reload the content for a very short time. While doing so, the video element will usually be reset to black and multiple APIs won't be usable. You will know when the player is reloading or not by checking when it goes in and out of the "RELOADING" state.

You can then re-enable the video track by using the usual track switching method: getAvailableVideoTracks, getVideoTrack and setVideoTrack:

function quitAudioOnly() {
  // example by just setting the first track
  const tracks = player.getAvailableVideoTracks();
  if (tracks.length !== 0) {
    player.setVideoTrack(tracks[0].id);
  }
}

It is also possible to begin without a video track enabled through the new preferredVideoTracks option, which is described later in this release note.

On a related note, you have to consider that, just like disableTextTrack, disableVideoTrack will only disable the video track for the current Period.
If you're playing a multi-Period content and if you want to disable the video track for every Period, you could either call disableVideoTrack at each periodChange event, or you could use the new preferredVideoTracks APIs (which are documented in this same release note) before loading the content.

We added the possiblity to disable the video track in our demo page if you want a quick peek of how it can look like.

signInterpreted accessibility info

Before writing about the new video track preference APIs, we need to add a note on the new signInterpreted boolean property added on video track APIs.

This new property is set to true when the corresponding video track contains a sign language interpretation.
We can extract this information from a manifest, thanks to accessibility features available in some DASH contents (you can refer to the DASH-IF IOP for more info).

This boolean was added to the object returned by getAvailableVideoTracks and getVideoTrack:

/**
 * Returns only video tracks containing a sign language interpretation.
 */
function getAvailableSignInterpretedVideoTracks() {
  return player.getAvailableVideoTracks
    .filter(track => track.signInterpreted === true);
}

/**
 * Returns true if the current video track contains a sign language
 * interpretation.
 */
function isCurrentTrackSignInterpreted() {
  const videoTrack = player.getVideoTrack();

  // videoTrack can be `undefined` or `null` for different reasons (see doc)
  if (typeof videoTrack === "object") {
    // Note: signInterpreted can still be `undefined` - when we don't know
    return videoTrack.signInterpreted === true;
  }
  return false;
}

Big thanks to @mattiaspalmgren for implementing this feature.

preferredVideoTracks

It is now possible to set a video track preference through the following new APIs:

• the preferredVideoTracks player option, when instantiating the RxPlayer
• the setPreferredVideoTracks method, at any time after instantiation

Exactly like the related audio and text APIs, those two allow to tell the RxPlayer which video track it should initially choose, according to various constraints of your choosing.

For now, you can only set those constraints in function of:

• the wanted video codecs
• whether the track should contain a sign language interpretation or not

The syntax is well explained in those two method's documentation, but let's still get through simple examples so you can quickly grasp what they can do.

Let's say that you would prefer video tracks with sign language interpretation. To spice things up, we will also prefer if the track contains only Representation in the vp9 codec. Here is what we could do:

const player = new RxPlayer({
  // ...
  preferredVideoTracks : [
    // at best, have both sign language and only the VP9 codec
    {
      signInterpreted: true,
      codec: { test: /^vp9/, all: true /* Representations should all be vp9 */ },
    },

    // If not found or supported, just sign language is fine
    { signInterpreted: true },

    // If not available either, fallback on track without sign language, but
    // still with VP9
    { codec: { all: true, test: /^vp9/ } },

    // Note: If the last one was still not available, we will here still have a
    // video track, which do not respect any of the constraints set here.
  ],
})

You could also do it at any time after instantiation, thanks to the setPreferredVideoTracks method:

player.setPreferredVideoTracks([
  { signInterpreted: true, codec: { all: true, test: /^vp9/ } },
  { signInterpreted: true },
  { codec: { all: true, test: /^vp9/ } },
]);

Last but not least, it is also possible to indicate that you wish that no video track is active, by adding a null in this list of preference.

You could add it as the only element to that array, meaning you want to play in audio-only (without a video track):

// As usual either through the constructor
const player = new RxPlayer({
  // ...
  preferredVideoTracks: [null],
});

// or through the setPreferredVideoTracks method
player.setPreferredVideoTracks([null]);

You could also fallback to no video track if none your constraints are supported, by putting null at the end of that same array. For example, if you want the vp9 codec or no video track, you can do:

// Through the constructor
const player = new RxPlayer({
  // ...
  preferredVideoTracks: [
    { codec: { all: true, test: /^vp9/ } },
    null // if not found play without a video track
  ],
});

// or through the setPreferredVideoTracks method
player.setPreferredVideoTracks([
  { codec: { all: true, test: /^vp9/ } },
  null // if not found play without a video track
]);

You can retrieve the last set list of preferrences at any time through the new getPreferredVideoTracks method.

improved preferredAudioTracks

We also improved the audio track preferences APIs by allowing you to set constraints on the wanted codecs.

Both setPreferredAudioTracks and the preferredAudioTracks constructor option now can take a new codec property.

When set, this property allows to declare that you would prefer that an audio track contains either:

• only Representation with a given codec
• at least one Representation with a given codec

As you might only want to choose a preference based on the codec and not on the language anymore, we also made the previous properties (both language and audioDescription) optional.

Let's look at examples. Let's say that you would prefer audio tracks containing Dolby Digital Plus only (also known as "ec-3"), you could do:

player.setPreferredAudioTracks([
  {
    codec: {
      test: /ec-3/, // Check for Dolby Digital Plus
  ...

v3.19.0

Release v3.19.0 (2020-03-11)

Overview

The v3.19.0 brings several new improvements:

• the RxPlayer will now be able to consider multiple URL per segment (for DASH contents)

• we've made several improvements related to the performance of the RxPlayer's Manifest parsing logic

• we've added yet again another external tool (still experimental) called createMetaPlaylist which allows to easily concatenate multiple DASH/Smooth (or even MetaPlaylist!) contents into a single MetaPlaylist content

• the RxPlayer now exports more types (for TypeScript) related to audio/video/text track choices, with 6 new types

• we fixed several minor issues. Most being related to recently released experimental tools and features

Multiple URLs per segment

DASH's MPD can declare multiple URL per segment through the declaration of multiple BaseURL elements.

This allows for example to fallback to another CDN when the main one become unavailable. It may also sometimes be used when a first source might not have all media segments yet.

The strategy taken by the RxPlayer when encountering multiple URL linked to a segment is the following:

1. It will try to request the first URL (the one described by the first BaseURL element encountered in the MPD)
2. If it fails, it will immediately request the second URL
3. If it also fail, it will immediately request the third one, and so on
4. If all URL fails, the usual retry logic starts: depending on your configuration, the RxPlayer will wait a small delay - which will grow exponentially each time the batch of requests fails - and after that delay it will try again to request each URL, one immediately after another (it's actually a little smarter than that, only requests which have failed with an error considered as retry-able will be retried. e.g., an URL returning an HTTP 403 Forbidden will not be retried).
5. If all requests have failed and either:
   - we have done the maximum number of retry configured
   - or all URLs failed on an error that the RxPlayer judged as not retry-able (like a 403)
   Then, the player will stop on error and emit the usual PIPELINE_LOAD_ERROR error event

So basically, the usual retry logic is not done between different URL (as those will usually be linked to different CDNs). All requests are batched together and the retry logic is done when all this batch fails.

This was done as it seems the most logical way to go with such use cases.

Please open an issue if that behavior does not suit you. In that case we will see whether we could update it or at least add supplementary configuration options to adapt to it.

Manifest parsing improvements

Many changes in this release are linked to performance improvements on the RxPlayer's Manifest-parsing logic.

We tried to:

1. reduce the time taken by the RxPlayer to parse a Manifest
2. do that parsing logic less often
3. split that work in multiple parts, so we avoid blocking the main thread for too long

but why?

We were confronted to the need to improve on those points because we were having performance issues on some "embedded" devices (more specifically on devices with low memory such as low-end set-top-boxes and limited devices like the ChromeCast).

This was also linked to the fact that Canal+ Group - the company which started the RxPlayer development and which employs the main RxPlayer contributors - distributes live DASH contents with a pretty big timeshift window (usually set to 8 hours).

We think that huge Manifests like those is in fine a good thing for end users, because it means an increased possibility to seek in previously broadcasted contents - contents which may not be available elsewhere.
Moreover, we just feel that it should be part of our job to better handle large contents without issues - much like we consider it would be part of the job of a text editor to handle large enough text files while still being fully usable (not thinking about any editor in particular here, this is just a - perhaps loosy - analogy).

After this long introduction to why we did some improvements let's jump into what we did.

Doing updates through a "shorter" Manifest through manifestUpdateUrl

One of the most effective change we did to reduce Manifest parsing time (more specifically update time for live contents) was to allow the consideration of two version of the Manifest for the same content:

1. The regular full Manifest
2. A shortened version of it with a smaller timeshift window

This sadly means that, to profit from that improvement, you will surely need to update the packaging logic of your streams.
But rest assured: the content does not have to be re-encoded. Both Manifest stay linked to the same content, just the size of their timeshift window change (such as DASH's timeShiftBufferDepth attribute).

In the RxPlayer, the first version of the Manifest (the full one) will in most cases only be used to perform the initial load of the content. It also may need to be requested at other times, e.g. when the RxPlayer thinks that its internal Representation of it is completely out-of-sync, but those cases are very rare and will in most cases not happen.

For regular Manifest updates, if needed, the RxPlayer will only request the shorter version.
Because it will still have information about newly generated segments from it and because the RxPlayer can just "guess" which old segments are not available anymore, no information will be lost compared to when the RxPlayer use the full version instead - as it did until now and still do if no short version of the Manifest is provided.

This optimization has a huge effect on performance. To simplify, we could say that the more there is a size difference between the full and the short version, the more that improvement grows.

Still, it does not improve the initial load time, as it only impacts Manifest updates.

If you want to profit from that optimization, you will have to set in loadVideo the new transportOptions property called manifestUpdateUrl. Every properties from transportOptions are documented here

DASH "lazy-parsing"

The RxPlayer will now only parse some parts of a DASH MPD at the time it needs them.
For example, only Period that it plays and only AdaptationSets (tracks) that it needs will be parsed, the rest won't be until we switch to the corresponding Period or AdaptationSet.

We called this behavior "lazy-parsing" as it is inspired from the lazy evaluation concept we can find in programming languages like Haskell.

It is generally beneficial but has some setbacks:

• the RxPlayer still needs a lot of information at parsing time, meaning that in many cases the improvement will be minimal (for example, it might need to parse every SegmentTimeline element in the last Period to calculate the available "edge" of a live content)
• the code become less predictable when it comes to performance issues. We could now be in a case where a simple harmless function call can trigger that lazy-parsing and lead us to several seconds of parsing (in the worst cases)

We found that this new behavior provided the most improvements on contents with multiple Periods. The more Periods a DASH MPD contained, the better the improvement was.

Also, we saw visible improvements with both the initial loading time as well as Manifest updates. This also can have an effect on VoD contents (as long as those have multiple Periods, which is usually not the case).

As this feature is always on, there is no option linked to it. As long as you have the right version, you will profit from it.

Adaptive delay between Manifest updates

While investigating what we could do to improve our performance on embedded device. we at first looked at what other DASH video players were doing.

One interesting thing we were seeing is that the shaka-player, another open-source player, didn't have as much trouble playing Canal+ DASH streams than the RxPlayer had.
The curious thing is that, by looking at their MPD-parsing logic, we didn't see much differences to ours in terms of efficiency. The improvement had to be from elsewhere.

After several false lead on how they were doing it, we finally spotted the main reason: a reduction in the frequency of MPD updates. On the shaka-player, when parsing the MPD took too much time, the next updates could be postponed. For example, instead of refreshing our Manifest every 2 or 3 seconds (a low interval, but that's how our contents were), they were on some devices doing it every 12 seconds.

This was a pretty smart thing to do.
On some devices, the RxPlayer spent almost all its time parsing the MPD, leading to visible repercussion on content playback (which appeared jerky), the delay after which user interactions were taken into account and the stability of the device as a whole.

The RxPlayer has now integrated a similar logic, where long parsing time can raise the delay we will wait until we do the next Manifest update.

What's next?

All those improvements lead to a much better experience on low-end devices.
But we can still do better, most notably for the initial loading time which can still be long for some type of contents.

To improve in that regard, we are still doing experimentation.
For example we're looking if we can even improve the impact lazy-parsing can have and even the possibility of using webassembly on the Manifest-parsing logic.

This is however still in an experimental stage and we cannot tell with certitude that such features will be available in future versions.

createMetaplaylist tool

A new experimental tool, createMetaplaylist has bee...

v3.18.0

Release v3.18.0 (2020-01-30)

Overview

The v3.18.0 is a big release with its share of important new features:

• we improved our support of directfile contents by now making them compatible with most of our track management APIs

• we added two events: "seeking" and "seeked" which are triggered each time that a seek began and finished respectively, even when the seek was done to an already buffered part.

• we added a new possible transport, called "local". It allows to play downloaded DASH, Smooth or MetaPlaylist contents.

• we added a new external tool, the TextTrackRenderer, which allows to parse and display subtitles on top of a video element.

• we added another external tool, parseBifThumbnails, which parses BIF files (thumbnails containers)

• we improved our TTML subtitles support (complex style inter-inheritance, correct handling of rgba colors)

• we should now be easier to import from node (if you want to include this library in your server-side-rendering strategy)

To prepare a potential "v4", we also deprecated some APIs (see corresponding chapter below).

You might have guessed it, this is going to be a very long release note :-o

Deprecated APIs

We'll begin with the sad part. We have deprecated a small number of APIs, which means that they will remain supported until their deletion which will coincide with a future major release - probably the v4.0.0 (no date for that one for now, but we begin to think about it).

The following loadVideo options have been deprecated:

• supplementaryTextTracks, which can be replaced by using the new TextTrackRenderer tool (described below)
• supplementaryImageTracks, here you can use the new parseBifThumbnails tool instead.

Speaking of image APIs, the getImageTrackData method has also been deprecated as well as the imageTrackUpdate event. For now, thumbnails management has entirely been moved out of the RxPlayer. Most of those can still be replaced by doing a regular HTTP request and a call to the new parseBifThumbnails function.

Lastly, we deprecated the hideNativeSubtitles loadVideo option, due to it being close to useless since we allow for the "html" textTrackMode also as a loadVideo option.

Every information on why those were deprecated and how to replace them is available in the corresponding documentation page.

Improvement of directfile API support

Before this release, multiple RxPlayer properties, methods or events had no effect when playing a directfile content:

• getAvailable{Video,Audio,Text}Tracks always returned an empty array
• get{Video,Audio,Text}Track returned undefined
• set{Video,Audio,Text}Track had no effect
• the track change events available{Video,Audio,Text}TracksChange and {video,audio,text}TrackChange were never emitted
• setPreferred{Audio,Text}Tracks and the corresponding constructor options had no effect
• getPreferred{Audio,Text}Tracks returned an empty array

This is because [audio, video or text] track management when playing a content in a streaming protocol (like DASH or Smooth) is radically different than when playing directly a single file:

• In a streaming protocol, it is mostly a task performed by the RxPlayer
• In directfile mode, the RxPlayer has to rely on native browser APIs (through these native API, you can get and set native tracks, as well as receive events on track removal/addition/change).

The RxPlayer was mostly used with streaming protocols. As such, we didn't take the time yet to plug those native APIs to the RxPlayer's methods and events.
However, we realized that it was becoming more and more used in an hybrid mode, playing both type of contents (HLS being supported in directfile mode on Apple browsers is one of the reasons why).

That's why we've now connected the RxPlayer track APIs and events to these native track API.

From now, only on browsers that offer the right native APIs, you should be able to load directfile contents with several tracks and manage them. The player will now also take the preferred tracks settings into account.

Please note that if those native APIs are not present in the current browser, those features will still have no effect.
Sadly, we found that chrome and firefox (if the right flags were not enabled) only provided text tracks APIs, which means that switching audio or video tracks on those might still not work.

But to end on a good note, you should now be able to play HLS contents on Safari and benefit from RxPlayer track management.

seeking and seeked events

We added two new events "seeking" and "seeked", which are triggered respectively when the player begin a "seek" operation (it moves to another position) and when the player finished that operation.

Before those events, an application had to listen to the playerStateChange events to check when the state went in and out of the SEEKING state for the same kind of indication.
However, the player might not switch to the SEEKING state if a seek is done in an already-buffered part of the content. This is because, in those cases, the seek could be instantaneous (and as such, the player would stay in its PLAYING or PAUSED state).

local manifests: Playing downloaded contents

We've worked for some time on the ability to download DASH, Smooth or Metaplaylist contents and play them afterward.
This could be useful for example to be able to play those contents when offline.

We splitted that feature into two tasks:

1. a tool, independent to the RxPlayer, to download and store contents
2. the possiblity to play those downloaded contents through the RxPlayer

In this release we bring the second part, which means that you can now play contents stored locally (or anywhere, really), as long as you create the right custom manifest format, called the "local manifest".

More information on this feature can be found in the corresponding documentation.

Please note that we're still working on a default implementation of a content downloader to work with it.
It will take the form of a tool, and will rely by default on IndexedDB, a native browser API allowing to store large amount of data in the browser. If you want to take a look at what it will look like, you can follow the corresponding PR: #548.

TextTrackRenderer: Displaying synchronized subtitles on top of any content

The TextTrackRenderer is a new tool which allows to parse subtitles in various formats (srt, TTML, webVTT or SAMI) and display them on top of a video element (even when the content is not played through the RxPlayer).

This tool is completely independent of the RxPlayer and can be used alone. You can look at its documentation here.

You can use this tool as a replacement of the supplementaryTextTracks loadVideo option

parseBifThumbnails: parse BIF files (thumbnails containers)

Another tool, parseBifThumbnails, has also been added in this release.

This tool is a simple function which can parse BIF files.
BIF files are simple containers for video thumbnails, it is usually used to provide an indication of where the user will seek when he/she/it hovers the progress bar.

This same format is used at Canal+ as well as Netflix, Amazon Prime or Disney+ as far as we know.

The parseBifThumbnails is totally independent of the RxPlayer and can be imported without it. Its documentation has been added here.

This tool can replace most deprecated image APIs

Importing us from node

Before this release, there was an issue when importing the library in node, a strategy sometimes used on applications with server-side rendering (SSR).

This is because of two elements:

1. The RxPlayer depend on the window object in the global scope to initialize some browser-specific constants that it will need to use often.
   In node.js, the window object is not defined. As SSR rely on node.js to pre-calculate the first rendering step, it could break if the RxPlayer was in some way imported here.

2. Many exported files are written with ES6-styles imports and exports. This is usually not a problem as most module bundler process those without any issue but it became one when node.js was directly used without a bundling step before. Node.js usually rely on another style for imports and exports, the "commonJS" style.

We've decided to work around those issues, thus improving our compatibility with SSR.
From now, the RxPlayer finds out if it is in a node environnment, by detecting the lack of window in the global scope. In that case, none of the browser compatibility strategies will be applied and the RxPlayer will stay in a mode with reduced features.

For the second problem, we could theoretically fix it in several ways: generate CommonJS imports and exports instead or signal that we were "JavaScript modules" in some ways seem to be possible solutions.
However, as we didn't have time to check on those possibilities for now. we advise people in SSR scenarios to use the esm npm module for now, as a quick and easy solution.

Changelog

Features

• directfile: support most audio tracks API when playing a directfile content
• directfile: support most text tracks API when playing a directfile content
• directfile: support most video tracks API when playing a directfile content
• api: add seeking and seeked events which anounce the beginning and end of a seek, e...