Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update how MINIMUM_SEGMENT_SIZE is used in the SegmentInventory #1424

Merged
merged 2 commits into from
Apr 9, 2024

Conversation

peaBerberian
Copy link
Collaborator

Fix a rebuffering issue seen since a fix proposed by #1416

The MINIMUM_SEGMENT_SIZE

Basically the MINIMUM_SEGMENT_SIZE value, which defined a minimum time duration a media segment can have to be considered by the RxPlayer (mainly to prevent some rounding errors in the RxPlayer, though this is far from the only mechanism for that) was also sometimes relied on for "chunks", which is the term the RxPlayer use for the units of media data that will be transmitted to media buffers.

The latter (chunks) can be smaller than the former (segments) in (and only in for now) low-latency scenarios.

Problem with chunks

"Chunks" can be much much smaller than "segments" (the unit of media data we actually load from the network through a single HTTP(S) request) and it is very possible for a chunk to be smaller than 5 milliseconds, the original value (before this PR) for MINIMUM_SEGMENT_SIZE.

Before the fix proposed by #1416, multiple chunks were generally grouped together before being pushed at once to a buffer (though still as a smaller group than the whole segment) and it seemed those groups always were bigger than MINIMUM_SEGMENT_SIZE.

However, the #1416 fix now separates every one of those chunks (this is actually performed at its lowest level, by always considering carefully a moof and an mdat box for ISOBMFF-compatible containers) which leads to frequently obtaining chunks smaller than the MINIMUM_SEGMENT_SIZE.

In turn, due to some processing of the SegmentInventory (which lists the chunks believed to be currently buffered by the browser), the RxPlayer would then believe that some of those chunks have been garbage-collected.

For some reasons not yet explored (this gets messy very quickly from there sadly), the RxPlayer first seems to think that it cannot buffer those segments, then any of the following segment, and then enters a rebuffering status for around 20 seconds, before restarting playback as if nothing happened.

This would merit more investigation, but for now I mostly spent time fixing the original MINIMUM_SEGMENT_SIZE issue.

The fixes

There's actually three fixes. It gets pretty technical so I guess only people knowing how the RxPlayer's SegmentInventory works could decipher this (even I, have issues explaining what I did :p):

  1. Before, we would have considered in the SegmentInventory that if a chunk only had less than MINIMUM_SEGMENT_SIZE (5 milliseconds originally) from its starting time to the end of the currently-considered contiguous buffered time range (so basically range.end - chunk.start < MINIMUM_SEGMENT_SIZE), then it was considered as part of the next range, unless there's no next range in which case it was considered garbage-collected (see fix 3).

    The fix here is to also check if there's more than MINIMUM_SEGMENT_SIZE after that same range's end in that scenario.

  2. MINIMUM_SEGMENT_SIZE has been updated from 5 milliseconds to 1 millisecond. This has risks but there's risks in both sides and I here noticed an issue due to its value being here "too high", so I chose to lower it for resiliency.

  3. We previously considered in the SegmentInventory that chunks that were not associated to any buffered time range (due to the fact that we already found the last chunk associated to the last buffered time range) must have since been garbage-collected.

    It theoretically makes sense, yet we didn't perform a check on their insertionTs, a property storing a timestamp of when the chunk has been pushed, and which protects against race condition where the browser wouldn't have yet updated its buffered time range, despite the chunk having been pushed.

    Now we do check that at least SEGMENT_SYNCHRONIZATION_DELAY milliseconds have elapsed since a chunk's insertionTs before defining if it has been garbage-collected or not.

Note that in the diff, thisSegment actually refers to a chunk and not a segment (it would have been too easy if we had the right variable name)

Fix a rebuffering issue seen since a fix proposed by #1416

The `MINIMUM_SEGMENT_SIZE`
--------------------------

Basically the `MINIMUM_SEGMENT_SIZE` value, which defined a minimum
time range a media segment should have to be considered (mainly to
prevent some rounding errors in the RxPlayer, though this is far from
the only mechanism for that) was also relied on for "chunks", which is
the term the RxPlayer use for the units of media data that will
be transmitted to media buffers, that can be smaller than segments in
(and only in for now) low-latency scenarios.

Problem with chunks
-------------------

However, such "chunks" can be much much smaller than "segments" (the
unit of media data we actually load from the network through a single
HTTP(S) request) and it was very possible that a chunk was smaller than
5 milliseconds, the previous value for `MINIMUM_SEGMENT_SIZE`.

Before the fix proposed by #1416, multiple chunks where the large
majority of time grouped together before being pushed at once to a
buffer (though still as a smaller group than the whole segment)
and it seemed those groups always were bigger than
`MINIMUM_SEGMENT_SIZE`.

However, the #1416 fix now separates every one of those chunks
(this is actually performed at its lowest level, by always considering
carefully a `moof` and an `mdat` box for ISOBMFF-compatible containers)
which leads to frequently obtaining chunks smaller than the
`MINIMUM_SEGMENT_SIZE`.

In turn, due to some processing of the `SegmentInventory` (which lists
the chunks believed to be currently buffered by the browser), the RxPlayer
would then believe that some of those chunks have been garbage-collected.

For some reasons not yet explored (this gets messy very quickly from there
sadly), the RxPlayer first seems to think that it cannot buffer
those segments, then any of the following segment, and then enters a
rebuffering status for around 20 seconds, before restarting playback as if
nothing happened.

This would merit more investigation, but for now I mostly spent time
fixing the original `MINIMUM_SEGMENT_SIZE` issue.

The fixes
---------

There's actually three fixes. It gets pretty technical so I guess only
people knowing how the RxPlayer's `SegmentInventory` works could
decipher this (even I, have issues explaining what I did :p):

  1. Before, we would have considered in the `SegmentInventory` that if
     a chunk only had less than `MINIMUM_SEGMENT_SIZE` (5 milliseconds
     originally) from its starting time to the end of the
     currently-considered contiguous buffered time range (so basically
     `range.end - chunk.start < MINIMUM_SEGMENT_SIZE`), then it was
     considered as part of the next range, unless there's no next range
     in which case it was considered garbage-collected (see fix 3).

     The fix here is to also check if there's more than
     `MINIMUM_SEGMENT_SIZE` after that same range's end in that scenario.

  2. `MINIMUM_SEGMENT_SIZE` has been updated from 5 milliseconds to 1
     millisecond. This has risks but there's risks in both sides and I
     here noticed an issue due to its value being here "too high", so I
     chose to lower it for resiliency.

  3. We previously considered in the `SegmentInventory` that chunks
     that were not associated to any buffered time range (due to the
     fact that we already found the last chunk associated to the last
     buffered time range) must have since been garbage-collected.

     It theoretically makes sense, yet we didn't perform a check on
     their `insertionTs`, a property storing a timestamp of when the chunk
     has been pushed, and which protects against race condition where the
     browser wouldn't have yet updated its buffered time range, despite
     the chunk having been pushed.

     Now we do check that at least `SEGMENT_SYNCHRONIZATION_DELAY`
     milliseconds have elapsed since a chunk's `insertionTs` before
     defining if it has been garbage-collected or not.
@peaBerberian peaBerberian added bug This is an RxPlayer issue (unexpected result when comparing to the API) low-latency Relative to low-latency (live playback close to the live edge) labels Apr 3, 2024
@peaBerberian peaBerberian added this to the 4.1.0 milestone Apr 3, 2024
`SI: synchronizing ${bufferType ?? "unknown"} buffered ranges:`,
prettyPrintedRanges,
);
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a log so it becomes easier to determine if a SegmentInventory's synchronization result (logged at the end) do seem to correspond to its SourceBuffer's buffered property at that time (logged here)

// Ambiguous, but `thisSegment` has more chance to be part of the
// next range than the current one
break;
}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I broke the while condition into separate if -> break for readability purposes (yes your heard right, I relied on the much-hated break statements for readability purposes :p)

@peaBerberian peaBerberian added DASH Relative to the DASH streaming protocol Priority: 0 (Very high) This issue or PR has a very high priority. Efforts should be concentrated on it first. labels Apr 9, 2024
}
if (
nextRangeStart !== undefined &&
rangeEnd - thisSegmentStart >= thisSegmentEnd - nextRangeStart
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it be the opposite of the previous condition ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right, I just replaced >= by <

@peaBerberian peaBerberian merged commit 79e116b into dev Apr 9, 2024
5 of 6 checks passed
@peaBerberian peaBerberian mentioned this pull request Jun 13, 2024
@peaBerberian peaBerberian deleted the fix/low-latency-minimum-segment-size branch July 26, 2024 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This is an RxPlayer issue (unexpected result when comparing to the API) DASH Relative to the DASH streaming protocol low-latency Relative to low-latency (live playback close to the live edge) Priority: 0 (Very high) This issue or PR has a very high priority. Efforts should be concentrated on it first.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants