Adaptive bitrate streaming, explained

The ladder

An ABR ladder is a list of renditions of the same video, each at a lower resolution and bitrate than the last. Every rendition is cut into segments at exactly the same timestamps, with a keyframe at the start of each. So the player can play segment 12 from the 1080p rendition and segment 13 from the 480p one. The decoder will not notice. The HLS guide covers how the playlist describes the ladder; the transcoding guide covers how the aligned renditions are made.

The spacing of the rungs is a design choice. With too few, the player has nowhere to go when bandwidth drops a little. With too many, transcoding and storage costs climb for little visible gain. A common rule of thumb is that neighbouring rungs should differ in bitrate by about 1.5 to 2 times. Then each step down is a real saving and each step up a real gain. Apple publishes a reference H.264 ladder in its authoring specification. It starts at 416×234 and ends at 1920×1080, with bitrates rising from a few hundred kilobits per second to several megabits.

The VideoBB ladder

Resolutions are the renditions VideoBB produces; the bitrate column is a rule-of-thumb range for H.264 at each height rather than a fixed platform figure, since the transcoder adapts to content.

What the player measures

The first input is throughput: how many bits arrived in how many seconds while the last segment downloaded. That is a noisy number. A single segment can arrive fast because it hit a warm cache. It can arrive slowly because a background download started. So players smooth the number, often with a weighted average that favours recent samples, and treat the estimate with suspicion.

The second input is the buffer: how many seconds of video are already downloaded and waiting to play. A full buffer is insurance. If it holds thirty seconds, a bad bandwidth guess costs nothing, because there is time to correct it. If it holds three, the same guess causes a stall. The most robust algorithms weight buffer occupancy heavily, and some, such as BOLA (used in dash.js and available in hls.js), are driven almost entirely by it.

Other inputs matter at the edges. The first is the size of the viewport: there is no point fetching 1080p into a 320-pixel-wide embed. Whether the tab is visible also counts. So do the device's decoding power and the CODECS strings in the playlist. Together they limit which rungs are even candidates.

The decision loop

Every player implements some version of this cycle for each segment.

1. Estimate bandwidth

   Combine recent segment download rates into a conservative estimate, usually discounting it by a safety factor so that the chosen rendition leaves headroom.

2. Check the buffer

   If the buffer is below a low-water mark, prefer a lower rung regardless of the estimate; if it is above a high-water mark, a higher rung is affordable even on a shaky estimate.

3. Filter the candidates

   Remove rungs whose resolution exceeds the viewport by a large margin, whose codec the device cannot decode, or whose BANDWIDTH exceeds the discounted estimate.

4. Apply hysteresis

   Require a clearer margin to step up than to step down, and limit how often switches happen. Then the picture does not flip between two rungs on a connection that hovers near a boundary.

5. Fetch and re-measure

   Request the next segment from the chosen rendition, record how long it took, and start again.

Why this is harder than it sounds

Bandwidth is a property of the next few seconds, and the player only has evidence about the last few. Mobile networks change by an order of magnitude when a handset switches cells. Home connections are shared with other devices that start and stop their own traffic. A CDN edge that has the next segment cached delivers it far faster than one that must fetch it from the origin. The CDN guide explains why. The player cannot tell which kind of edge it is talking to.

The failure modes are lopsided. A stall is far more annoying than a few seconds at 480p. So good algorithms are pessimists: they step down quickly and step up slowly. The cost of that pessimism is small. A viewer on an excellent connection may sit at 720p for the first segment or two. Then the player trusts its own measurements enough to move to 1080p.

Three families of algorithm

What you can do at the source

The player's job is easier when the ladder is well made, and the ladder is only as good as the upload.

• Upload the highest quality source you have. Every rung is transcoded from it, and a compressed source caps every rendition; the blurry after upload guide shows the effect.
• Keep the frame rate the camera recorded. Mixed or variable frame rates complicate segment alignment; the frame rate guide explains the options.
• Do not pre-shrink to 720p in the hope of a faster upload. The ladder would then stop at 720p, and viewers on big screens would notice. Use resumable uploads instead and let the platform handle size.
• Leave the quality setting on automatic in the embedded player. Forcing 1080p defeats the mechanism that prevents stalls.