Video Compression Basics: How Pixels Become Streamable Bits

Raw video is enormous. A single second of uncompressed 1080p at 30 fps and 8-bit 4:4:4 color runs to roughly 1080 × 1920 × 3 bytes × 30 ≈ 180 MB/s. No network or disk wants to move that, so every streaming workflow begins with compression. This article covers the core ideas that every later topic in this series — codecs, transcoding, packaging — builds on.

Lossy vs lossless

There are two broad families of compression:

• Lossless reconstructs the original bit-for-bit (think PNG or FLAC). It exploits statistical redundancy only, so it rarely beats ~2:1 on natural video.
• Lossy throws away information the human visual system is unlikely to notice, reaching ratios of 100:1 or more. Every mainstream video codec — H.264, HEVC, VP9, AV1 — is lossy.

The art of a lossy codec is deciding what to discard. It leans on two kinds of redundancy and on the limits of human perception.

Spatial redundancy (intra-frame)

Within a single frame, neighboring pixels are highly correlated — a patch of sky is nearly one color. Codecs divide each frame into blocks (macroblocks in H.264, coding tree units in HEVC) and apply a frequency transform, usually a DCT (Discrete Cosine Transform). The transform concentrates energy into a few low-frequency coefficients; high-frequency detail is then quantized — divided down and rounded — which is where most of the loss (and the savings) happens. This is conceptually the same trick JPEG uses for still images.

Temporal redundancy (inter-frame)

Consecutive frames are usually almost identical. Rather than re-encode each one, the codec stores motion vectors that say "this block moved 4 pixels right from the previous frame" plus a small residual for whatever changed. This motion compensation is where video compression earns most of its advantage over still-image compression.

I, P, and B frames

That leads directly to the three frame types:

• I-frame (Intra) — coded standalone, using spatial compression only. A decoder can start here without any other frame. Also called a keyframe.
• P-frame (Predicted) — references one or more earlier frames via motion compensation. Much smaller than an I-frame.
• B-frame (Bi-directional) — references both earlier and later frames, yielding the best compression but adding decode complexity and latency.

GOP structure

A GOP (Group of Pictures) is the repeating pattern between I-frames, e.g. IBBPBBPBBP. The GOP length matters for streaming:

• A closed GOP does not reference frames outside itself, so a segment can be decoded independently — essential for adaptive streaming, where players switch renditions at segment boundaries.
• GOP length is typically aligned to the segment duration (e.g. a 2-second GOP for 2-second segments) so every segment begins with a keyframe.

A common encoder setting is a keyframe interval matching the segment size, for example -g 48 for 2-second segments at 24 fps.

Chroma subsampling (4:2:0)

Human eyes are far more sensitive to brightness (luma) than to color (chroma). Video exploits this by storing color at reduced resolution:

• 4:4:4 — full color resolution; used in mastering and graphics.
• 4:2:2 — chroma horizontally halved; common in professional capture.
• 4:2:0 — chroma halved in both dimensions; the standard for streaming H.264, HEVC, VP9, and AV1.

Moving from 4:4:4 to 4:2:0 discards three-quarters of the color samples for a barely perceptible quality drop — a free 50% reduction in raw data before the codec even starts. This is why nearly all delivery video is 4:2:0, signaled in a codec string as a profile/format like yuv420p.

Bitrate vs quality, and rate control

Bitrate is how many bits per second you spend; quality is how good the result looks. They correlate but are not the same — a well-tuned encoder at 4 Mbps can beat a sloppy one at 6 Mbps. How an encoder spends its bit budget is called rate control:

• CBR (Constant Bitrate) holds a steady rate, wasting bits on easy scenes and starving hard ones. Useful when the delivery channel has a fixed capacity, e.g. broadcast or some live contribution.
• VBR (Variable Bitrate) lets the rate rise on complex scenes and fall on simple ones, giving better quality for the same average size. Standard for VOD.
• CRF (Constant Rate Factor) targets a constant perceptual quality and lets the bitrate float wherever it needs to. This is the go-to for archival and mezzanine encodes.

A typical x264/x265 CRF encode looks like:

ffmpeg -i input.mov -c:v libx264 -crf 20 -preset slow -pix_fmt yuv420p out.mp4

Lower CRF means higher quality and bigger files; 18–23 is the usual range for H.264. For adaptive streaming you usually pair VBR with a -maxrate and -bufsize cap so no rendition exceeds the bandwidth its bitrate ladder rung promises.

How this maps to streaming

Every adaptive-streaming decision traces back to these fundamentals: segment boundaries land on closed-GOP keyframes, the bitrate ladder is a set of quality/bitrate trade-offs, and codec choice (next article) determines how efficiently the encoder can exploit spatial and temporal redundancy.

To see these properties on a real file — resolution, chroma format, codec profile, GOP hints — run it through the media info inspector. When you are ready to dig into specific codecs, continue with Video Codecs Explained, and to learn how files get re-encoded into a streaming ladder see Transcoding Explained.