Video Compression

The difference between two adjacent frames in a video could either be large or the same, hence a possible way to save on the number of bits to be sent could be to just send the delta to encode the differences. High spatial details and complex textures would definitely take more bits as the delta would be greater but even in the worst case we are just sending at most bits equal to the maximum size of a frame, and in the expected case we save bits sent over network.

The compression mode greatly depends on the use case, as the use case dictates the encoding scheme and tolerable loss in compression. Major use cases are:

1. Archival — storing files to archive. Since files are to be stored on disk or network storage, they should be space sensitive.
2. Streaming — like Netflix or other content providers. Here videos are streamed on demand over HTTP and may have to maintain Adaptive Streaming.
3. Live Streaming — instead of video-on-demand, video is consumed in real-time (FaceTime or live sports streaming). Here time is of the essence and encoding should be very fast without any prior knowledge.
4. Encoding for devices — transferring/copying files from one storage type to another. Here compression should be as lossless as possible.

---

Constant QP (CQP)

Quantization is a lossy technique to compress frames at a block level (usually 8×8). The quantization parameter controls the amount of compression. Since quantization values used are mostly static, the resulting bitrate is strongly dependent on these values — hence it should be avoided in most cases.

It can be highly useful if we are aware of the frame-by-frame encoding, as good quantization parameters could be chosen for compression. This is the reason why Netflix uses fixed-QP encoding for its per-shot encoding optimization.

---

Average Bitrate (ABR)

This declares the target bitrate ahead of time without consideration for data ahead, and hence is not suitable for archival compression. It throttles after reaching the target bitrate. It is a Variable Bitrate mode with variation within short segments.

Constant Bitrate (CBR) on the other hand forces the encoder to always use a certain bitrate. Even if there are not many bits to encode, it still maintains its constant bitrate, which leads to bandwidth wastage. It could be useful in the streaming use case but is not suitable for archival.

---

2-Pass Average Bitrate (2-Pass ABR)

Makes use of 2 passes to ensure output quality is best under a certain bitrate constraint.

The encoder is "dumb" when used in 1-pass mode and cannot predict the complexity and motion of a future frame, which eventually leads to blocking and other artifacts, unless you give it a bitrate high enough to eliminate these issues. Not so when using 2-pass mode: the first pass stores information of every frame to a log file, and when the second pass starts, the encoder reads that log file with the frame/motion/complexity information for each frame and bases its decisions on how much bitrate to allocate per frame.

From the community:

"Lossy encoders (like x264 and x265) have to decide how much information to throw away in each frame. This is represented by the quantization parameter (QP). The larger the QP for a given frame, the more information is thrown away.

Constant Rate Factor (CRF) tries to address the inefficiency in CQP by increasing the QP in high-motion frames and decreasing it in low-motion frames. This works very well in most scenarios. CRF is a 'sort-of' constant quality rate control. 2-pass also tries to achieve constant perceptual quality, but it does so with an additional constraint: average bitrate."

This mode works well in most cases except when time is of the essence (live streaming).

---

Constant Quality (CQ) / Constant Rate Factor (CRF)

One of the most general modes to obtain optimal quality under most cases — except when there are strict constraints on file size and/or bitrate. Under this you can specify a CRF without consideration for file size.

---

Constrained Encoding (VBV)

Video Buffering Verifier constrains bitrate to a certain maximum. It can be used with CRF or 2-Pass ABR. It works well for streaming under bandwidth constraints, live streaming, and Video-on-Demand streaming — but is not a good choice for archival.

---

Summary

---

Additional Resources

• Toward A Practical Perceptual Video Quality Metric