Signal Compression

Learning Objectives

• Distinguish lossless and lossy compression and explain the trade-off each makes
• Describe how Huffman coding and run-length encoding achieve lossless compression
• Explain how the Discrete Cosine Transform enables lossy compression by concentrating energy
• Describe the three-stage compression pipeline: analysis, transformation, quantization
• Explain psychoacoustic modeling and why it improves perceived audio quality at a given bitrate
• Identify appropriate compression approaches for a given application (medical data vs. streaming audio)

Quick Answer

Signal compression reduces the number of bits needed to represent a signal, trading off between file size and fidelity. Lossless compression (Huffman coding, run-length encoding, arithmetic coding) removes only statistical redundancy, so the original signal can be reconstructed exactly — essential for medical data, text, and program files. Lossy compression (as used in MP3, JPEG, H.264) additionally discards information the human eye or ear is unlikely to notice, achieving far higher compression ratios by transforming the signal (commonly with a Discrete Cosine Transform or wavelet transform) and then quantizing — reducing precision on — the less perceptually important components. The right choice depends entirely on whether perfect reconstruction is required or a good-enough approximation is acceptable in exchange for a much smaller file.

Why Compress a Signal at All?

Raw digital signals are large: a single second of uncompressed CD-quality stereo audio is about 1.4 megabits; an uncompressed HD video frame can be several megabytes. Storing or transmitting signals in this raw form quickly becomes expensive in bandwidth and storage. Compression exploits the fact that most real signals contain redundancy — repeated patterns, predictable structure, or components the intended receiver (a human eye or ear) can't actually perceive — and re-encodes the signal using fewer bits without giving up everything that matters.

Lossless Compression: Exact Recovery

Lossless methods guarantee the decompressed signal is bit-for-bit identical to the original. They work purely by exploiting statistical redundancy in the data, not by discarding any information:

• Run-Length Encoding (RLE) — replaces long runs of a repeated value with a single value-count pair. Highly effective on signals with large uniform stretches (like a fax image or a simple graphic), nearly useless on noisy or highly variable signals.
• Huffman coding — assigns shorter binary codes to more frequent values and longer codes to rarer ones, based on their measured frequency of occurrence. This guarantees an optimal prefix code for a given symbol frequency distribution.
• Arithmetic coding — encodes an entire sequence as a single fractional number within a continuously subdivided range, generally achieving slightly better compression than Huffman coding by not being restricted to whole-bit code lengths per symbol.

Lossless compression is mandatory whenever any change to the data is unacceptable — medical images used for diagnosis, financial records, program executables, and text documents.

Lossy Compression: Trading Fidelity for Size

Lossy methods permanently discard some information, chosen specifically to be information a human is unlikely to notice missing, achieving far higher compression ratios than lossless methods can:

• Discrete Cosine Transform (DCT) — transforms a block of signal or image data into the frequency domain, concentrating most of the signal's energy into a small number of low-frequency coefficients. The many remaining high-frequency coefficients, which carry little perceptual information, can then be aggressively quantized or discarded. This is the mathematical core of JPEG image compression and MP3 audio compression.
• Wavelet transform — like the DCT, re-expresses a signal in a transformed domain, but using basis functions that are localized in both time and frequency (rather than pure frequency alone). This gives a multi-resolution view — capturing both broad trends and sharp local details — and underlies formats like JPEG2000.

The Three-Stage Compression Pipeline

Most practical lossy compression pipelines follow the same three stages:

1. Analysis — break the input signal into manageable blocks or segments and identify structure, often using a Fourier or related transform to reveal which frequency components carry the most energy.
2. Transformation — apply a mathematical transform (DCT, wavelet) that concentrates the signal's energy into fewer, more significant coefficients, separating perceptually important content from less important content.
3. Quantization — reduce the numerical precision of the transformed coefficients, more aggressively for the less important (typically high-frequency) ones. This is the step where information is actually discarded, and it's the primary control knob for the trade-off between file size and quality.

Worked Example: Why DCT Coefficients Compress Well

Imagine an 8×8 block of a photograph that is mostly a smooth blue sky with a little texture. Applying a DCT to that block produces one large coefficient (representing the average brightness — the DC term) and a handful of small coefficients representing subtle low-frequency variation, with most of the remaining high-frequency coefficients close to zero. JPEG's quantization table then divides the high-frequency coefficients by larger numbers than the low-frequency ones, rounding many of them to exactly zero — those zeros compress extremely well with a subsequent lossless stage (typically Huffman coding). This is exactly why a smooth sky compresses far more than a busy, detailed texture: the DCT concentrates the sky's energy into very few coefficients, while a detailed texture spreads energy across many coefficients that all need to be preserved.

Psychoacoustic Modeling

Audio codecs like MP3 go a step further than generic transform-based compression by modeling human hearing directly. Psychoacoustic modeling identifies frequencies that are masked — made inaudible — by louder nearby frequencies or by the ear's reduced sensitivity at certain frequency ranges, and allocates fewer bits (or zero bits) to those masked components. This lets MP3 discard a large fraction of the raw audio data while remaining perceptually close to transparent for most listeners, because the discarded information was largely inaudible anyway.

Real-World Applications

• Audio and video streaming — MP3, AAC, and H.264/H.265 use lossy compression to deliver acceptable quality within limited bandwidth.
• Data transmission — compressing sensor telemetry or network payloads reduces bandwidth usage and transmission cost.
• Storage optimization — compressed formats reduce the disk space needed for large media libraries or datasets.
• Medical imaging — DICOM images typically use lossless compression, since diagnostic accuracy cannot be compromised by discarded detail.

Key Terms

Term	Definition	Related Concept
Lossless compression	Compression allowing exact reconstruction of the original data	RLE, Huffman coding
Lossy compression	Compression that permanently discards some information for higher compression ratios	DCT, quantization
Run-Length Encoding	Replaces repeated runs of the same value with a value-count pair	Lossless compression
Huffman coding	Assigns shorter codes to more frequent symbols based on their statistical frequency	Entropy coding
Discrete Cosine Transform (DCT)	Transform that concentrates a signal's energy into few low-frequency coefficients	JPEG, MP3
Quantization	Reducing the numerical precision of transformed coefficients to reduce data size	Lossy compression stage
Psychoacoustic modeling	Identifying inaudible components of an audio signal to remove without perceptible quality loss	Auditory masking, MP3
Compression ratio	The ratio of original data size to compressed data size	Trade-off with quality

Common Mistakes

Misconception: Lossy compression always produces noticeably worse quality than lossless compression. Why it's wrong: Well-designed lossy compression (like a properly encoded MP3 or JPEG) discards information specifically chosen to be imperceptible to human senses, so at reasonable bitrates the perceived quality difference can be negligible even though the file is dramatically smaller. Correct understanding: Lossy compression quality depends heavily on the bitrate and encoder quality chosen; at low enough bitrates artifacts become audible or visible, but at appropriate settings the perceptual loss is often imperceptible for the intended use.

---

Misconception: The DCT and the Fourier transform are unrelated techniques. Why it's wrong: The DCT is closely related to the Fourier transform — it can be derived from a Fourier transform of a symmetrically extended (mirrored) version of the signal, using only real-valued cosine basis functions instead of complex exponentials. Correct understanding: The DCT inherits the frequency-domain energy-concentration property of the Fourier transform, and is preferred in compression specifically because it tends to produce fewer significant coefficients for typical real-world signals than a full Fourier transform.

---

Misconception: Applying compression multiple times in sequence (compressing an already-compressed file again) keeps reducing file size proportionally. Why it's wrong: After the first lossy compression pass, most of the removable redundancy and perceptually unimportant information is already gone; a second pass on already-compressed data tends to add further quality loss (generation loss) with little to no additional size reduction, and can even increase size for already near-random compressed data. Correct understanding: Compression should generally be applied once, from the highest-quality original source available, rather than repeatedly re-compressing already-compressed material.

Comparison and Connections

Aspect	Lossless Compression	Lossy Compression
Reconstruction	Exact, bit-for-bit	Approximate
Typical compression ratio	Modest (2:1 to 4:1 typical)	High (10:1 or more common)
Underlying technique	Entropy coding (Huffman, arithmetic, RLE)	Transform (DCT, wavelet) + quantization + entropy coding
Used for	Medical data, text, executables	Streaming audio/video, web images
Repeatable without further loss?	Yes	No — each pass discards more data

Practice Questions

Recall

1. Name three lossless compression algorithms. Answer guidance: Run-Length Encoding (RLE), Huffman coding, and arithmetic coding.

2. What are the three stages of a typical lossy compression pipeline? Answer guidance: Analysis (identify structure/redundancy), transformation (e.g., DCT or wavelet transform to concentrate energy), and quantization (reduce precision of less important coefficients).

Understanding

3. Explain why the DCT is effective for image compression, using the concept of energy concentration. Answer guidance: The DCT re-expresses a block of pixel data as a sum of cosine basis functions of increasing frequency. For typical smooth image regions, most of the signal's energy ends up concentrated in a few low-frequency coefficients, leaving many high-frequency coefficients small or near zero — these can be discarded or heavily quantized with little visible effect.

4. Why does psychoacoustic modeling let MP3 achieve much higher compression than a generic lossless audio compressor? Answer guidance: Psychoacoustic modeling specifically identifies audio content that is inaudible to human hearing (due to frequency masking or limited sensitivity at certain frequencies) and removes it entirely, rather than only removing statistical redundancy. This targets perceptual, not just statistical, redundancy, enabling much greater size reduction while remaining close to perceptually transparent.

Application

5. A hospital needs to store MRI scans for long-term diagnostic use. Which compression approach should be used, and why? Answer guidance: Lossless compression, because any information loss could obscure or alter diagnostically relevant detail, and legal/medical standards typically require exact reconstruction of the original imaging data.

6. A video streaming service needs to deliver content over limited mobile bandwidth while keeping visual quality acceptable. Which compression approach is appropriate, and what technique underlies it? Answer guidance: Lossy compression (e.g., H.264/H.265), which uses DCT-like transforms and quantization on video frames (plus motion compensation between frames) to achieve the high compression ratios needed for streaming over limited bandwidth.

Analysis

7. Compare the consequences of choosing too aggressive a quantization step in a lossy compression pipeline versus too conservative a quantization step. Answer guidance: Too aggressive quantization removes too many coefficients, producing visible/audible artifacts (blockiness in images, "swishy" artifacts in audio) but yields very small file sizes. Too conservative quantization preserves quality closely but yields disappointing compression ratios, undermining the purpose of using lossy compression in the first place. Codecs aim to quantize just enough that discarded information is imperceptible.

8. A student claims that because JPEG and MP3 both use the DCT, compressing an image with an audio codec's settings (or vice versa) should work equally well. Evaluate this claim. Answer guidance: False — although both use a DCT-like transform, the block structure, dimensionality (2D spatial blocks for images versus 1D time-domain frames for audio), and — critically — the perceptual model (human visual system for images, psychoacoustic model for audio) are completely different. Applying settings tuned for one modality to the other would produce poor compression and/or unacceptable quality loss.

FAQ

Why doesn't lossless compression achieve the same high ratios as lossy compression? Lossless compression can only exploit statistical redundancy already present in the data — repeated patterns, predictable structure — while guaranteeing exact reconstruction. Lossy compression additionally throws away information that's genuinely present in the signal but imperceptible to the intended receiver, which is a much larger pool of removable data.

Can you convert a lossy-compressed file back to lossless without quality loss? No — once information has been discarded during lossy compression, it cannot be recovered. Re-saving a JPEG as a PNG, for instance, preserves the already-degraded pixel values exactly, but does not restore any detail lost during the original JPEG compression.

Why do some codecs let you choose a "bitrate" or "quality" setting? That setting directly controls how aggressively the quantization stage rounds off less important coefficients — a higher bitrate/quality setting keeps more coefficients with finer precision (larger file, closer to original), while a lower setting discards more (smaller file, more visible/audible artifacts).

Is arithmetic coding always better than Huffman coding? Arithmetic coding can achieve compression closer to the theoretical entropy limit because it isn't restricted to assigning a whole number of bits per symbol, but it is more computationally complex and was historically encumbered by patents, which is part of why Huffman coding remains widely used despite being slightly less efficient.

Does compressing a signal always reduce its quality? Only lossy compression reduces quality (which may or may not be perceptible, depending on settings). Lossless compression, by definition, produces an output that decompresses back to bit-for-bit the same signal, with zero quality loss.

Quick Revision

• Signal compression reduces data size by exploiting redundancy or perceptually unimportant information
• Lossless compression (RLE, Huffman, arithmetic coding) allows exact reconstruction, used for medical/text/program data
• Lossy compression (DCT-based, wavelet-based) discards perceptually unimportant information for much higher compression ratios
• DCT concentrates a signal's energy into few low-frequency coefficients, enabling efficient quantization
• Compression pipeline: analysis, transformation, quantization (quantization is where data is actually lost)
• Psychoacoustic modeling removes audio content masked by human hearing limitations, key to MP3's efficiency
• Compression ratio and perceptual quality trade off against each other via the quantization step
• Repeated lossy compression passes cause generation loss without proportional size benefit
• JPEG uses DCT-based lossy compression on 8×8 image blocks; MP3 uses similar transform ideas plus psychoacoustic modeling
• Medical and diagnostic data generally require lossless compression by standard practice

Related Topics

Prerequisites: Fourier Transform, Digital Filters, Frequency-Domain Analysis

Related Topics: Image Processing, Speech Processing, Fourier Transform

Next Topics: Image Processing, Speech Processing