Papalocal
Loading…
Papalocal · Things
Work

Shannon's source coding theorem

Shannon's source coding theorem is a Data compression theory.

All 2 facts, and where they come from

Facts

2 facts · 0 confirmed by two sources · 0 cited · 2 from one source

What it is
theorem One source
Named after
Claude Shannon One source

Where this comes from

Papalocal keeps its own record. It started from these sources; what each one said is kept here as it was copied, so a change on their side never changes this page by itself. Tap one to read its version.

What Wikidata says · copied November 16, 2026 · revision 2561150640
Described as
Data compression theory
What it is
theorem
Named after
Claude Shannon
Wikipedias with a page
18

Wikidata item Q2411312 (wikidata.org/wiki/Q2411312), public domain (CC0). Copied 2026-11-16.

What Wikipedia says · copied November 16, 2026 · revision 1385212739

In information theory, Shannon's source coding theorem (or noiseless coding theorem) establishes the statistical limits to possible data compression for data whose source is an independent identically distributed random variable, and the operational meaning of the Shannon entropy.
Named after Claude Shannon, the source coding theorem shows that, in the limit, as the length of a stream of independent and identically-distributed random variable (i.i.d.) data tends to infinity, it is impossible to compress such data such that the code rate (average number of bits per symbol) is less than the Shannon entropy of the source, without it being virtually certain that information will be lost. However, it is possible to get the code rate arbitrarily close to the Shannon entropy, with negligible probability of loss.
The source coding theorem for symbol codes places upper and lower bounds on the minimal possible expected length of codewords as a function of the entropy of the input word (which is viewed as a random variable) and of the size of the target alphabet.
Note that for data that exhibits more dependencies (whose source is not an i.i.d. random variable), the Kolmogorov complexity, which quantifies the minimal description length of an object, is more suitable to describe the limits of data compression. Shannon entropy takes into account only frequency regularities, while Kolmogorov complexity takes into account all algorithmic regularities, so in general the latter is smaller. On the other hand, if an object is generated by a random process in such a way that it has only frequency regularities, entropy is close to complexity with high probability (Shen et al. 2017).

From the Wikipedia article “Shannon's source coding theorem” (en.wikipedia.org/wiki/Shannon's_source_coding_theorem), by its contributors, under CC BY-SA 4.0 (creativecommons.org/licenses/by-sa/4.0/). Copied 2026-11-16; not changed.

Confirmed Two sources agree. Cited One source, and it names where it got it. One source One source, without a reference of its own. Sources disagree The sources disagree. We show the best-supported value and keep theirs below.

People connected 1

Known for it