Shannon's source coding theorem
Shannon's source coding theorem is a Data compression theory.
All 2 facts, and where they come from
Facts
2 facts · 0 confirmed by two sources · 0 cited · 2 from one source
Where this comes from
Papalocal keeps its own record. It started from these sources; what each one said is kept here as it was copied, so a change on their side never changes this page by itself. Tap one to read its version.
What Wikidata says · copied November 16, 2026 · revision 2561150640
- Described as
- Data compression theory
- What it is
- theorem
- Named after
- Claude Shannon
- Wikipedias with a page
- 18
Wikidata item Q2411312 (wikidata.org/wiki/Q2411312), public domain (CC0). Copied 2026-11-16.
What Wikipedia says · copied November 16, 2026 · revision 1385212739
In information theory, Shannon's source coding theorem (or noiseless coding theorem) establishes the statistical limits to possible data compression for data whose source is an independent identically distributed random variable, and the operational meaning of the Shannon entropy.
Named after Claude Shannon, the source coding theorem shows that, in the limit, as the length of a stream of independent and identically-distributed random variable (i.i.d.) data tends to infinity, it is impossible to compress such data such that the code rate (average number of bits per symbol) is less than the Shannon entropy of the source, without it being virtually certain that information will be lost. However, it is possible to get the code rate arbitrarily close to the Shannon entropy, with negligible probability of loss.
The source coding theorem for symbol codes places upper and lower bounds on the minimal possible expected length of codewords as a function of the entropy of the input word (which is viewed as a random variable) and of the size of the target alphabet.
Note that for data that exhibits more dependencies (whose source is not an i.i.d. random variable), the Kolmogorov complexity, which quantifies the minimal description length of an object, is more suitable to describe the limits of data compression. Shannon entropy takes into account only frequency regularities, while Kolmogorov complexity takes into account all algorithmic regularities, so in general the latter is smaller. On the other hand, if an object is generated by a random process in such a way that it has only frequency regularities, entropy is close to complexity with high probability (Shen et al. 2017).
From the Wikipedia article “Shannon's source coding theorem” (en.wikipedia.org/wiki/Shannon's_source_coding_theorem), by its contributors, under CC BY-SA 4.0 (creativecommons.org/licenses/by-sa/4.0/). Copied 2026-11-16; not changed.
Confirmed Two sources agree. Cited One source, and it names where it got it. One source One source, without a reference of its own. Sources disagree The sources disagree. We show the best-supported value and keep theirs below.
