Information as negative log probability

Perfect compression linked an $n$-bit codeword to probability $2^{-n}$, so $n = -\log_2 p$ for that message . The logarithm turns tiny probabilities into additive bit counts: rare events need long descriptions, likely events short ones.

For one symbol with probability $p$, Shannon information is $I = -\log_2 p$ bits. Uniform four-way instructions at probability 1/4 each carry two bits; skew lowers the common symbol and raises rare ones. Independent symbols add information because probabilities multiply and logs convert products to sums .

Shannon information is not colloquial "news value"; it counts bits under a probabilistic model. Fractional bit lengths make sense when averaging over many draws from a skewed source .