Nats, bits, and ln 2

Information has two units, and $\ln 2$ is the exchange rate. Entropy in base 2 is denominated in bits; in base $e$, in nats; and

$$1 \text{ bit} \;=\; \ln 2 \text{ nats} \;=\; 0.6931\ldots \text{ nats}.$$

The bookkeeping matters daily. Deep-learning frameworks compute cross-entropy with natural logarithms, so training loss arrives in nats per token; language-modeling papers report bits per character or byte. Divide by $\ln 2$ to convert: a model at $0.90$ nats/token is at $1.30$ bits/token. Perplexity ties the two together as $e^{H_{\text{nats}}} = 2^{H_{\text{bits}}}$. And the floor under all of it is Shannon’s source-coding theorem: no code beats the entropy in bits per symbol on average — a fair coin costs one bit, which is to say $\ln 2$ nats, per toss.1

1

Shannon (1948). The same conversion applies to KL divergence, mutual information, and every other quantity built from logarithms of probabilities.

The number itself

The constant is the alternating harmonic series,

$$\ln 2 \;=\; 1-\frac12+\frac13-\frac14+\cdots,$$

convergent but only conditionally — Riemann’s rearrangement theorem applies, and concretely: take one positive term per two negatives and the sum halves,

$$1-\frac12-\frac14+\frac13-\frac16-\frac18+\cdots \;=\; \frac{\ln 2}{2}.$$

The same series reordered, a different number: a fact worth carrying around as a portable warning about infinite sums.

For actual digits, sum the geometric-flavored series

$$\ln 2 \;=\; \sum_{k\ge1}\frac{1}{k\,2^k},$$

which delivers roughly one binary digit per term — pleasingly, the series for the price of a bit converges at about a bit per step.