$Power Laws

Relationships of the form y = ax^k, which appear as straight lines on log-log axes.

Aug 1, 2026

A power law relates two quantities by a fixed exponent:

For the logarithms below, $x>0$ and $a>0$. Multiplying $x$ by a factor $c$ multiplies $y$ by $c^k$. That multiplier rule is what the exponent says.

Taking logarithms of both sides turns it into a straight line:

So an exact power law plots as a straight line on log-log axes, with the exponent $k$ as the slope. A straight-looking plot is evidence to investigate, not proof: other functions can appear straight over a narrow range, and noise can hide curvature.

Reading the exponent

The exponent says how a multiplicative change in the input changes the output. A small-magnitude exponent means a large input multiplier may be needed for a substantial output change.

With $y\propto x^{-0.34}$, halving $y$ requires

2^(1/0.34) ≈ 7.7x    the input

Nearly eight times the resource for one halving. Under the fitted relationship, progress is expensive in a way that can be quantified within the supported range.

Why they are useful

If the relationship remains stable, a fitted exponent can support extrapolation beyond the measured runs. This is useful for planning expensive training, but uncertainty usually grows with distance from the data.

Extrapolation assumes the model family, data distribution, training recipe, and functional form remain relevant outside the fitted range. Each is an assumption.

Where it shows up

Reducible-loss terms against parameters and data, compute-optimal allocation, and any quantity quoted as “scales with the square of” something.