Why Parameter Count Is Not Model Complexity: An Introduction to Effective Dimension

Why Parameter Count Is Not Model Complexity: An Introduction to Effective Dimension | Towards Data Science

Machine Learning

Why Parameter Count Is Not Model Complexity: An Introduction to Effective Dimension

What does effective dimension reveal about redundancy, generalization, and trainability that parameter count, loss, and accuracy do not?

Adrian Panasiewicz

October 23, 2026

15 min read

Prefer us on Google

Share

Effective dimension in machine learning explained through parameter redundancy, Fisher information, and the difference between global and local measures.

The infographic, titled “Effective dimension in machine learning: From parameter count to informational degrees of freedom,” contains four panels. 1. Why parameter count can be misleading Two models each have two parameters. In the first, f(x;θ)=θ1x+θ2f(x;θ)=θ1​x+θ2​, one parameter controls the slope and the other controls the intercept. Increasing each parameter separately produces different prediction curves. In the second, f(x;θ)=(θ1+θ2)xf(x;θ)=(θ1​+θ2​)x, both parameters control the same slope. Increasing either parameter by the same amount produces identical predictions, while equal and opposite changes cancel. The first model therefore has two independent parameter directions; the second has only one. 2. The Fisher information matrix Fisher information measures how strongly predicted probabilities respond to small parameter changes. Two Gaussian curves share the same mean but have different widths. The narrower distribution makes the mean easier to estimate. For a known standard deviation, the Fisher information about the mean per observation equals one divided by the variance. The matrix equation averages products of derivatives of the log probability. Diagonal entries describe sensitivity to individual parameters; off-diagonal entries describe how their effects align. 3. Global versus local effective dimension Two diagrams show the same two-dimensional parameter domain. In the global diagram, coloured evaluation points cover the domain. In the local diagram, only points within a circular neighbourhood of the trained parameters are highlighted; locations outside it are grey. Global effective dimension aggregates Fisher information across the parameter domain. Local effective dimension restricts the aggregation to the chosen neighborhood and depends on its radius and the trained solution. The dots represent illustrative Fisher-information evaluation locations, not observations or predictions. 4. What effective dimension tells us Four applications are listed: describing model capacity beyond parameter count, studying generalization to unseen data, examining sensitivity relevant to training, and identifying redundancy when designing models. The footer summarizes the central question: “How many parameter directions can the data actually distinguish?” It also points to a second part on computing effective dimension in practice.

This article was co-authored by Jacob Cybulski

Introduction

This article is the first in a two-part series on effective dimension. Here, we develop the intuition and theory behind the metric and introduce its main ingredients. In Part 2, we will show how to implement effective dimension calculation in machine-learning experiments.

The majority of machine-learning models are described by the number of their trainable parameters. While the number of parameters is a useful model property, it can be misleading.

A model may contain many parameters that do not create genuinely independent ways of influencing its output. Conversely, a compact model may use nearly every parameter in a distinct and meaningful way. Effective dimension is a way of quantifying that difference.

More parameters do not imply more capacity

Consider a simple example (see Figure 1). Imagine two models with parameters θ 1 \theta_1 θ 1 ​ and θ 2 \theta_2 θ 2 ​ .

Figure 1. Independent parameters create distinct functional directions, while linear combinations of parameters create redundancy. Image by authors.

In the first model, the parameters affect the output independently. Changing θ 1 \theta_1 θ 1 ​ produces an effect that cannot, in general, be reproduced by changing θ 2 \theta_2 θ 2 ​ . This model can vary in two independent parameter “directions”.

In the second model, the output depends only on their sum:

f

(

x

;

θ

1

,

2

)

=

+

.

f(x;\theta_1,\theta_2)=f(x;\theta_1+\theta_2).

​

Here, many parameter pairs produce exactly the same function. For instance, increasing θ 1 \theta_1 θ 1 ​ by c c c and decreasing θ 2 \theta_2 θ 2 ​ by c c c leaves the output unchanged. Although the model formally has two parameters, it has only one degree of freedom:

eff

\theta_{\text{eff}}=\theta_1+\theta_2.

This example illustrates the central idea: parameter count does not necessarily represent the model functional complexity.

What effective dimension measures

Effective dimension is a scale-dependent measure of information complexity [1]. Originally developed in statistical information theory to quantify the number of truly active parameters in a model, it transforms the coordinate system to eliminate redundant parameter combinations based on the size of available data. It has since been adopted in classical and quantum machine learning to precisely evaluate a model's expressivity, trainability, and ability to generalize to new data.

At a high level, effective dimension asks:

Across a chosen parameter region and data distribution, how many distinguishable and informational directions does this model have?

This question differs from the questions that could be answered by relying on the concepts of model accuracy, loss, or the raw parameter count.

Quantity

What it can tell us

What it can miss

Parameter count

The nominal size of the parameter vector

Redundant or functionally inactive parameters

Training loss

How well optimization fits the training data

Whether poor performance comes from optimization, model mismatch, or initialization

Test loss / accuracy

Likely (generalization) performance on the population data

If the model could be still improved by further training using a different optimization approach

Effective dimension

The number of distinguishable parameter directions that can be derived from data

It still depends on the data distribution, parameter region, and sample scale

Importantly, effective dimension is related to generalization. Under the assumptions of the framework, it can provide a complexity-dependent bound on generalization performance [2]. Intuitively, a model with fewer distinguishable parameter directions should be less effective at fitting arbitrary fluctuations in a finite dataset.

It can also help analyse trainability. Highly expressive models are not automatically easy to train: some parameter directions may have almost no detectable effect on the output distribution, while others may be strongly correlated. These effects can make optimization poorly conditioned or effectively flat [3]. Furthermore, such models may also be more susceptible to overfitting, where a model captures noise or idiosyncrasies in the training data rather than patterns that transfer to unseen data, leading to poor generalization.

Training and test losses are still essential diagnostics, but they depend on the optimization trajectory. A poor initialization, an unsuitable learning rate, or a weak optimizer can make a trainable architecture look untrainable in a particular run. Effective dimension instead characterizes the model’s parameter geometry relative to data, making it a complementary diagnostic rather than a replacement for empirical training curves [4].

Beyond these diagnostics, effective dimension matters because parameter count can misrepresent how complex a model really is. Every additional parameter has a cost: more circuit evaluations for gradients, more gates exposed to noise, and a larger search space to optimize. This cost is worthwhile only if the parameter adds a direction in which the model can actually change its behavior. Effective dimension estimates how many such directions exist, and so indicates whether a model's size is being used. When two models perform similarly, the one with fewer effective directions is usually the better choice, since it achieves the same result with less complexity. This is Occam's razor: prefer the simpler explanation when it accounts for the observations equally well. The aim is not to minimize complexity at all costs, but to find a model whose complexity suits the problem.

The key tool used in calculating the effective dimension is the Fisher information matrix.

Fisher information: the intuition

The Fisher information matrix describes how sensitive a distribution of model predictions is in the face of changes to its parameters. Put differently, it quantifies how much the observed data can tell us about the parameters of the model’s statistical profile.

Consider samples drawn from a Gaussian distribution with an unknown mean μ \mu μ and known standard deviation σ \sigma σ

∼

N

μ

σ

x\sim\mathcal{N}(\mu,\sigma^2).

Suppose we want to estimate μ \mu μ from n n n samples. Compare two cases (see Figure 2):

• A narrow Gaussian, with small σ \sigma σ ,

• A wide Gaussian, with large σ \sigma σ .

When σ \sigma σ is small, sampled points cluster tightly around the mean. Even a modest dataset gives strong evidence about where μ \mu μ lies.

When σ \sigma σ is large, samples are much more dispersed. The same number of observations gives a less precise estimate of μ \mu μ , because many nearby mean values remain plausible.

Figure 2. Estimating the mean is easier when the Gaussian distribution is narrow and harder when it is broad. Image by authors.

This is exactly what Fisher information captures. For a single observation from N ( μ , σ 2 ) , \mathcal{N}(\mu, \sigma^2), N ( μ , σ 2 ) , the Fisher information about μ \mu μ is

F(\mu)=\frac{1}{\sigma^2}.

for n n n independent observations, information adds:

F_n(\mu)=\frac{n}{\sigma^2}.

More data increases information linearly. More noise decreases it.

From likelihood to Fisher information

To formalize this intuition, fix an observed data point x x x and view the probability density p ( x ∣ θ ) p(x\mid\theta) p ( x ∣ θ ) as a function of the parameter θ . \theta. θ . This is the likelihood function .

A likelihood tells us which parameter values make the observed data more or less plausible. It is not a probability distribution over θ \theta θ unless we introduce a prior and normalize it, as in Bayesian inference.

Now consider two likelihood curves (see Figure 3):

• A flat likelihood: nearby parameter values explain the data almost equally well.

• A sharp likelihood: small changes in the parameter make the observation much less plausible.

A sharp likelihood means the observation is informative about the parameter. A flat likelihood means that it is not.

Figure 3. A sharply curved likelihood indicates that the observation strongly constrains the parameter. A flatter curve indicates weaker local information. Image by authors.

The local sensitivity of the likelihood is captured by differentiating its logarithm with respect to the parameter:

s

∂

log

⁡

p

∣

s(x;\theta)=\frac{\partial}{\partial\theta}\log p(x\mid\theta).

lo

g

This quantity is called the score function , which provides the gradient (the vector of partial derivatives).

We use the logarithm for a practical and mathematical reason. Products of probabilities become sums, and the derivative becomes a relative change:

\frac{\partial}{\partial\theta}\log p(x\mid\theta)=\frac{1}{p(x\mid\theta)}\frac{\partial}{\partial\theta}p(x\mid\theta).

The score can be positive or negative, depending on the direction in which the likelihood changes. Fisher information should measure magnitude rather than direction, so we square the score and average over all possible observations generated by the model:

E

[

|

]

F(\theta)=\mathbb{E}\left[\left( \frac{\partial}{\partial\theta}\log p(x\mid\theta) \right)^2 \middle| \ \theta \right].

For a continuous random variable, this expectation becomes

∫

d

F(\theta)=\int p(x\mid\theta) \left( \frac{\partial}{\partial\theta} \log p(x\mid\theta)\right)^2dx.

This is the Fisher information for a model with one parameter.

Gaussian example

Let us calculate the Fisher information for the mean μ \mu μ of a Gaussian distribution with known σ \sigma σ (see Figure 4).

Figure 4. From a Gaussian likelihood to the score and Fisher information. Image by authors.

The Gaussian distribution can be represented as:

π

exp

−

p(x \mid \mu) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp \left[ -\frac{(x-\mu)^2}{2\sigma^2} \right].

Taking the logarithm gives the log-likelihood:

\log p(x\mid\mu)= -\frac{1}{2}\log\left(2\pi\sigma^2\right)-\frac{(x-\mu)^2}{2\sigma^2}.

Differentiating with respect to μ \mu μ produces the score:

\frac{\partial}{\partial\mu}\log(x\mid\mu)=\frac{x-\mu}{\sigma^2}.

Squaring it gives:

4

\left( \frac{\partial}{\partial\mu}\log p(x\mid\mu)^2 \right)^2 = \frac{(x-\mu)^2}{\sigma^4}.

Finally, taking the expectation with respect to x ∼ N ( μ , σ 2 ) x\sim\mathcal{N}(\mu,\sigma^2) x ∼ N ( μ , σ 2 ) yields

F(\mu)=\mathbb{E}\left[ \frac{(x-\mu)^2}{\sigma^4} \right] = \frac{\mathbb{E}[(x-\mu)^2]}{\sigma^4}=\frac{\sigma^2}{\sigma^4}=\frac{1}{\sigma^2}.

[(

The result matches our intuition. A small variance makes the mean easier to estimate, so the Fisher information is large. A large variance makes the mean harder to estimate, so the Fisher information is small.

From a scalar to a matrix

Real machine-learning models have many parameters:

…

k

⊤

\boldsymbol{\theta}=\left( \theta_1,\theta_2,\dots ,\theta_k\right)^\top.

For a model with multiple parameters, the Fisher information is a matrix whose ( i , j ) (i,j) ( i , j ) entry is the expected product of the corresponding score functions:

i

j

\boldsymbol{F}_{(i,j)}(\boldsymbol{\theta})=\mathbb{E}\left[ \left( \frac{\partial}{\partial\theta_i} \log p(x;\boldsymbol{\theta})\right) \left( \frac{\partial}{\partial\theta_j} \log p(x;\boldsymbol{\theta}) \right) \middle| \ \boldsymbol{\theta}\right].

The diagonal entries measure sensitivity to individual parameters. The off-diagonal entries measure how parameter effects are “directionally” coupled, thus defining the geometry of the model parameters.

This coupling is crucial. If two parameters affect the output in nearly the same way, the Fisher information matrix becomes close to singular along one direction. That is the multivariate version of the earlier example in which only θ 1 + θ 2 \theta_1+\theta_2 θ 1 ​ + θ 2 ​ mattered.

Effective dimension uses this Fisher-information geometry to move beyond raw parameter count. It asks not how many parameters the model contains, but how many directions in parameter space the data can actually resolve.

With the Fisher information matrix in hand, we can now define and estimate effective dimension.

The intuitive picture of effective dimension

So far, we have motivated the central question behind effective dimension:

How many informationally distinct directions does a model have?

The Fisher information matrix tells us how sensitive a model’s predictions are to changes in its parameters. However, to turn that local sensitivity into a measure of model complexity, we first need to define what we mean by a model.

A statistical model can be written as a family of probability distributions indexed by its trainable parameters:

M

{

⋅

:

∈

}

\mathcal{M}_{\Theta}=\{ p(\cdot,\cdot,\boldsymbol{\theta}) : \boldsymbol{\theta}\in\Theta \}.

Here, θ \boldsymbol{\theta} θ is a parameter vector and Θ \Theta Θ is the parameter space. Depending on the machine-learning setting, p ( ⋅ , ⋅ , θ ) p(\cdot,\cdot,\boldsymbol{\theta}) p ( ⋅ , ⋅ , θ ) may describe a joint distribution p ( x , y ∣ θ ) p(x,y\mid\boldsymbol{\theta}) p ( x , y ∣ θ ) or, more commonly for supervised learning, a conditional predictive distribution p ( y ∣ x , θ ) p(y \mid x,\boldsymbol{\theta}) p ( y ∣ x , θ ) .

The important point is that every value of θ \boldsymbol{\theta} θ defines a potentially different predictive distribution. Together, all of these distributions form the model family M Θ \mathcal{M}_\Theta M Θ ​ .

There are two useful ways to study the effective dimension of this family:

• Global effective dimension , which considers the entire parameter space [1].

• Local effective dimension , which considers a neighborhood around a particular parameter configuration, usually one reached during or after training [2].

Local or global?

The global effective dimension asks how many informational parameter directions the model has on average across the full parameter space Θ \Theta Θ .

This makes it useful for architecture-level comparisons. For example, if we want to compare a variational quantum circuit with a classical neural network while controlling for other factors, global effective dimension can reveal whether one architecture has more distinguishable functional degrees of freedom than the other.

Because it averages over the entire parameter space, global effective dimension is not tied to one specific initialization or one optimization trajectory. This is valuable when making controlled, ceteris paribus comparisons between model classes.

However, global effective dimension should not be interpreted as completely independent of data. The Fisher information matrix depends on the model’s predictive distribution and, in supervised learning, typically on the distribution of input data. What global effective dimension removes is the dependence on a single trained solution or a particular region visited by one optimizer run.

That broad perspective is also its main limitation.

A model may contain highly informational regions of parameter space that are never reached in practice. Perhaps those regions are difficult to optimize, perhaps the initialization places training far away from them, or perhaps regularization prevents the model from entering them. In such cases, global effective dimension may describe the theoretical capacity of the model family without accurately reflecting the capacity used by a particular trained model.

This motivates the local effective dimension .

Instead of integrating over all of Θ \Theta Θ , local effective dimension restricts the analysis to a neighborhood around a reference parameter vector θ ∗ \boldsymbol{\theta}^* θ ∗ , often the parameters obtained after training:

B

ε

∗

∥

≤

\mathcal{B}_\varepsilon(\boldsymbol{\theta}^*)=\{\boldsymbol{\theta} : \lVert \boldsymbol{\theta} - \boldsymbol{\theta}^*\rVert \leq\varepsilon\}.

The local measure therefore asks a more practical question:

Near the solution that training actually found, how many parameter directions are distinguishable from the available data?

In this sense, local effective dimension can reflect the interaction between the architecture, the dataset, initialization, and optimization procedure. It is especially useful when we want to understand the complexity and generalization behavior of a trained model rather than the theoretical capacity of the entire architecture. Furthermore, local effective dimension can help with verifying if the trained model still has any unused learning capacity.

To calculate local effective dimension, we restrict the analysis to a small ε \varepsilon ε -ball around a specific parameter configuration. Although this is a local approximation, the Fisher information matrix captures the sensitivity and correlations of the model’s output distribution within that neighborhood. This makes it possible to investigate broader questions about the model’s local complexity and generalization behavior.

The following diagram illustrates this difference in the approach to effective dimension.

Figure 5. Global effective dimension averages over the entire parameter space, whereas local effective dimension considers a neighborhood around a chosen parameter configuration, such as a configuration θ ( i ) \boldsymbol{\theta}^{(i)} θ ( i ) , for each step labelled by i i i , encountered during training. Image by authors.

The effective-dimension formula

We can now introduce the global effective dimension. In the formulation used by Abbas et al. [1], it is defined as

γ

V

det

^

d_{n,\gamma}(\mathcal{M}_{\Theta}):= 2\frac{\log \left( \frac{1}{V_{\Theta}} \int_{\Theta} \sqrt{ \det\left( \boldsymbol{I}_k + \frac{\gamma n}{2\pi\log n} {\boldsymbol{\hat{F}}(\boldsymbol{\theta})} \right) }d\boldsymbol{\theta} \right)}{\log \left( \frac{\gamma n}{2\pi\log n} \right)}.

:=

l

o

There is a lot of notation here, but the basic idea is to look at the Fisher information throughout a chosen region of parameter space and combine what we find into a single number. Here, k k k is the number of trainable parameters, I k \boldsymbol{I}_k I k ​ is the k × k k\times k k × k identity matrix, n n n is the sample size, and γ \gamma γ is a constant in the construction. The integral runs over the parameter region Θ \Theta Θ ; dividing by its volume, V Θ = ∫ Θ d θ V_{\Theta}=\int_\Theta d\boldsymbol{\theta} V Θ ​ = ∫ Θ ​ d θ , turns the integral into an average.

The unfamiliar object is F ^ ( θ ) \boldsymbol{\hat{F}}(\boldsymbol{\theta}) F ^ ( θ ) . This is the Fisher information matrix F ( θ ) \boldsymbol{F}(\boldsymbol{\theta}) F ( θ ) , but with its overall scale adjusted. To see how, first take the trace of F \boldsymbol{F} F , which adds up its eigenvalues. Then average that trace over the parameter region. The normalized matrix for the ( i , j ) (i,j) ( i , j ) element becomes:

Tr

ϑ

\boldsymbol{\hat{F}}_{(i,j)}:=k\cdot \frac{V_\Theta}{\int_\Theta \operatorname{Tr}\left( \boldsymbol{F}(\boldsymbol{\vartheta})\right)d\boldsymbol{\vartheta}}\boldsymbol{F}_{(i,j)}(\boldsymbol{\theta}).

The denominator is computed once using the whole region Θ \Theta Θ . Every Fisher matrix in that region is then multiplied by the same factor. Why choose this particular factor? It makes the average trace of the normalized Fisher matrix equal to k k k :

\frac{1}{V_\Theta}\int_\Theta \operatorname{Tr} \left( \boldsymbol{\hat{F}}(\boldsymbol{\theta}) \right)d\boldsymbol{\theta}=k.

In other words, the average Fisher eigenvalue is set to one. This gives the matrices a common reference scale, but it does not make every eigenvalue equal to one. For example, in a two-parameter model, eigenvalues of 1 1 1 and 1 1 1 have the same trace as eigenvalues of 1.9 1.9 1.9 and 0.1 0.1 0.1 . The normalization treats their total scale equally while retaining the important difference between two similarly informative directions and one dominant direction.

For local effective dimension, the same expression is used, but Θ \Theta Θ is replaced by a local ball B ε ( θ ∗ ) \mathcal{B}_\varepsilon(\boldsymbol{\theta}^*) B ε ​ ( θ ∗ ) , and V Θ V_\Theta V Θ ​ is replaced by the volume V ε V_\varepsilon V ε ​ of that ball.

That difference enters the effective dimension through the determinant:

\det \left( \boldsymbol{I}_K + \frac{\gamma n}{2n \log n} \boldsymbol{\hat{F}}(\boldsymbol{\theta}) \right).

This is where the Fisher-information geometry enters the definition.

Reading the Fisher spectrum

To see what this determinant does, define the scale factor

κ

\kappa_{n,\gamma}=\frac{\gamma n}{2\pi \log n}.

This factor is the same at every parameter value θ \boldsymbol{\theta} θ and is used to achieve desirable asymptotic properties of effective dimension.

Now diagonalize the normalized Fisher information matrix:

U

diag

λ

\boldsymbol{\hat{F}}(\boldsymbol{\theta})= \boldsymbol{U} \operatorname{diag} \left( \lambda_1(\boldsymbol{\theta}),\dots,\lambda_k(\boldsymbol{\theta}) \right) \boldsymbol{U}^\top.

The eigenvectors in U \boldsymbol{U} U identify independent directions in parameter space. The corresponding eigenvalues λ i ( θ ) \lambda_i(\boldsymbol{\theta}) λ i ​ ( θ ) quantify how strongly changes along those directions affect the model’s output distribution.

Because determinants are invariant under rotations, we obtain

∏

\det \left( \boldsymbol{I}_k + \kappa_{n,\gamma} \boldsymbol{\hat{F}}(\boldsymbol{\theta})\right) = \prod_{i=1}^k \left( 1+\kappa_{n,\gamma}\lambda_i(\boldsymbol{\theta}) \right).

The square root in the effective-dimension formula then gives

\sqrt{\det \left( \boldsymbol{I}_k + \kappa_{n,\gamma} \boldsymbol{\hat{F}}(\boldsymbol{\theta})\right)} = \prod_{i=1}^k \sqrt{ 1+\kappa_{n,\gamma}\lambda_i(\boldsymbol{\theta})}.

At a fixed point θ \boldsymbol{\theta} θ , taking the logarithm turns this product into a sum:

∑

\log \sqrt{\det \left( \boldsymbol{I}_k + \kappa_{n,\gamma} \boldsymbol{\hat{F}}(\boldsymbol{\theta})\right)} = \frac{1}{2}\sum_{i=1}^k \log \left( 1+\kappa_{n,\gamma}\lambda_i(\boldsymbol{\theta})\right).

This is the key expression for understanding effective dimension. Each Fisher eigen-direction contributes according to

\log (1+\kappa_{n,\gamma}\lambda_i).

The contribution depends on whether the eigenvalue is large or small relative to the resolution scale 1 / κ n , γ 1/\kappa_{n,\gamma} 1/ κ n , γ ​ .

Nearly invisible directions

First, consider a direction with

≪

1.

\kappa_{n,\gamma}\lambda_i \ll 1.

Using the first-order Taylor approximation,

≈

\log (1+\kappa_{n,\gamma}\lambda_i) \approx\kappa_{n,\gamma}\lambda_i

This contribution is small.

Such an eigen-direction corresponds to a parameter combination that barely changes the model’s predictive distribution at the chosen resolution. The data cannot reliably distinguish movement in that direction, so it should contribute little to the model’s effective complexity.

Informational directions

Now consider the opposite regime:

≫

\kappa_{n,\gamma}\lambda_i \gg1.

Then the 1 1 1 becomes negligible:

\log \left( 1 + \kappa_{n,\gamma}\lambda_i \right) \approx \log \left( \kappa_{n,\gamma}\lambda_i \right).

Using the logarithm rule gives:

\log (\kappa_{n,\gamma}\lambda_i)=\log (\kappa_{n,\gamma}) + \log(\lambda_i).

In the effective-dimension definition, the numerator is normalized by log ⁡ ( κ n , γ ) \log(\kappa_{n,\gamma}) lo g ( κ n , γ ​ ) . In the large- n n n regime, the scale-dependent term dominates:

\frac{\log \kappa_{n,\gamma}}{\log \kappa_{n,\gamma}} = 1.

The remaining eigenvalue-dependent correction,

\frac{\log \lambda_i}{\log \kappa_{n,\gamma}},

becomes comparatively small when κ n , γ \kappa_{n,\gamma} κ n , γ ​ is large and the eigenvalue remains finite.

This is why effective dimension behaves like a soft, resolution-dependent count of informational parameter directions.

It does not simply ask, “How many parameters are in the model?”

Instead, it asks:

At the resolution set by n n n , how many parameter directions can the data distinguish?

In the large- n n n limit, the effective dimension approaches the maximal rank of the Fisher information matrix under the assumptions of the construction.

Why the scale matters

The parameter n n n is often described as the dataset size. That is broadly correct, because the effective-dimension framework connects n n n to finite-sample generalization. But it is also helpful to interpret n n n as a resolution scale .

A larger dataset can distinguish smaller differences between two nearby predictive distributions. As n n n increases, directions that were previously too weak to observe may become informative.

The geometric analogy is useful here.

Imagine examining an object at different magnifications. At a coarse scale, it may look like a simple two-dimensional surface. At a finer scale, you may discover that the surface is actually a narrow curve winding through space. At an even finer scale, that curve may reveal small oscillations or additional structure.

This is why it is good practice to evaluate effective dimension across a range of n n n values rather than reporting only one number. In a real application, the most relevant values are those near the size of the datasets on which the model will actually be trained.

Figure 6. Model family can reveal different effective structure at different statistical resolutions. Adapted from [1].

Final words

Effective dimension is a complexity measure based on the geometry of the model’s statistical profile. It goes beyond raw parameter count by identifying how many parameter directions are actually distinguishable from data.

Its value comes from connecting several ideas that are often studied separately:

• Model capacity and expressivity.

• Parameter redundancy and identifiability.

• Fisher-information geometry.

• Finite-data generalization.

• Trainability and optimization landscapes.

Under the assumptions of the underlying theory, effective dimension can be used to derive bounds on generalization error. It should not be read as a direct prediction of test accuracy, but it provides a principled complexity quantity that can complement empirical training curves and validation results.

Importantly, effective dimension quantifies learning capacity through parameter space, a perspective that applies equally to classical and parameterized quantum models. In quantum machine learning, alternative frameworks characterize capacity directly over the Hilbert space of quantum states, which this metric does not explicitly address.

In this article, we introduced the motivation for effective dimension and the Fisher-information geometry behind it. We then saw how the Fisher spectrum determines which parameter directions contribute to the complexity of a model at a chosen data scale.

The next article in this series will move from theory to practice: how to implement the Fisher information matrix and effective dimension calculation for a machine-learning model.

Bibliography

[1] A. Abbas, D. Sutter, C. Zoufal, A. Lucchi, A. Figalli, and S. Woerner, “The power of quantum neural networks,” Nat. Comput. Sci. , vol. 1, no. 6, pp. 403–409, Jun. 2021, doi: 10.1038/s43588-021-00084-1.

[2] A. Abbas, D. Sutter, A. Figalli, and S. Woerner, “Effective dimension of machine learning models,” Dec. 09, 2021, arXiv : arXiv:2112.04807. doi: 10.48550/arXiv.2112.04807.

[3] E. Grant, L. Wossnig, M. Ostaszewski, and M. Benedetti, “An initialization strategy for addressing barren plateaus in parametrized quantum circuits,” Quantum , vol. 3, p. 214, Dec. 2019, doi: 10.22331/q-2019-12-09-214.

[4] S. Amari and H. Nagaoka, Methods of Information Geometry , vol. 191. in Translations of Mathematical Monographs, vol. 191. Providence, Rhode Island: American Mathematical Society, 2007. doi: 10.1090/mmono/191.