From Spread MLE to Implicit MLE

1. Implicit Generative Models and IMLE

Implicit generative models are easy to sample from but often do not have a tractable likelihood. Let

Z\sim p(z), \qquad Y=G_\theta(Z),

and let $q_\theta$ denote the resulting distribution of $Y$ . We can generate samples from $q_\theta$ , but evaluating $q_\theta(x)$ may be impossible. The problem is even more severe when $q_\theta$ is supported on a low-dimensional manifold, because an ordinary density may not exist at all.

Implicit Maximum Likelihood Estimation (IMLE) trains such a generator without ever requiring access to its density. Given a data distribution $p_{\mathrm{data}}$ , the population IMLE objective can be written as

\mathcal L_{\mathrm{IMLE}}(\theta) = \mathbb E_{X\sim p_{\mathrm{data}}} \mathbb E_{Y_{1:m}\sim q_\theta^m} \left[ \min_{1\le j\le m} \|X-Y_j\|^2 \right].

For an empirical dataset $x_1,\ldots,x_n$ , this becomes

\widehat{\mathcal L}_{\mathrm{IMLE}}(\theta) = \mathbb E_{Y_{1:m}\sim q_\theta^m} \left[ \frac1n \sum_{i=1}^n \min_{1\le j\le m} \|x_i-Y_j\|^2 \right].

In practice, IMLE repeatedly performs three steps:

Sample a pool of latent codes $z_1,\ldots,z_m$ .
Generate candidates $Y_j=G_\theta(z_j)$ .
For each data point, find its nearest generated candidate and update the generator to reduce the matched distance.

The matching direction is important: every data point selects a generated sample. This encourages the generator to cover all observed modes rather than only improving samples in regions it already generates well.

The expectation over $Y_{1:m}$ averages over the random candidate pools produced by the model. The empirical average over the data can equivalently be viewed as an expectation under the empirical data distribution.

This objective does not immediately resemble a likelihood-based objective. The following sections show how it can be derived from a finite-sample approximation to spread maximum likelihood.

2. From an Ill-Defined KL to the Spread KL Divergence

Consider a data distribution $p_{\mathrm{data}}$ and an implicit model distribution $q_\theta$ . A natural objective is the forward KL divergence, which, whenever both distributions admit densities with respect to the same base measure, can be written as

\operatorname{KL} \left( p_{\mathrm{data}} \|q_\theta \right) = \mathbb E_{X\sim p_{\mathrm{data}}} \left[ \log p_{\mathrm{data}}(X) - \log q_\theta(X) \right].

Since the first term does not depend on $\theta$ , minimizing the forward KL is equivalent to maximum likelihood:

\arg\min_\theta \operatorname{KL} \left( p_{\mathrm{data}} \|q_\theta \right) = \arg\max_\theta \mathbb E_{X\sim p_{\mathrm{data}}} \left[ \log q_\theta(X) \right].

That premise fails for the implicit generator above. Mapping a low-dimensional latent into a higher-dimensional observation space leaves $q_\theta$ supported on a lower-dimensional manifold, with no ordinary density with respect to the ambient Lebesgue measure. Worse, even when $p_{\mathrm{data}}$ and $q_\theta$ both concentrate on low-dimensional manifolds, their supports may fail to overlap exactly, in which case $\operatorname{KL}(p_{\mathrm{data}}\|q_\theta)$ is infinite or not meaningfully defined through ordinary density ratios.

Spreading the Distributions

Spread divergence addresses this by applying the same smoothing kernel to both distributions. Using Gaussian spreading,

\widetilde X = X+\sigma\epsilon, \qquad \widetilde Y = Y+\sigma\epsilon', \qquad \epsilon,\epsilon'\sim\mathcal N(0,I),

the corresponding spread distributions are the convolutions

p_\sigma = p_{\mathrm{data}}*\phi_\sigma, \qquad q_{\theta,\sigma} = q_\theta*\phi_\sigma, \qquad \phi_\sigma(u) = \mathcal N(u;0,\sigma^2I).

Gaussian convolution turns the original, possibly singular distributions into smooth distributions with densities over the full observation space. The spread KL divergence

\operatorname{KL} \left( p_\sigma \|q_{\theta,\sigma} \right)

is therefore well defined, whatever the supports of $p_{\mathrm{data}}$ and $q_\theta$ happen to be.

Why Spreading Loses Nothing

Well-definedness alone would not be enough. Smoothing could in principle blur away exactly the differences we are trying to detect, leaving an objective that is finite but no longer informative about the original distributions. This does not happen for Gaussian spreading: the spread KL is a genuine divergence on the original distributions,

\boxed{ \operatorname{KL} \left( p_\sigma \|q_{\theta,\sigma} \right) = 0 \quad\Longleftrightarrow\quad p_{\mathrm{data}} = q_\theta . }

This holds for every fixed $\sigma>0$ , and follows from the fact that Gaussian convolution is injective, so that $p_\sigma=q_{\theta,\sigma}$ already forces $p_{\mathrm{data}}=q_\theta$ . See the spread divergence paper for the proof and for the general conditions a noise distribution must satisfy for its spread divergence to be a valid divergence.

So nothing is lost by spreading: driving the spread KL to zero matches the two distributions we actually care about, even though $\operatorname{KL}(p_{\mathrm{data}}\|q_\theta)$ may itself be ill-defined. What remains is to turn this divergence into something we can optimize.

3. Spread Maximum Likelihood

Expanding the spread KL exactly as before,

\operatorname{KL} \left( p_\sigma \|q_{\theta,\sigma} \right) = \mathbb E_{\widetilde X\sim p_\sigma} \left[ \log p_\sigma(\widetilde X) - \log q_{\theta,\sigma}(\widetilde X) \right],

the first term again does not depend on $\theta$ . Minimizing the spread KL is therefore maximum likelihood under the spread model, which gives the Spread Maximum Likelihood objective

\boxed{ \mathcal L_{\mathrm{spread}}(\theta) = \mathbb E_{\widetilde X\sim p_\sigma} \left[ \log q_{\theta,\sigma}(\widetilde X) \right]. }

Because the two objectives differ only by a $\theta$ -independent constant, the identifiability of the spread KL carries over directly. Assuming the data distribution is realizable,

\arg\max_\theta \mathcal L_{\mathrm{spread}}(\theta) = \left\{ \theta: q_\theta = p_{\mathrm{data}} \right\} .

Spread MLE is thus not a compromise objective that merely happens to be computable. Its optimum recovers the unspread data distribution, so it estimates the same thing ordinary MLE would have estimated had the latter been well defined.

For an implicit model, the spread density is a Gaussian mixture over model samples,

q_{\theta,\sigma}(x) = \mathbb E_{Y\sim q_\theta} \left[ \mathcal N(x;Y,\sigma^2I) \right],

which is a smooth density even when the original model is implicit or singular. Substituting it in,

\mathcal L_{\mathrm{spread}}(\theta) = \mathbb E_{\widetilde X} \left[ \log \mathbb E_{Y\sim q_\theta} \exp\left( -\frac{\|\widetilde X-Y\|^2}{2\sigma^2} \right) \right] +C.

This is a valid likelihood objective, but the inner expectation is generally intractable.

4. A Finite-Sample Approximation

Draw $m$ samples from the implicit model:

Y_1,\ldots,Y_m\sim q_\theta.

The spread density can be approximated by a finite Gaussian mixture:

\widehat q_{\theta,\sigma,m}(x) = \frac1m \sum_{j=1}^m \mathcal N(x;Y_j,\sigma^2I).

This gives the multi-sample objective

\mathcal L_{m,\sigma}(\theta) = \mathbb E_{\widetilde X,Y_{1:m}} \left[ \log \frac1m \sum_{j=1}^m \exp\left( -\frac{\|\widetilde X-Y_j\|^2}{2\sigma^2} \right) \right] +C.

Because the finite mixture is an unbiased estimator of the spread density, Jensen's inequality gives

\mathcal L_{m,\sigma}(\theta) \le \mathcal L_{\mathrm{spread}}(\theta).

This is the same general construction used in multi-sample likelihood bounds such as the importance weighted autoencoder: a finite Monte Carlo average is placed inside the logarithm, and the expectation of the resulting log gives a lower bound on the exact log marginal likelihood.

As $m\to\infty$ , the finite mixture approaches the exact spread density under suitable conditions.

5. The Small-Noise Limit

Consider the core log-sum-exp term:

-2\sigma^2 \log \sum_{j=1}^m \exp\left( -\frac{\|\widetilde X-Y_j\|^2}{2\sigma^2} \right).

Fix $X$ , $\epsilon$ and $Y_{1:m}$ for a moment. As $\sigma\to0$ , the largest exponential term dominates, so that pointwise

-2\sigma^2 \log \sum_{j=1}^m \exp\left( -\frac{\|\widetilde X-Y_j\|^2}{2\sigma^2} \right) \longrightarrow \min_j \|X-Y_j\|^2.

The same limit performs two operations at once:

The data noise disappears because

\widetilde X=X+\sigma\epsilon\to X.

The soft Gaussian responsibilities become a hard nearest-neighbour assignment.

Both the $\frac1m$ inside the logarithm and the additive constant $C$ are killed by the $\sigma^2$ factor, so rescaling $\mathcal L_{m,\sigma}$ by $-2\sigma^2$ yields

-2\sigma^2\,\mathcal L_{m,\sigma}(\theta) \;\longrightarrow\; \mathbb E_{X\sim p_{\mathrm{data}}} \mathbb E_{Y_{1:m}\sim q_\theta^m} \left[ \min_j \|X-Y_j\|^2 \right] = \mathcal L_{\mathrm{IMLE}}(\theta),

which is exactly the population IMLE objective of Section 1. This last step exchanges the limit with the expectation, which is not automatic: assuming $p_{\mathrm{data}}$ and $q_\theta$ have finite second moments, and under suitable uniform-integrability conditions, the limit may be passed through the expectation. Bounded $G_\theta$ , or a compact parameter set with uniformly bounded second moments, is enough in practice.

Replacing $p_{\mathrm{data}}$ by the empirical distribution of $x_1,\ldots,x_n$ gives $\widehat{\mathcal L}_{\mathrm{IMLE}}(\theta)$ ,

\mathbb E_{Y_{1:m}\sim q_\theta^m} \left[ \frac1n \sum_{i=1}^n \min_j \|x_i-Y_j\|^2 \right].

Note that the rescaling factor $-2\sigma^2$ is negative, which is what reconciles the two directions of optimisation. Therefore, at the objective-function level, IMLE is the small-noise limit of the rescaled finite-sample spread likelihood. Each training example is matched to its nearest generated sample, and the generator is trained to reduce those distances.

6. The Connection

The relationship can be summarized as

\boxed{ \text{Spread MLE} \rightarrow \text{finite Monte Carlo Gaussian mixture} \rightarrow \text{small-noise hard assignment} \rightarrow \text{IMLE}. }

More explicitly, working throughout with the $-2\sigma^2$ rescaling of the previous section,

\begin{aligned} & -2\sigma^2 \log \mathbb E_{Y\sim q_\theta} \exp\left( -\frac{\|X+\sigma\epsilon-Y\|^2}{2\sigma^2} \right) \\ &\quad\approx -2\sigma^2 \log \frac1m \sum_{j=1}^m \exp\left( -\frac{\|X+\sigma\epsilon-Y_j\|^2}{2\sigma^2} \right) \\ &\quad\xrightarrow[\sigma\to0]{} \min_j \|X-Y_j\|^2 . \end{aligned}

The first step is a finite Monte Carlo approximation. The second is an exact pointwise limit; passing the limit through the expectation requires standard integrability conditions. The $\frac1m$ inside the logarithm contributes $2\sigma^2\log m$ , which vanishes as $\sigma\to0$ and so leaves no trace in the limit.

7. What Is Lost Along the Way?

The connection is useful, but it is not an exact equivalence.

First, the finite-sample objective is a lower bound on spread likelihood, rather than the exact likelihood itself. For fixed $m$ , it generally has a Jensen gap:

\mathbb E \left[ \log \widehat q_{\theta,\sigma,m}(x) \right] \le \log q_{\theta,\sigma}(x).

Second, the small-noise limit replaces the full Gaussian mixture with only its nearest component. All soft responsibilities are discarded.

Spread MLE therefore remains a probabilistically well-defined likelihood method, while IMLE is its finite-sample, hard-assignment limit.

This perspective explains several otherwise surprising features of IMLE:

Generated samples act as mixture centers.
Squared Euclidean distance appears naturally under Gaussian spreading.
Each data point selects its nearest generated sample.
An expectation over random generated sample sets is required.
The resulting objective encourages mode coverage.

The cleanest interpretation is:

IMLE can be viewed as the zero-noise hard-assignment limit of a finite-sample approximation to spread maximum likelihood.

References

Spread Divergence — Mingtian Zhang, Peter Hayes, Thomas Bird, Raza Habib, David Barber. ICML 2020.
Implicit Maximum Likelihood Estimation — Ke Li, Jitendra Malik. arXiv preprint, 2018.
Importance Weighted Autoencoders — Yuri Burda, Roger Grosse, Ruslan Salakhutdinov. ICLR 2016.