The ELBO Went Up And The Model Got Worse

Two checkpoints of the same VAE on the same data, reported per example in nats. The ELBO is the reconstruction term minus the KL term, $\mathcal{L} = \mathbb{E}$ .

checkpoint	reconstruction term	KL term
epoch 5	-102.0	18.4
epoch 40	-96.5	0.2

Which checkpoint has the higher ELBO, and what is the tightest statement you can make about the latent at epoch 40?

Epoch 40, at $-96.7$ ; the KL constrains the encoder only, so nothing follows about $I(x; z)$ without a separate mutual-information estimator

Epoch 5, at $-83.6$ : the KL term is added to the reconstruction term, and a larger KL means a more expressive latent

Epoch 40, at $-96.7$ ; the average KL upper-bounds the mutual information $I(x; z)$ , so the latent carries at most $0.2$ nats about the input

Epoch 40, at $-96.7$ ; the near-zero KL shows the encoder now matches the prior, which is what the regularizer is asking for, so this is the checkpoint to ship