Why can an InfoNCE mutual-information bound saturate?

Question: Suppose an InfoNCE objective uses one positive pair and N−1 negative candidates. Why can its standard mutual-information lower-bound interpretation become loose when the true mutual information is much larger than log N? A) InfoNCE becomes an upper bound once the critic separates every positive pair B) The lower bound becomes independent of the critic after one negative example is classified correctly C) For fixed N, the conventional bound cannot certify a value larger than approximately log N D) The mutual information of continuous representations is itself bounded above by log N Correct: C Explanation: A conventional form of the InfoNCE result is I(X;Y) ≥ log N − L_InfoNCE. Because the loss is non-negative in its standard formulation, the value certified by this expression is capped around log N. This limits the bound, not necessarily the true mutual information. Topic: advanced ML / contrastive learning / InfoNCE