Why can an InfoNCE mutual-information bound saturate?
Suppose an InfoNCE objective uses one positive pair and N−1 negative candidates. Why can its standard mutual-information lower-bound interpretation become loose when the true mutual information is much larger than log N?
Sign in to answer questions and track your progress
Sign In