Who's Looking at Whom?
In the decoder of an encoder-decoder transformer (e.g., the original machine translation model), the cross-attention sublayer computes attention where the queries come from the decoder's own sequence, but the keys and values come from where?
Sign in to answer questions and track your progress
Sign In