10 / 10 KV Cache & Decoding

EAGLE Eats Medusa's Lunch

Both EAGLE and Medusa bolt lightweight prediction modules onto a frozen target LLM to draft several future tokens for tree-attention-verified speculative decoding, yet EAGLE reaches roughly 85-95% acceptance rates versus Medusa's roughly 60-80% at similar draft budgets. What's the key architectural reason EAGLE's drafts are accepted more often?

A

EAGLE quantizes the target model's weights during drafting to run more candidate tokens per cycle

B

EAGLE verifies drafts with greedy decoding only, while Medusa's rejection sampling is strictly more conservative and rejects more tokens

C

EAGLE drafts autoregressively over the target model's own hidden-state features, feeding each sampled draft token's embedding back in before predicting the next, so later draft tokens are conditioned on earlier ones; Medusa's heads predict all future positions independently and in parallel from a single hidden state

D

EAGLE trains an entirely separate small transformer from scratch on a much larger corpus than Medusa's heads ever see

E

Medusa cannot use tree attention for verification at all, while EAGLE introduced tree-structured verification from the start